An analytical short- and long-term memory model of presynaptic plasticity

A mathematical model, called the Learning Gate Model (LGM), that describes phenomena responsible for biological synaptic plasticity, is presented. The functionality of the model are mainly based on the work of Kandel and colleagues on the most elementary forms of learning observed in the Aplysia Californica marine mollusc. In particular, emphasis is placed on the double temporal dynamics of synaptic plasticity and the temporal specificity of classical conditioning. By properly modeling the effect of the binding of Ca⁺⁺ ions to the serotonin-sensitive adenylate cyclase enzyme, it is shown how a positively accelerated learning curve can be obtained for sensitization and classical conditioning. Phenomena of spontaneous recovery and second-order conditioning are reproduced through simulations. Mathematical analyses of the temporal trace of conditioned stimulus and of the Short-Term Memory steady state are also given.     

Towards a unified model of pavlovian conditioning: short review of trace conditioning models

There are three basic paradigms of classical conditioning: delay, trace and context conditioning where presentation of a conditioned stimulus (CS) or a context typically predicts an unconditioned stimulus (US). In delay conditioning CS and US normally coterminate, whereas in trace conditioning an interval of time exists between CS termination and US onset. The modeling of trace conditioning is a rather difficult computational problem and is a challenge to the behavior and connectionist approaches mainly due to a time gap between CS and US. To account for trace conditioning, Pavlov (Conditioned reflexes: an investigation of the physiological activity of the cerebral cortex, Oxford University Press, London, 1927) postulated the existence of a stimulus “trace” in the nervous system. Meanwhile, there exist many other options for solving this association problem. There are several excellent reviews of computational models of classical conditioning but none has thus far been devoted to trace conditioning. Eight representative models of trace conditioning aimed at building a prospective model are being reviewed below in a brief form. As a result, one of them, comprising the most important features of its predecessors, can be suggested as a real candidate for a unified model of trace conditioning.     

Conditioning, attention, and the CS-US interval. A theoretical statement of relations

A revised version of the Rescorla-Wagner (1972) mathematical model is presented. A metatheoretical assumption of an attentional process, the added revision, is conceived as an independent alpha-salience growth factor determining both rate of association and performance. Conditioned stimulus-unconditioned stimulus (CS-US) correlation, and CS-US interval (two primary conditioning parameters) are incorporated in the mathematical model as alpha-salience growth rate and as alpha-salience and association asymptote factors, respectively. In this manner, the long-standing issue of necessary and sufficient factors in classical conditioning is resolved. An empirical assessment of the model's parameters has been included.     

Ascertainment: An Overwiew of the Classical Segregation Analysis Model for Independent Sibships

Several different methodologies for parameter estimation under various ascertainment sampling schemes have been proposed in the past. In this article, some of the methodologies that have been proposed for independent sibships under the classical segregation analysis model are synthesized, and the general likelihoods derived for single, multiple and complete ascertainment. The issue of incorporating the sibship size distribution into the analysis is addressed, and the effect of conditioning the likelihood on the observed sibship sizes is discussed. It is shown that when the number of probands in a sibship is not specified, the corresponding likelihood can be used for a broader class of ascertainment schemes than is subsumed by the classical model.     

Classical and quantum dynamics on p-adic trees of ideas

We propose mathematical models of information processes of unconscious and conscious thinking (based on p-adic number representation of mental spaces). Unconscious thinking is described by classical cognitive mechanics (which generalizes Newton's mechanics). Conscious thinking is described by quantum cognitive mechanics (which generalizes the pilot wave model of quantum mechanics). The information state and motivation of a conscious cognitive system evolve under the action of classical information forces and a new quantum information force, namely, conscious force. Our model might provide mathematical foundations for some cognitive and psychological phenomena: collective conscious behavior, connection between physiological and mental processes in a biological organism, Freud's psychoanalysis, hypnotism, homeopathy. It may be used as the basis of a model of conscious evolution of life.     

Modelling Individual Differences in the Form of Pavlovian Conditioned Approach Responses: A Dual Learning Systems Approach with Factored Representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.     

Likelihood based observability analysis and confidence intervals for predictions of dynamic models

ABSTRACT: BACKGROUND: Predicting a system's behavior based on a mathematical model is a primary task in Systems Biology. If the model parameters are estimated from experimental data, the parameter uncertainty has to be translated into confidence intervals for model predictions. For dynamic models of biochemical networks, the nonlinearity in combination with the large number of parameters hampers the calculation of prediction confidence intervals and renders classical approaches as hardly feasible. RESULTS: In this article reliable confidence intervals are calculated based on the prediction profile likelihood. Such prediction confidence intervals of the dynamic states can be utilized for a data-based observability analysis. The method is also applicable if there are non-identifiable parameters yielding to some insufficiently specified modelpredictions that can be interpreted as non-observability. Moreover, a validation profile likelihood is introduced that should be applied when noisy validation experiments are to be interpreted. CONCLUSIONS: The presented methodology allows the propagation of uncertainty from experimental to model pre-dictions. Although presented in the context of ordinary differential equations, the concept is general and also applicable to other types of models. Matlab code which can be used as a template to implement the method is provided at http://www.fdmold.uni-freiburg.de/~ckreutz/PPL .     

Conditioning and time representation in long short-term memory networks

Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107–130, 2010). In this paper, that model’s ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.     

BIO-LGCA: A cellular automaton modelling class for analysing collective cell migration

Collective dynamics in multicellular systems such as biological organs and tissues plays a key role in biological development, regeneration, and pathological conditions. Collective tissue dynamics—understood as population behaviour arising from the interplay of the constituting discrete cells—can be studied with on- and off-lattice agent-based models. However, classical on-lattice agent-based models, also known as cellular automata, fail to replicate key aspects of collective migration, which is a central instance of collective behaviour in multicellular systems. To overcome drawbacks of classical on-lattice models, we introduce an on-lattice, agent-based modelling class for collective cell migration, which we call biological lattice-gas cellular automaton (BIO-LGCA). The BIO-LGCA is characterised by synchronous time updates, and the explicit consideration of individual cell velocities. While rules in classical cellular automata are typically chosen ad hoc, rules for cell-cell and cell-environment interactions in the BIO-LGCA can also be derived from experimental cell migration data or biophysical laws for individual cell migration. We introduce elementary BIO-LGCA models of fundamental cell interactions, which may be combined in a modular fashion to model complex multicellular phenomena. We exemplify the mathematical mean-field analysis of specific BIO-LGCA models, which allows to explain collective behaviour. The first example predicts the formation of clusters in adhesively interacting cells. The second example is based on a novel BIO-LGCA combining adhesive interactions and alignment. For this model, our analysis clarifies the nature of the recently discovered invasion plasticity of breast cancer cells in heterogeneous environments.     

Reward,motivation, and reinforcement learning

There is substantial evidence that dopamine is involved in reward learning and appetitive conditioning. However, the major reinforcement learning-based theoretical models of classical conditioning (crudely, prediction learning) are actually based on rules designed to explain instrumental conditioning (action learning). Extensive anatomical, pharmacological, and psychological data, particularly concerning the impact of motivational manipulations, show that these models are unreasonable. We review the data and consider the involvement of a rich collection of different neural systems in various aspects of these forms of conditioning. Dopamine plays a pivotal, but complicated, role.     

The Pavlovian analysis of instrumental conditioning.

An account was given of the development within the Russian literature of a uniprocess formulation of classical and instrumental conditioning, known as the bidirectional conditioning hypothesis. The hypothesis purports to offer a single set of Pavlovian principles to account for both paradigms, based upon a neural model which assumes that bidirectional (forward and backward) connections are formed in both calssical and instrumental conditioning situations. In instrumental conditioning, the bidirectional connections are hypothesized to be simply more complex than those in classical conditioning, and any differences in empirical functions are presumed to lie not in difference in mechanism, but in the strength of the forward and backward connections. Although bidirectional connections are assumed to develop in instrumental conditioning, the experimental investigation of the bidirectional conditioning hypothesis has been essentially restricted to the classical conditioning operations of pairing two CSs (sensory preconditioning training), a US followed by a CS (backward conditioning training) and two USs. However, the paradigm involving the pairing of two USs, because of theoretical and analytical considerations, is the one most commonly employed by Russian investigators. The results of an initial experiment involving the pairing of two USs, and reference to the results of a more extensive investigation, leads us to tentatively question the validity of the bidirectional conditioning account of instrumental conditioning.     

A note on equivalent mathematical models

A mathematical model of a process contains parameters supposedly characterizing the system which manifests the process. If the parameters are statistically distributed in a population of such systems, the process manifested by the entire population will in general be described by a different mathematical model. Thus a choice is always at hand between two or more mathematical models, depending on which parameters (if any) are assumed to be distributed and, if so, how. Examples of such alternative interpretations are given for mathematical models of some behavioral processes.     

Power-law models of signal transduction pathways

The mathematical modelling of signal transduction pathways has become a valuable aid to understanding the complex interactions involved in intracellular signalling mechanisms. An important aspect of the mathematical modelling process is the selection of the model type and structure. Until recently, the convention has been to use a standard kinetic model, often with the Michaelis-Menten steady state assumption. However this model form, although valuable, is only one of a number of choices, and the aim of this article is to consider the mathematical structure and essential features of an alternative model form--the power-law model. Specifically, we analyse how power-law models can be applied to increase our understanding of signal transduction pathways when there may be limited prior information. We distinguish between two kinds of power law models: a) Detailed power-law models, as a tool for investigating pathways when the structure of protein-protein interactions is completely known, and; b) Simplified power-law models, for the analysis of systems with incomplete structural information or insufficient quantitative data for generating detailed models. If sufficient data of high quality are available, the advantage of detailed power-law models is that they are more realistic representations of non-homogenous or crowded cellular environments. The advantages of the simplified power-law model formulation are illustrated using some case studies in cell signalling. In particular, the investigation on the effects of signal inhibition and feedback loops and the validation of structural hypotheses are discussed.     

A comparison of the effects of vicariously instigated classical conditioning and direct classical conditioning procedures.

Two groups of Ss received either two or 16 paired classical conditioning trails beyond the peak CR. A third group received the same stimuli as the 16 postpeak condition but in an unpaired and random order. The stimuli in all three groups were delivered directly to S. Subsequently, all three groups, including a fourth which was not given any prior direct classical conditioning, were exposed to vicariously instigated classical conditioning. This consisted of having S observe someone (model) employed by E who received the same CS as was delivered during direct conditioning. The CS was paired with the feigned arm movement of the model, simulating a reaction to shock. This vicarious classical conditioning procedure when compared to direct classical conditioning resulted in smaller GSR magnitudes for both the CRs and UCRs. Previous experience with direct classical conditional seems to have had an attenuating effect on GSR magnitude during the vicarious situation. A postexperimental questionnaire tended to support the results, and the relationship between the present study and current classical conditioning theory is discussed.     

Mutations, evolutionary theory and cancer

When searching for mutations that may be responsible for tumourigenesis and interpreting their significance, molecular oncologists often make a number of implicit assumptions about how and why tumour genotypes develop. These assumptions are based on an underlying classical model of tumourigenesis. The classical model has a number of similarities to models of evolution: given the parallels between the growth of tumours and the evolution of whole organisms, this is to be expected. However, consideration of tumourigenesis as an evolutionary process also suggests some modifications that might be made to the classical model. The experimental methods and data analysis of molecular oncology must take full account of the potential contribution of evolutionary theory. As the study of mutations in cancer expands, molecular oncologists are starting to do this.     

Modeling non-homologous end joining

Non-homologous end joining (NHEJ) is an important DNA repair pathway for DNA double-strand breaks. Several proteins, including Ku, DNA-PKcs, Artemis, XRCC4/Ligase IV and XLF, are involved in the NHEJ for the DNA damage detection, DNA free end processing and ligation. The classical model of NHEJ is a sequential model in which DNA-PKcs is first recruited by the Ku bound DNA prior to any other repair proteins. Recent experimental study ( [McElhinny et al., 2000], [Costantini et al., 2007], [17] and [Yano and Chen, 2008]) suggested that the recruitment ordering is not crucial. In this work, by proposing a mathematical model in terms of biochemical reaction network and performing stability and related analysis, we demonstrate theoretically that if DSB repair pathway independent of DNA-PKcs exists, then the classical sequential model and new two-phase model are essentially indistinguishable in the sense that DSB can be repaired thoroughly in both models when the repair proteins are sufficient.     

Regression analysis of an instrumental conditioned tentacular reflex in the edible snail

Regression analysis revealed the opportunity of approximation with exponential mathematical model of the learning curves of conditioned tentacle reflex. Retention of the reflex persisted for more than three weeks. There were some quantitative differences between conditioning of the right and the left tentacle. There was formation of the reflex in every session during spring period, but there was no retention between sessions. The conditioned tentacle reflex may be employed in neuropharmacological studies.     

A two-sex demographic model with single-dependent divorce rate

Divorce appears to be one of the least studied demographic processes, both empirically and in two-sex demographic models. In this paper, we study mathematical as well as biological implications of the assumption that the divorce rate is positively affected by the amount of single (i.e., unmarried/unpaired) individuals in the population. We do that by modifying the classical exponential two-sex model accounting for pair formation and separation. We model the divorce rate as an increasing function of the single population size and show that the single population pressure on the established couples alters the exponential behavior of the classical model in which the divorce rate is assumed constant. In particular, the total population size becomes bounded and a unique positive equilibrium exists. In addition, a Hopf bifurcation analysis around the positive equilibrium shows that the modified model may exhibit sustained oscillations.     

Steady state and (bi-) stability evaluation of simple protease signalling networks

Signal transduction networks are complex, as are their mathematical models. Gaining a deeper understanding requires a system analysis. Important aspects are the number, location and stability of steady states. In particular, bistability has been recognised as an important feature to achieve molecular switching. This paper compares different model structures and analysis methods particularly useful for bistability analysis.The biological applications include proteolytic cascades as, for example, encountered in the apoptotic signalling pathway or in the blood clotting system. We compare three model structures containing zero-order, inhibitor and cooperative ultrasensitive reactions, all known to achieve bistability. The combination of phase plane and bifurcation analysis provides an illustrative and comprehensive understanding of how bistability can be achieved and indicates how robust this behaviour is.Experimentally, some so-called “inactive” components were shown to have a residual activity. This has been mostly ignored in mathematical models. Our analysis reveals that bistability is only mildly affected in the case of zero-order or inhibitor ultrasensitivity. However, the case where bistability is achieved by cooperative ultrasensitivity is severely affected by this perturbation.     

The Importance of Age Dependent Mortality and the Extrinsic Incubation Period in Models of Mosquito-Borne Disease Transmission and Control

Nearly all mathematical models of vector-borne diseases have assumed that vectors die at constant rates. However, recent empirical research suggests that mosquito mortality rates are frequently age dependent. This work develops a simple mathematical model to assess how relaxing the classical assumption of constant mortality affects the predicted effectiveness of anti-vectorial interventions. The effectiveness of mosquito control when mosquitoes die at age dependent rates was also compared across different extrinsic incubation periods. Compared to a more realistic age dependent model, constant mortality models overestimated the sensitivity of disease transmission to interventions that reduce mosquito survival. Interventions that reduce mosquito survival were also found to be slightly less effective when implemented in systems with shorter EIPs. Future transmission models that examine anti-vectorial interventions should incorporate realistic age dependent mortality rates.