The faithful copy neuron

Theoretical and experimental evidence is presented for the presence in nervous tissue of neurons whose firing rate faithfully follow their input stimulus. Such neurons are shown to deliver their spikes with minimum dissipation per spike. This optimal performance is likely accomplished by use of local circuitry that adjusts conductances to match input currents so that the neuron operates near the threshold for firing. This results in an unusual mechanism for neuronal firing that uses background noise to achieve the desired firing rate. This framework takes place dynamically, and the present deliberations apply under time varying conditions. It is shown that an analytically explicit probability distribution function, which depends on one dimensionless parameter, can account for the interspike interval statistics under general time varying conditions. An innovative analysis based on the unsteady firing rate fits data to the appropriate probability distribution function.     

The resonance frequency shift, pattern formation, and dynamical network reorganization via sub-threshold input

We describe a novel mechanism that mediates the rapid and selective pattern formation of neuronal network activity in response to changing correlations of sub-threshold level input. The mechanism is based on the classical resonance and experimentally observed phenomena that the resonance frequency of a neuron shifts as a function of membrane depolarization. As the neurons receive varying sub-threshold input, their natural frequency is shifted in and out of its resonance range. In response, the neuron fires a sequence of action potentials, corresponding to the specific values of signal currents, in a highly organized manner. We show that this mechanism provides for the selective activation and phase locking of the cells in the network, underlying input-correlated spatio-temporal pattern formation, and could be the basis for reliable spike-timing dependent plasticity. We compare the selectivity and efficiency of this pattern formation to a supra-threshold network activation and a non-resonating network/neuron model to demonstrate that the resonance mechanism is the most effective. Finally we show that this process might be the basis of the phase precession phenomenon observed during firing of hippocampal place cells, and that it may underlie the active switching of neuronal networks to locking at various frequencies.     

Improved neuronal models for studying neural networks

Previous neuronal models used for the study of neural networks are considered. Equations are developed for a model which includes: 1) a normalized range of firing rates with decreased sensitivity at large excitatory or large inhibitory input levels, 2) a single rate constant for the increase in firing rate following step changes in the input, 3) one or more rate constants, as required to fit experimental data for the adaptation of firing rates to maintained inputs. Computed responses compare well with the types of neuronal responses observed experimentally. Depending on the parameters, overdamped increases and decreases, damped oscillatory or maintained oscillatory changes in firing rate are observed to step changes in the input. The integrodifferential equations describing the neuronal models can be represented by a set of first-order differential equations. Steady-state solutions for these equations can be obtained for constant inputs, as well as the stability of the solutions to small perturbations. The linear frequency response function is derived for sufficiently small time-varying inputs. The linear responses are also compared with the computed solutions for larger non-linear responses.     

The influence of limited presynaptic growth and synapse removal on adaptive synaptogenesis

This report continues our research into the effectiveness of adaptive synaptogenesis in constructing feed-forward networks which perform good transformations on their inputs. Good transformations are characterized by the maintenance of input information and the removal of statistical dependence. Adaptive synaptogenesis stochastically builds and sculpts a synaptic connectivity in initially unconnected networks using two mechanisms. The first, synaptogenesis, creates new, excitatory, feed-forward connections. The second, associative modification, adjusts the strength of existing synapses. Our previous implementations of synaptogenesis only incorporated a postsynaptic regulatory process, receptivity to new innervation (Adelsberger-Mangan and Levy 1993a, b). In the present study, a presynaptic regulatory process, presynaptic avidity, which regulates the tendency of a presynaptic neuron to participate in a new synaptic connection as a function of its total synaptic weight, is incorporated into the synaptogenesis process. In addition, we investigate a third mechanism, selective synapse removal. This process removes synapses between neurons whose firing is poorly correlated. Networks that are constructed with the presynaptic regulatory process maintain more information and remove more statistical dependence than networks constructed with postsynaptic receptivity and associative modification alone. Selective synapse removal also improves network performance, but only when implemented in conjunction with the presynaptic regulatory process. Received: 20 August 1993/Accepted in revised form: 16 April 1994     

Analysis of the activity of single neurons in stochastic settings

This paper presents a new way of modeling the activity of single neurons in stochastic settings. It incorporates in a natural way many physiological mechanisms not usually found in stochastic models, such as spatial integration, non-linear membrane characteristics and non-linear interactions between excitation and inhibition. The model is based on the fact that most of the neuronal inputs have a finite lifetime. Thus, the stochastic input can be modeled as a simple finite markov chain, and the membrane potential becomes a function of the state of this chain. Firing occurs at states whose membrane potential is above threshold. The main mathematical results of the model are: (i) the input-output firing rate curve is convex at low firing rates and is saturated at high firing rates, and (ii) at low firing rates, firing usually occurs when there is synchronous convergence of many excitatory events.     

Information maintenance and statistical dependence reduction in simple neural networks

This study compares the ability of excitatory, feed-forward neural networks to construct good transformations on their inputs. The quality of such a transformation is judged by the minimization of two information measures: the information loss of the transformation and the statistical dependency of the output. The networks that are compared differ from each other in the parametric properties of their neurons and in their connectivity. The particular network parameters studied are output firing threshold, synaptic connectivity, and associative modification of connection weights. The network parameters that most directly affect firing levels are threshold and connectivity. Networks incorporating neurons with dynamic threshold adjustment produce better transformations. When firing threshold is optimized, sparser synaptic connectivity produces a better transformation than denser connectivity. Associative modification of synaptic weights confers only a slight advantage in the construction of optimal transformations. Additionally, our research shows that some environments are better suited than others for recoding. Specifically, input environments high in statistical dependence, i.e. those environments most in need of recoding, are more likely to undergo successful transformations.     

Information coding and oscillatory activity in synfire neural networks with and without inhibitory coupling

When a population spike (pulse-packet) propagates through a feedforward network with random excitatory connections, it either evolves to a sustained stable level of synchronous activity or fades away (Diesmann et al. in Nature 402:529-533 1999; Cateau and Fukai Neur Netw 14:675-685 2001). Here I demonstrate that in the presence of noise, the probability of the survival of the pulse-packet (or, equivalently, the firing rate of output neurons) reflects the intensity of the input. Furthermore, inhibitory coupling between layers can result in quasi- periodic alternation between several levels of firing activity. These results are obtained by analyzing the evolution of pulse-packet activity as a Markov chain. For the Markov chain analysis, the output of the chain is a linear mapping of the input into a lower-dimensional space, and the eigenvalues and eigenvectors of the transition matrix determine the dynamics of the evolution. Synchronous propagation of firing activity in successive pools of neurons are simulated in networks of integrate-and-fire and compartmental model neurons, and, consistent with the discrete Markov process, the activation of each pool is observed to be predominantly dependent upon the number of cells that fired in the previous pool. Simulation results agree with the numerical solutions of the Markov model. When inhibitory coupling between layers are included in the Markov model, some eigenvalues become complex numbers, implying oscillatory dynamics. The quasiperiodic dynamics is validated with simulation with leaky integrate-and-fire neurons. The networks demonstrate different modes of quasiperiodic activity as the inhibition or excitation parameters of the network are varied.     

Mono-synaptic reflex responses of individual motoneurons

Individual motoneuron responses to a variety of afferent inputs have been examined. At a given input some motoneurons respond to every trial, some to no trial, and some respond to a certain percentage of trials that is characteristic for the motoneuron at that input. The performance of a motoneuron is expressed by means of a firing index that relates the number of responses to the number of trials. In a representative assemblage of individual motoneurons some 20 to 30 per cent display intermediate firing indices. This number, comprising an "intermediate zone" remains fairly constant at different levels of input although the individuals within it may be entirely different at two different levels of input. Frequency distribution of individuals with respect to firing indices is U-shaped. Intermediacy of firing indices depends upon temporal fluctuation of excitability which, in the first approximation, is normal. The individual motoneurons are approximately equally frequently distributed with respect to transmitter potentiality of their monosynaptic reflex afferent connections. The distribution of motoneurons with respect to transmitter potentiality of their monosynaptic reflex connections is considered representative of a natural pool in that the sum of their individual post-tetanic response behaviors accurately reproduces the course of post-tetanic potentiation in a natural pool.     

Theoretical Analysis of Time-to-Peak Responses in Biological Reaction Networks

Processing of information by signaling networks is characterized by properties of the induced kinetics of the activated pathway components. The maximal extent of pathway activation (maximum amplitude) and the time-to-peak-response (position) are key determinants of biological responses that have been linked to specific outcomes. We investigate how the maximum amplitude of pathway activation and its position depend on the input and wiring of a signaling network. For this purpose, we consider a simple reaction A→B that is regulated by a transient input and extended this to include back-reaction and additional partners. In particular, we show that a unique maximum of B(t) exists. Moreover, we prove that the position of the maximum is independent of the applied input but regulated by degradation reactions of B. Indeed, the time-to-peak-response decreases with increasing degradation rate, which we prove for small models and show in simulations for more complex ones. The identified dependencies provide insights into design principles that facilitate the realization dynamical characteristics like constant position of maximal pathway activation and thereby guide the characterization of unknown kinetics within larger protein networks.     

Langevin machine: a neural network based on stochastically justifiable sigmoidal function

In neural networks the activation process controls the output as a nonlinear function of the input; and, this output remains bounded between limits as decided by a logistic function known as the sigmoid (S-shaped). Presently, by applying the considerations of Maxwell-Boltzmann statistics, the Langevin function is shown as the appropriate and justifiable sigmoid (instead of the conventional hyperbolic tangent function) to depict the bipolar nonlinear logic-operation enunciated by the collective stochastical response of artificial neurons under activation. That is, the graded response of a large network of neurons such as Hopfield's can be stochastically justified via the proposed model. The model is consistent with the established link between the Hopfield model and the statistical mechanics. The possible outcomes and implications of using the Langevin function (in lieu of conventional hyperbolic tangent and/or exponential sigmoids) in determining nonlinear decision boundaries, in characterizing the neural networks by the Langevin machine versus the Boltzmann machine, in sharpening and annealing schedules and in the optimization of nonlinear detector performance are discussed.     

Modulation in firing pattern and oscillation in nerve cells of Lymnaea during network reconstruction

The modulation and reconstruction of the cardio-respiratory neural circuit of Lymnaea stagnalis L. was compared to that of Helix ponatia L. where the input variation and signal molecules were found to have primary importance in network reorganization. From the cardio-respiratory circuit only neurons connected by afferent or efferent pathways to the peripheral chemosensory organ, the osphradium, were used. It was shown that, the general principles of the network reorganization is similar in the two species. The firing pattern of the neurons altered in Lymnaea depending on the input activation or presence of signal molecules in the vicinity of the neurons. The responses of the neurons to the same sensory information, originating from osphradium varied depending on their firing patterns. On central neurones the generation of phasic pattern and/or oscillation was an indicator of network disintegration leading to insensibility to the osphradial sensory inputs. Co-application of signal molecules (5HT, DA, GABA with opioid peptides) to the neurons caused a phasic firing pattern and/or oscillation leading to disintegration of one network and activation of another one. The effect of mu-opioid peptides on GABA-induced and voltage activated ion currents were shown to be the cellular target in reconstruction of neural networks in Lymnaea. The neural network reconstruction in vertebrate brain evoked by signal molecules can be compared to that observed in the identified network of Lymnaea stagnalis making this latter a useful model in further studies, too.     

The functional role of different neural activation profiles during precision grip: An artificial neural network approach

A dynamic and recurrent artificial neural network was used to investigate the functional properties of firing patterns observed in the primary motor (M1) and the primary somatosensory (S1) cortex of the behaving monkey during control of precision grip force. In the behaving monkey it was found that neurons in M1 and in S1 increase their firing activity with increasing grip force, as do the intrinsic and extrinsic hand muscles implicated in the task. However, some neurons also decreased their activity as a function of increasing force. The functional implication of these latter neurons is not clear and has not been elucidated so far. In order to explore their functional implication, we therefore simulated patterns of neural activity in artificial neural networks that represent cortical, spinal and afferent neural populations and tested whether particular activity profiles would emerge as a function of the input and of the connectivity of these networks. The functional implication of units with emergent or imposed decreasing activity was then explored.Decreasing patterns of activity in M1 units did not emerge from the networks. However, the same networks generated decreasing activity if imposed as target patterns. As indicated by the emerging weight space, M1 projection units with decreasing patterns are functionally less involved in driving alpha motoneurons than units with increasing profiles. Furthermore, these units did not provide significant fusimotor drive, whereas those with increasing profiles did. Fusimotor drive was a function of the (imposed) form of muscle spindle afferent activity: with gamma (fusimotor) drive, muscle spindle afferents provided signals other than muscle length (as observed experimentally). The network solutions thus predict a functional dichotomy between increasing and decreasing M1 neurons: the former primarily drive alpha and gamma motoneurons, the latter only weakly alpha motoneurons.     

Sensitivity of firing rate to input fluctuations depends on time scale separation between fast and slow variables in single neurons

Neuronal responses are often characterized by the firing rate as a function of the stimulus mean, or the f–I curve. We introduce a novel classification of neurons into Types A, B−, and B+ according to how f–I curves are modulated by input fluctuations. In Type A neurons, the f–I curves display little sensitivity to input fluctuations when the mean current is large. In contrast, Type B neurons display sensitivity to fluctuations throughout the entire range of input means. Type B− neurons do not fire repetitively for any constant input, whereas Type B+ neurons do. We show that Type B+ behavior results from a separation of time scales between a slow and fast variable. A voltage-dependent time constant for the recovery variable can facilitate sensitivity to input fluctuations. Type B+ firing rates can be approximated using a simple “energy barrier” model.     

Efficiency of neural transmission as a function of synaptic noise, threshold, and source characteristics

There has been a growing interest in the estimation of information carried by a single neuron and multiple single units or population of neurons to specific stimuli. In this paper we analyze, inspired by article of Levy and Baxter (2002), the efficiency of a neuronal communication by considering dendrosomatic summation as a Shannon-type channel (1948) and by considering such uncertain synaptic transmission as part of the dendrosomatic computation. Specifically, we study Mutual Information between input and output signals for different types of neuronal network architectures by applying efficient entropy estimators. We analyze the influence of the following quantities affecting transmission abilities of neurons: synaptic failure, activation threshold, firing rate and type of the input source. We observed a number of surprising non-intuitive effects. It turns out that, especially for lower activation thresholds, significant synaptic noise can lead even to twofold increase of the transmission efficiency. Moreover, the efficiency turns out to be a non-monotonic function of the activation threshold. We find a universal value of threshold for which a local maximum of Mutual Information is achieved for most of the neuronal architectures, regardless of the type of the source (correlated and non-correlated). Additionally, to reach the global maximum the optimal firing rates must increase with the threshold. This effect is particularly visible for lower firing rates. For higher firing rates the influence of synaptic noise on the transmission efficiency is more advantageous. Noise is an inherent component of communication in biological systems, hence, based on our analysis, we conjecture that the neuronal architecture was adjusted to make more effective use of this attribute.     

Interplay of Intrinsic and Synaptic Conductances in the Generation of High-Frequency Oscillations in Interneuronal Networks with Irregular Spiking

High-frequency oscillations (above 30 Hz) have been observed in sensory and higher-order brain areas, and are believed to constitute a general hallmark of functional neuronal activation. Fast inhibition in interneuronal networks has been suggested as a general mechanism for the generation of high-frequency oscillations. Certain classes of interneurons exhibit subthreshold oscillations, but the effect of this intrinsic neuronal property on the population rhythm is not completely understood. We study the influence of intrinsic damped subthreshold oscillations in the emergence of collective high-frequency oscillations, and elucidate the dynamical mechanisms that underlie this phenomenon. We simulate neuronal networks composed of either Integrate-and-Fire (IF) or Generalized Integrate-and-Fire (GIF) neurons. The IF model displays purely passive subthreshold dynamics, while the GIF model exhibits subthreshold damped oscillations. Individual neurons receive inhibitory synaptic currents mediated by spiking activity in their neighbors as well as noisy synaptic bombardment, and fire irregularly at a lower rate than population frequency. We identify three factors that affect the influence of single-neuron properties on synchronization mediated by inhibition: i) the firing rate response to the noisy background input, ii) the membrane potential distribution, and iii) the shape of Inhibitory Post-Synaptic Potentials (IPSPs). For hyperpolarizing inhibition, the GIF IPSP profile (factor iii)) exhibits post-inhibitory rebound, which induces a coherent spike-mediated depolarization across cells that greatly facilitates synchronous oscillations. This effect dominates the network dynamics, hence GIF networks display stronger oscillations than IF networks. However, the restorative current in the GIF neuron lowers firing rates and narrows the membrane potential distribution (factors i) and ii), respectively), which tend to decrease synchrony. If inhibition is shunting instead of hyperpolarizing, post-inhibitory rebound is not elicited and factors i) and ii) dominate, yielding lower synchrony in GIF networks than in IF networks.     

Analysis of clustered firing patterns in synaptically coupled networks of oscillators

Oscillators in networks may display a variety of activity patterns. This paper presents a geometric singular perturbation analysis of clustering, or alternate firing of synchronized subgroups, among synaptically coupled oscillators. We consider oscillators in two types of networks: mutually coupled, with all-to-all inhibitory connections, and globally inhibitory, with one excitatory and one inhibitory population of oscillators, each of arbitrary size. Our analysis yields existence and stability conditions for clustered states, along with formulas for the periods of such firing patterns. By using two different approaches, we derive complementary conditions, the first set stated in terms of time lengths determined by intrinsic and synaptic properties of the oscillators and their coupling and the second set stated in terms of model parameters and phase space structures directly linked to parameters. These results suggest how biological components may interact to produce the spindle sleep rhythm in thalamocortical networks. Received: 9 September 1999 / Revised version: 7 July 2000 / Published online: 24 November 2000     

Spike synchronization and firing rate in a population of motor cortical neurons in relation to movement direction and reaction time

 We studied the dynamics of precise spike synchronization and rate modulation in a population of neurons recorded in monkey motor cortex during performance of a delayed multidirectional pointing task and determined their relation to behavior. We showed that at the population level neurons coherently synchronized their activity at various moments during the trial in relation to relevant task events. The comparison of the time course of the modulation of synchronous activity with that of the firing rate of the same neurons revealed a considerable difference. Indeed, when synchronous activity was highest, at the end of the preparatory period, firing rate was low, and, conversely, when the firing rate was highest, at movement onset, synchronous activity was almost absent. There was a clear tendency for synchrony to precede firing rate, suggesting that the coherent activation of cell assemblies may trigger the increase in firing rate in large groups of neurons, although it appeared that there was no simple parallel shifting in time of these two activity measures. Interestingly, there was a systematic relationship between the amount of significant synchronous activity within the population of neurons and movement direction at the end of the preparatory period. Furthermore, about 400 ms later, at movement onset, the mean firing rate of the same population was also significantly tuned to movement direction, having roughly the same preferred direction as synchronous activity. Finally, reaction time measurements revealed a directional preference of the monkey with, once again, the same preferred direction as synchronous activity and firing rate. These results lead us to speculate that synchronous activity and firing rate are cooperative neuronal processes and that the directional matching of our three measures – firing rate, synchronicity, and reaction times – might be an effect of behaviorally induced network cooperativity acquired during learning. Received: 16 January 2002 / Accepted in revised form: 26 November 2002 / Published online: 7 April 2003 RID="*" ID="*" Present address: Istituto di Fisiologia Umana, Università di Parma, Via Volturno 39, 43100 Parma, Italy Correspondence to: A. Riehle (e-mail: ariehle@lnf.cnrs-mrs.fr, Tel.: +33-491-164329, Fax: +33-491-774969) Acknowledgements. We wish to thank Sonja Grün, Markus Diesmann, and Bill MacKay for many helpful and exciting discussions and one anonymous referee for her/his helpful comments. Special thanks go to Annette Bastian for her help in data collection, Michèle Coulmance for writing data acquisition and parts of data analysis software, and Marc Martin for animal welfare. This research was supported in part by the CNRS, GIS (Sciences de la Cognition), and ACI Cognitique (Invariants and Variability). FG was supported by the French government (MENRT).     

A model for steady isometric muscle activation

The present model of the motoneuronal (MN) pool – muscle complex (MNPMC) is deterministic and designed for steady isometric muscle activation. Time-dependent quantities are treated as time-averages. The character of the model is continuous in the sense that the motor unit (MU) population is described by a continuous density function. In contrast to most already published models, the wiring (synaptic weight) between the input fibers to the MNPMC and the MNs (about which no detailed data are known) is deduced, whereas the input–force relation is given. As suggested by experimental data, this relation is assumed to be linear during MU recruitment, but the model allows other, nonlinear relations. The input to the MN pool is defined as the number of action potentials per second in all input fibers, and the excitatory postsynaptic potential (EPSP) conductance in MNs evoked by the input is assumed to be proportional to the input. A single compartment model with a homogeneous membrane is used for a MN. The MNs start firing after passing a constant voltage threshold. The synaptic current–frequency relation is described by a linear function and the frequency–force transformation of a MU by an exponential function. The sum of the MU contraction forces is the muscle force, and the activation of the MUs obeys the size principle. The model parameters were determined a priori, i.e., the model was not used for their estimation. The analysis of the model reveals special features of the activation curve which we define as the relation between the input normalized by the threshold input of the MN pool and the force normalized by the maximal muscle force. This curve for any muscle turned out to be completely determined by the activation factor, the slope of the linear part of the activation curve (during MU recruitment). This factor determines quantitatively the relation between MU recruitment and rate modulation. This property of the model (the only known model with this property) allows a quantification of the recruitment gain (Kernell and Hultborn 1990). The interest of the activation factor is illustrated using two human muscles, namely the first dorsal interosseus muscle, a small muscle with a relatively small force at the end of recruitment, and the medial gastrocnemius muscle, a strong muscle with a relatively large force at the end of recruitment. It is concluded that the present model allows us to reproduce the main features of muscle activation in the steady state. Its analytical character facilitates a deeper understanding of these features. Received: 24 November 1997 / Accepted in revised form: 30 November 1998     

The chronotron: a neuron that learns to fire temporally precise spike patterns

In many cases, neurons process information carried by the precise timings of spikes. Here we show how neurons can learn to generate specific temporally precise output spikes in response to input patterns of spikes having precise timings, thus processing and memorizing information that is entirely temporally coded, both as input and as output. We introduce two new supervised learning rules for spiking neurons with temporal coding of information (chronotrons), one that provides high memory capacity (E-learning), and one that has a higher biological plausibility (I-learning). With I-learning, the neuron learns to fire the target spike trains through synaptic changes that are proportional to the synaptic currents at the timings of real and target output spikes. We study these learning rules in computer simulations where we train integrate-and-fire neurons. Both learning rules allow neurons to fire at the desired timings, with sub-millisecond precision. We show how chronotrons can learn to classify their inputs, by firing identical, temporally precise spike trains for different inputs belonging to the same class. When the input is noisy, the classification also leads to noise reduction. We compute lower bounds for the memory capacity of chronotrons and explore the influence of various parameters on chronotrons' performance. The chronotrons can model neurons that encode information in the time of the first spike relative to the onset of salient stimuli or neurons in oscillatory networks that encode information in the phases of spikes relative to the background oscillation. Our results show that firing one spike per cycle optimizes memory capacity in neurons encoding information in the phase of firing relative to a background rhythm.     

Recurrent networks of submucous neurons controlling intestinal secretion: a modeling study

Secretomotor neurons, immunoreactive for vasoactive intestinal peptide (VIP), are important in controlling chloride secretion in the small intestine. These neurons form functional synapses with other submucosal VIP neurons and transmit via slow excitatory postsynaptic potentials (EPSPs). Thus they form a recurrent network with positive feedback. Intrinsic sensory neurons within the submucosa are also likely to form recurrent networks with positive feedback, provide substantial output to VIP neurons, and receive input from VIP neurons. If positive feedback within recurrent networks is sufficiently large, then neurons in the network respond to even small stimuli by firing at their maximum possible rate, even after the stimulus is removed. However, it is not clear whether such a mechanism operates within the recurrent networks of submucous neurons. We investigated this question by performing computer simulations of realistic models of VIP and intrinsic sensory neuron networks. In the expected range of electrophysiological properties, we found that activity in the VIP neuron network decayed slowly after cessation of a stimulus, indicating that positive feedback is not strong enough to support the uncontrolled firing state. The addition of intrinsic sensory neurons produced a low stable firing rate consistent with the common finding that basal secretory activity is, in part, neurogenic. Changing electrophysiological properties enables these recurrent networks to support the uncontrolled firing state, which may have implications with hypersecretion in the presence of enterotoxins such as cholera-toxin.