Reservoir Computing Properties of Neural Dynamics in Prefrontal Cortex

Primates display a remarkable ability to adapt to novel situations. Determining what is most pertinent in these situations is not always possible based only on the current sensory inputs, and often also depends on recent inputs and behavioral outputs that contribute to internal states. Thus, one can ask how cortical dynamics generate representations of these complex situations. It has been observed that mixed selectivity in cortical neurons contributes to represent diverse situations defined by a combination of the current stimuli, and that mixed selectivity is readily obtained in randomly connected recurrent networks. In this context, these reservoir networks reproduce the highly recurrent nature of local cortical connectivity. Recombining present and past inputs, random recurrent networks from the reservoir computing framework generate mixed selectivity which provides pre-coded representations of an essentially universal set of contexts. These representations can then be selectively amplified through learning to solve the task at hand. We thus explored their representational power and dynamical properties after training a reservoir to perform a complex cognitive task initially developed for monkeys. The reservoir model inherently displayed a dynamic form of mixed selectivity, key to the representation of the behavioral context over time. The pre-coded representation of context was amplified by training a feedback neuron to explicitly represent this context, thereby reproducing the effect of learning and allowing the model to perform more robustly. This second version of the model demonstrates how a hybrid dynamical regime combining spatio-temporal processing of reservoirs, and input driven attracting dynamics generated by the feedback neuron, can be used to solve a complex cognitive task. We compared reservoir activity to neural activity of dorsal anterior cingulate cortex of monkeys which revealed similar network dynamics. We argue that reservoir computing is a pertinent framework to model local cortical dynamics and their contribution to higher cognitive function.     

Limnology of a Warm Hypertrophic Wastewater Reservoir in Israel. II. Changes in Water Quality

Factor analysis was used to identify the processes responsible for the changes in water quality in a stratified wastewater reservoir. These reservoirs are well suited to study hypertrophic conditions. In the epilimnion, the main source of variablility was the external loading due to variations in wastewater inputs and water level (volume) of the reservoir. The second source was the seasonal climatic changes affecting photosynthesis. The third source was differences between night and day processes affecting REDOX potential. In the hypolimnion, the main source of variability was organic loading but due to both external loading (wastewater inputs) and internal loading (compounds released from the sediments). The second source was related to the input of chlorophyll into the hypolimnion, which were external (entry of chlorophyll rich inflow from a waste stabilization pond) and/or internal (input of chlorophyll from upper layers due to water column mixing).     

Storage of correlated patterns in standard and bistable Purkinje cell models

The cerebellum has long been considered to undergo supervised learning, with climbing fibers acting as a 'teaching' or 'error' signal. Purkinje cells (PCs), the sole output of the cerebellar cortex, have been considered as analogs of perceptrons storing input/output associations. In support of this hypothesis, a recent study found that the distribution of synaptic weights of a perceptron at maximal capacity is in striking agreement with experimental data in adult rats. However, the calculation was performed using random uncorrelated inputs and outputs. This is a clearly unrealistic assumption since sensory inputs and motor outputs carry a substantial degree of temporal correlations. In this paper, we consider a binary output neuron with a large number of inputs, which is required to store associations between temporally correlated sequences of binary inputs and outputs, modelled as Markov chains. Storage capacity is found to increase with both input and output correlations, and diverges in the limit where both go to unity. We also investigate the capacity of a bistable output unit, since PCs have been shown to be bistable in some experimental conditions. Bistability is shown to enhance storage capacity whenever the output correlation is stronger than the input correlation. Distribution of synaptic weights at maximal capacity is shown to be independent on correlations, and is also unaffected by the presence of bistability.     

Spike Triggered Covariance in Strongly Correlated Gaussian Stimuli

Many biological systems perform computations on inputs that have very large dimensionality. Determining the relevant input combinations for a particular computation is often key to understanding its function. A common way to find the relevant input dimensions is to examine the difference in variance between the input distribution and the distribution of inputs associated with certain outputs. In systems neuroscience, the corresponding method is known as spike-triggered covariance (STC). This method has been highly successful in characterizing relevant input dimensions for neurons in a variety of sensory systems. So far, most studies used the STC method with weakly correlated Gaussian inputs. However, it is also important to use this method with inputs that have long range correlations typical of the natural sensory environment. In such cases, the stimulus covariance matrix has one (or more) outstanding eigenvalues that cannot be easily equalized because of sampling variability. Such outstanding modes interfere with analyses of statistical significance of candidate input dimensions that modulate neuronal outputs. In many cases, these modes obscure the significant dimensions. We show that the sensitivity of the STC method in the regime of strongly correlated inputs can be improved by an order of magnitude or more. This can be done by evaluating the significance of dimensions in the subspace orthogonal to the outstanding mode(s). Analyzing the responses of retinal ganglion cells probed with Gaussian noise, we find that taking into account outstanding modes is crucial for recovering relevant input dimensions for these neurons.     

农田合理投能区域与最优投能配比的研究

研究表明,农田代辅助能投入有一个合理的投区域,在该区域内增加水肥投能可增加产出能和产投比,总产出能的合理投能区域大于产投比的合理投能区域;低产农田的合理投区域大小高产农田,通过人工辅助能的增加,可能低产农田与高产农田一样达到作物的最高产量水。在合理投能区域内存在水分投 效线和肥料投入高效线,在两条线上相应的水分和肥料投入效益最高;也存在最优投能配比线和最优经济投入配比线,在两条线上的水肥投入,其能     

Lateral input of particulate organic matter from bank slopes surpasses direct litter fall in the uppermost reaches of a headwater stream in Hokkaido, Japan

In forested streams, surrounding riparian forests provide essential supplies of organic matter to aquatic ecosystems. We focused on two pathways of particulate organic matter inputs: direct input from upper riparian forests and indirect lateral input from bank slopes, for which there are limited quantitative data. We investigated the inputs of coarse particulate organic matter (CPOM) and carbon and nitrogen in the CPOM into the uppermost reaches of a headwater stream with steep bank slopes in Hokkaido, Japan. CPOM collected by litter traps was divided into categories (e.g., leaves, twigs) and weighed. Monthly nitrogen and carbon inputs were also estimated. The annual direct input of CPOM (ash-free dry mass) was 472 g m⁻², a common value for temperate riparian forests. The annual lateral CPOM input was 353 g m⁻¹ and 941 g m⁻² when they were converted to area base. This value surpassed the direct input. Organic matter that we could not separate from inorganic sediments contributed to the total lateral input from the bank slopes (124 g m⁻¹); this organic matter contained relatively high amounts of nitrogen and carbon. At uppermost stream reaches, the bank slope would be a key factor to understanding the carbon and nitrogen pathways from the surrounding terrestrial ecosystem to the aquatic ecosystem.     

Identifying and Tracking Simulated Synaptic Inputs from Neuronal Firing: Insights from In Vitro Experiments

Accurately describing synaptic interactions between neurons and how interactions change over time are key challenges for systems neuroscience. Although intracellular electrophysiology is a powerful tool for studying synaptic integration and plasticity, it is limited by the small number of neurons that can be recorded simultaneously in vitro and by the technical difficulty of intracellular recording in vivo. One way around these difficulties may be to use large-scale extracellular recording of spike trains and apply statistical methods to model and infer functional connections between neurons. These techniques have the potential to reveal large-scale connectivity structure based on the spike timing alone. However, the interpretation of functional connectivity is often approximate, since only a small fraction of presynaptic inputs are typically observed. Here we use in vitro current injection in layer 2/3 pyramidal neurons to validate methods for inferring functional connectivity in a setting where input to the neuron is controlled. In experiments with partially-defined input, we inject a single simulated input with known amplitude on a background of fluctuating noise. In a fully-defined input paradigm, we then control the synaptic weights and timing of many simulated presynaptic neurons. By analyzing the firing of neurons in response to these artificial inputs, we ask 1) How does functional connectivity inferred from spikes relate to simulated synaptic input? and 2) What are the limitations of connectivity inference? We find that individual current-based synaptic inputs are detectable over a broad range of amplitudes and conditions. Detectability depends on input amplitude and output firing rate, and excitatory inputs are detected more readily than inhibitory. Moreover, as we model increasing numbers of presynaptic inputs, we are able to estimate connection strengths more accurately and detect the presence of connections more quickly. These results illustrate the possibilities and outline the limits of inferring synaptic input from spikes.     

Periodic disturbances,trophic gradient and phytoplankton characteristics related to cyanobacterial growth in Guarapiranga Reservoir,São Paulo State,Brazil

The relationships between phytoplankton assemblage, disturbance and trophic characteristics were investigated in Guarapiranga Reservoir, which supplies water to approximately 25% of the São Paulo metropolitan region. Two open-water stations were sampled weekly during 1991–92, in order to determine the main factors promoting of Cyanobacterial growth. These were found to include: rainfall-induced nutrient inputs via runoff and percolation from the drainage basin; circulation patterns that favored resurgence of nutrients and transported littoral algae to the open water; senescence of algae (especially Chlorococcales and Zygnemaphyceae) which increased the availability of nutrients in the water column; and high nutrient concentrations during the driest season. Restricting nutrient inputs and controlling eutrophication are necessary if the quality of water supplied from this reservoir is to be improved.     

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

A large body of experimental and theoretical work on neural coding suggests that the information stored in brain circuits is represented by time-varying patterns of neural activity. Reservoir computing, where the activity of a recurrently connected pool of neurons is read by one or more units that provide an output response, successfully exploits this type of neural activity. However, the question of system robustness to small structural perturbations, such as failing neurons and synapses, has been largely overlooked. This contrasts with well-studied dynamical perturbations that lead to divergent network activity in the presence of chaos, as is the case for many reservoir networks. Here, we distinguish between two types of structural network perturbations, namely local (e.g., individual synaptic or neuronal failure) and global (e.g., network-wide fluctuations). Surprisingly, we show that while global perturbations have a limited impact on the ability of reservoir models to perform various tasks, local perturbations can produce drastic effects. To address this limitation, we introduce a new architecture where the reservoir is driven by a layer of oscillators that generate stable and repeatable trajectories. This model outperforms previous implementations while being resistant to relatively large local and global perturbations. This finding has implications for the design of reservoir models that capture the capacity of brain circuits to perform cognitively and behaviorally relevant tasks while remaining robust to various forms of perturbations. Further, our work proposes a novel role for neuronal oscillations found in cortical circuits, where they may serve as a collection of inputs from which a network can robustly generate complex dynamics and implement rich computations.     

A modelling study on discrimination tasks

We investigate the discrimination capacity of a single neuron model including the integrate-and-fire (IF) model and the IF-FHN model [Neural Networks, 14 (2001) 955], both theoretically and numerically. Both magnitude and frequency discrimination tasks are considered. It is found that for the magnitude discrimination task, with a small fraction of coherent inputs, an IF model and an IF-FHN model with inhibitory inputs can tell one input from the other. The total probability of misclassifications (TPM) is considerably reduced with the increasing of inhibitory inputs. However, the IF model and the IF-FHN model are incapable of performing the frequency discrimination task.     

Consumers can enhance ecosystem productivity and stability in changing environments

There is increasing interest in the impacts of pulsed resource inputs (e.g., seasonal fluctuations in primary production) on ecosystem properties. However, theoretical models have focused on the ecosystem role of consumers only in stable environments. We investigated how the consumption rate affects ecosystem properties when resource supplies are pulsed using a simple model with numerical simulations. We initially investigated how primary production responds to resource supply shortages under various consumption rates and found that a vigorous consumer attenuates a rapid reduction in primary production. We next examined ecosystem properties under pulsed resource supplies. Intermediate consumption enhances primary production. The consumption rate that maximizes productivity differs for producers and consumers in a given resource supply environment, irrespective of the pattern of pulsation. The enhancement of primary production and the maximization of secondary production occur simultaneously in a stable environment but are not always the same when the resource supply is pulsed. We also investigated the variability of primary production and found that intermediate consumption rates reduce its coefficient of variation. In addition, we found that consumers with either very low or very high consumption rates are vulnerable to extinction when resource supplies are pulsed. Therefore, consumers with intermediate consumption rates contribute not only to the enhancement and stabilization of primary production but also to the stability of the consumer population itself.     

Improved neuronal models for studying neural networks

Previous neuronal models used for the study of neural networks are considered. Equations are developed for a model which includes: 1) a normalized range of firing rates with decreased sensitivity at large excitatory or large inhibitory input levels, 2) a single rate constant for the increase in firing rate following step changes in the input, 3) one or more rate constants, as required to fit experimental data for the adaptation of firing rates to maintained inputs. Computed responses compare well with the types of neuronal responses observed experimentally. Depending on the parameters, overdamped increases and decreases, damped oscillatory or maintained oscillatory changes in firing rate are observed to step changes in the input. The integrodifferential equations describing the neuronal models can be represented by a set of first-order differential equations. Steady-state solutions for these equations can be obtained for constant inputs, as well as the stability of the solutions to small perturbations. The linear frequency response function is derived for sufficiently small time-varying inputs. The linear responses are also compared with the computed solutions for larger non-linear responses.     

Inhibitory synchrony as a mechanism for attentional gain modulation

Recordings from area V4 of monkeys have revealed that when the focus of attention is on a visual stimulus within the receptive field of a cortical neuron, two distinct changes can occur: The firing rate of the neuron can change and there can be an increase in the coherence between spikes and the local field potential (LFP) in the gamma-frequency range (30-50 Hz). The hypothesis explored here is that these observed effects of attention could be a consequence of changes in the synchrony of local interneuron networks. We performed computer simulations of a Hodgkin-Huxley type neuron driven by a constant depolarizing current, I, representing visual stimulation and a modulatory inhibitory input representing the effects of attention via local interneuron networks. We observed that the neuron's firing rate and the coherence of its output spike train with the synaptic inputs was modulated by the degree of synchrony of the inhibitory inputs. When inhibitory synchrony increased, the coherence of spiking model neurons with the synaptic input increased, but the firing rate either increased or remained the same. The mean number of synchronous inhibitory inputs was a key determinant of the shape of the firing rate versus current (f-I) curves. For a large number of inhibitory inputs (approximately 50), the f-I curve saturated for large I and an increase in input synchrony resulted in a shift of sensitivity-the model neuron responded to weaker inputs I. For a small number (approximately 10), the f-I curves were non-saturating and an increase in input synchrony led to an increase in the gain of the response-the firing rate in response to the same input was multiplied by an approximately constant factor. The firing rate modulation with inhibitory synchrony was highest when the input network oscillated in the gamma frequency range. Thus, the observed changes in firing rate and coherence of neurons in the visual cortex could be controlled by top-down inputs that regulated the coherence in the activity of a local inhibitory network discharging at gamma frequencies.     

Identification of nonlinear systems using random impulse train inputs

Nonlinear systems that require discrete inputs can be characterized by using random impulse train (Poisson process) inputs. The method is analagous to the Wiener method for continuous input systems, where Gaussian white-noise is the input. In place of the Wiener functional expansion for the output of a continuous input system, a new series for discrete input systems is created by making certain restrictions on the integrals in a Volterra series. The kernels in the new series differ from the Wiener kernels, but also serve to identify a system and are simpler to compute. For systems whose impulse responses vary in amplitude but maintain a similar shape, one argument may be held fixed in each kernel. This simplifies the identification problem. As a test of the theory presented, the output of a hypothetical second order nonlinear system in response to a random impulse train stimulus was computer simulated. Kernels calculated from the simulated data agreed with theoretical predictions. The Poisson impulse train method is applicable to any system whose input can be delivered in discrete pulses. It is particularly suited to neuronal synaptic systems when the pattern of input nerve impulses can be made random.     

The chronotron: a neuron that learns to fire temporally precise spike patterns

In many cases, neurons process information carried by the precise timings of spikes. Here we show how neurons can learn to generate specific temporally precise output spikes in response to input patterns of spikes having precise timings, thus processing and memorizing information that is entirely temporally coded, both as input and as output. We introduce two new supervised learning rules for spiking neurons with temporal coding of information (chronotrons), one that provides high memory capacity (E-learning), and one that has a higher biological plausibility (I-learning). With I-learning, the neuron learns to fire the target spike trains through synaptic changes that are proportional to the synaptic currents at the timings of real and target output spikes. We study these learning rules in computer simulations where we train integrate-and-fire neurons. Both learning rules allow neurons to fire at the desired timings, with sub-millisecond precision. We show how chronotrons can learn to classify their inputs, by firing identical, temporally precise spike trains for different inputs belonging to the same class. When the input is noisy, the classification also leads to noise reduction. We compute lower bounds for the memory capacity of chronotrons and explore the influence of various parameters on chronotrons' performance. The chronotrons can model neurons that encode information in the time of the first spike relative to the onset of salient stimuli or neurons in oscillatory networks that encode information in the phases of spikes relative to the background oscillation. Our results show that firing one spike per cycle optimizes memory capacity in neurons encoding information in the phase of firing relative to a background rhythm.     

Modelling spatiotemporal properties of directionally sensitive multi-input single-output systems

The dynamics of directionally tuned linear multi-input single-output systems varies generally as a function of the spatial orientation of the inputs. A linear system receiving directionally specific inputs is represented by a linear combination of the respective input transfer functions. The input-output behaviour of such systems can be described by a vector transfer function which specifies the polarization directions of the system in real space. These directions, which can be either one (unidirectional vector transfer function) or two (bidirectional vector transfer function) but never three, are obtained by computing the eigenvectors and eigenvalues of the system matrix that is defined by the gain and phase values of the system's response to harmonic stimulation directed along three orthogonal directions in space. The spatial tuning behaviour is determined by the quadratic form associated with the system matrix. Neuronal systems with bidirectional vector transfer functions process input information in a plane-specific way and exhibit novel characteristics, very much different from those of systems with unidirectional vector transfer functions.     

Balanced synaptic input shapes the correlation between neural spike trains

Stimulus properties, attention, and behavioral context influence correlations between the spike times produced by a pair of neurons. However, the biophysical mechanisms that modulate these correlations are poorly understood. With a combined theoretical and experimental approach, we show that the rate of balanced excitatory and inhibitory synaptic input modulates the magnitude and timescale of pairwise spike train correlation. High rate synaptic inputs promote spike time synchrony rather than long timescale spike rate correlations, while low rate synaptic inputs produce opposite results. This correlation shaping is due to a combination of enhanced high frequency input transfer and reduced firing rate gain in the high input rate state compared to the low state. Our study extends neural modulation from single neuron responses to population activity, a necessary step in understanding how the dynamics and processing of neural activity change across distinct brain states.     

The response of a nerve cylinder to spatially distributed white noise inputs

The response of a passive nerve cylinder (or dendritic tree in the equivalent cylinder representation) to random white noise input currents is determined. Results for the mean, variance and covariance of the depolarization are obtained for an arbitrary number of independent spatially distributed inputs. The case of a cylinder with sealed ends is considered in detail. The differences that arise when the input currents are distributed over a small but finite region of space instead of concentrated at a point are investigated. In the case of distributed inputs, the expectation is smoother near the stimulus and the variance becomes finite over the entire cable length including the region of the applied stimulus. Away from the stimulus, there are no appreciable differences between the responses for the two cases. The interaction between an excitatory input and an inhibitory input at various locations is examined and one case of more than two inputs is also analysed to study effects which could not have been discerned from point models for a neuron with random inputs.     

A universal RNAi-based logic evaluator that operates in mammalian cells

Molecular automata that combine sensing, computation and actuation enable programmable manipulation of biological systems. We use RNA interference (RNAi) in human kidney cells to construct a molecular computing core that implements general Boolean logic to make decisions based on endogenous molecular inputs. The state of an endogenous input is encoded by the presence or absence of 'mediator' small interfering RNAs (siRNAs). The encoding rules, combined with a specific arrangement of the siRNA targets in a synthetic gene network, allow direct evaluation of any Boolean expression in standard forms using siRNAs and indirect evaluation using endogenous inputs. We demonstrate direct evaluation of expressions with up to five logic variables. Implementation of the encoding rules through sensory up- and down-regulatory links between the inputs and siRNA mediators will allow arbitrary Boolean decision-making using these inputs.