Two’s company,three (or more) is a simplex

As a candidate mechanism of neural representation, large numbers of synfire chains can efficiently be embedded in a balanced recurrent cortical network model. Here we study a model in which multiple synfire chains of variable strength are randomly coupled together to form a recurrent system. The system can be implemented both as a large-scale network of integrate-and-fire neurons and as a reduced model. The latter has binary-state pools as basic units but is otherwise isomorphic to the large-scale model, and provides an efficient tool for studying its behavior. Both the large-scale system and its reduced counterpart are able to sustain ongoing endogenous activity in the form of synfire waves, the proliferation of which is regulated by negative feedback caused by collateral noise. Within this equilibrium, diverse repertoires of ongoing activity are observed, including meta-stability and multiple steady states. These states arise in concert with an effective connectivity structure (ECS). The ECS admits a family of effective connectivity graphs (ECGs), parametrized by the mean global activity level. Of these graphs, the strongly connected components and their associated out-components account to a large extent for the observed steady states of the system. These results imply a notion of dynamic effective connectivity as governing neural computation with synfire chains, and related forms of cortical circuitry with complex topologies.     

Modeling Compositionality by Dynamic Binding of Synfire Chains

This paper examines the feasibility of manifesting compositionality by a system of synfire chains. Compositionality is the ability to construct mental representations, hierarchically, in terms of parts and their relations. We show that synfire chains may synchronize their waves when a few orderly cross links are available. We propose that synchronization among synfire chains can be used for binding component into a whole. Such synchronization is shown both for detailed simulations, and by numerical analysis of the propagation of a wave along a synfire chain. We show that global inhibition may prevent spurious synchronization among synfire chains. We further show that selecting which synfire chains may synchronize to which others may be improved by including inhibitory neurons in the synfire pools. Finally we show that in a hierarchical system of synfire chains, a part-binding problem may be resolved, and that such a system readily demonstrates the property of priming. We compare the properties of our system with the general requirements for neural networks that demonstrate compositionality.     

Development of neural circuitry for precise temporal sequences through spontaneous activity, axon remodeling, and synaptic plasticity

Temporally precise sequences of neuronal spikes that span hundreds of milliseconds are observed in many brain areas, including songbird premotor nucleus, cat visual cortex, and primary motor cortex. Synfire chains-networks in which groups of neurons are connected via excitatory synapses into a unidirectional chain-are thought to underlie the generation of such sequences. It is unknown, however, how synfire chains can form in local neural circuits, especially for long chains. Here, we show through computer simulation that long synfire chains can develop through spike-time dependent synaptic plasticity and axon remodeling-the pruning of prolific weak connections that follows the emergence of a finite number of strong connections. The formation process begins with a random network. A subset of neurons, called training neurons, intermittently receive superthreshold external input. Gradually, a synfire chain emerges through a recruiting process, in which neurons within the network connect to the tail of the chain started by the training neurons. The model is robust to varying parameters, as well as natural events like neuronal turnover and massive lesions. Our model suggests that long synfire chain can form during the development through self-organization, and axon remodeling, ubiquitous in developing neural circuits, is essential in the process.     

A Model for Representing the Dynamics of a System of Synfire Chains

Competitive synchronization among synfire chains may model the dynamics of binding and compositionality. Typically, such models require simulations of hundreds of thousands of neurons. Here we show that the behavior of such large systems can be numerically analyzed by representing the neuronal activity in a synfire chain as a wave. The position and velocity of waves are the only parameters needed to represent the neural activity within a synfire chain. With this wave model we describe how waves are generated, decay, interact within a single chain and among chains. The behavior of the wave model is compared to the behavior of detailed simulations of synfire chains with no qualitative difference. We show that interacting waves tend to become locked to each other (wave synchronization). Finally we prove that: (1) Within a system of many synfire chains with symmetric interchain connections, as long as waves do not fade away or become fully synchronized, the total synchrony among waves can only increase (or stay constant), but never decrease. (2) A wave that increases its speed during the synchronization process becomes more stable.     

High-capacity embedding of synfire chains in a cortical network model

Synfire chains, sequences of pools linked by feedforward connections, support the propagation of precisely timed spike sequences, or synfire waves. An important question remains, how synfire chains can efficiently be embedded in cortical architecture. We present a model of synfire chain embedding in a cortical scale recurrent network using conductance-based synapses, balanced chains, and variable transmission delays. The network attains substantially higher embedding capacities than previous spiking neuron models and allows all its connections to be used for embedding. The number of waves in the model is regulated by recurrent background noise. We computationally explore the embedding capacity limit, and use a mean field analysis to describe the equilibrium state. Simulations confirm the mean field analysis over broad ranges of pool sizes and connectivity levels; the number of pools embedded in the system trades off against the firing rate and the number of waves. An optimal inhibition level balances the conflicting requirements of stable synfire propagation and limited response to background noise. A simplified analysis shows that the present conductance-based synapses achieve higher contrast between the responses to synfire input and background noise compared to current-based synapses, while regulation of wave numbers is traced to the use of variable transmission delays.     

Synconset Waves and Chains: Spiking Onsets in Synchronous Populations Predict and Are Predicted by Network Structure

Synfire waves are propagating spike packets in synfire chains, which are feedforward chains embedded in random networks. Although synfire waves have proved to be effective quantification for network activity with clear relations to network structure, their utilities are largely limited to feedforward networks with low background activity. To overcome these shortcomings, we describe a novel generalisation of synfire waves, and define ‘synconset wave’ as a cascade of first spikes within a synchronisation event. Synconset waves would occur in ‘synconset chains’, which are feedforward chains embedded in possibly heavily recurrent networks with heavy background activity. We probed the utility of synconset waves using simulation of single compartment neuron network models with biophysically realistic conductances, and demonstrated that the spread of synconset waves directly follows from the network connectivity matrix and is modulated by top-down inputs and the resultant oscillations. Such synconset profiles lend intuitive insights into network organisation in terms of connection probabilities between various network regions rather than an adjacency matrix. To test this intuition, we develop a Bayesian likelihood function that quantifies the probability that an observed synfire wave was caused by a given network. Further, we demonstrate it''s utility in the inverse problem of identifying the network that caused a given synfire wave. This method was effective even in highly subsampled networks where only a small subset of neurons were accessible, thus showing it''s utility in experimental estimation of connectomes in real neuronal-networks. Together, we propose synconset chains/waves as an effective framework for understanding the impact of network structure on function, and as a step towards developing physiology-driven network identification methods. Finally, as synconset chains extend the utilities of synfire chains to arbitrary networks, we suggest utilities of our framework to several aspects of network physiology including cell assemblies, population codes, and oscillatory synchrony.     

Time representing cortical activities: two models inspired by prefrontal persistent activity

 Timing information in the range of seconds is significantly correlated with our behavior. There is growing interest in the cognitive behaviors that rely on perception, comparison, or generation of timing. However, little is known about the neural mechanisms underlying such behaviors. Here we model two different neural mechanisms to represent timing information in the range of seconds. In one model, a recurrent network of bistable spiking neurons shows a quasistable state that is initiated by a brief input and typically lasts for a few to several seconds. The duration of this quasistable activity may be regarded as the neural representation of internal time obeying a psychophysical law of time recognition. Another model uses synfire chains to provide the timing information necessary for predicting the times of anticipated events. In this model, the neurons projected to by multiple synfire chains are conditioned to fire synchronously at the times when an external event (GO signal) is expected. The conditioning is accomplished by spike-timing-dependent plasticity. The two models are inspired by the prefrontal activities of the monkeys engaging in different timing-information-related tasks. Thus, this cortical region may provide the timing information required for organizing various behaviors. Received: 12 March 2002 / Accepted in revised form: 26 November 2002 / Published online: 28 March 2003 Correspondence to: T. Fukai (e-mail: tfukai@eng.tamagawa.ac.jp, Tel.: +81-42-7398434, Fax: +81-42-7397135) Acknowledgements. K. Kitano was supported by Japan Society for the Promotion of Science.     

Activity dynamics and propagation of synchronous spiking in locally connected random networks

Random network models have been a popular tool for investigating cortical network dynamics. On the scale of roughly a cubic millimeter of cortex, containing about 100,000 neurons, cortical anatomy suggests a more realistic architecture. In this locally connected random network, the connection probability decreases in a Gaussian fashion with the distance between neurons. Here we present three main results from a simulation study of the activity dynamics in such networks. First, for a broad range of parameters these dynamics exhibit a stationary state of asynchronous network activity with irregular single-neuron spiking. This state can be used as a realistic model of ongoing network activity. Parametric dependence of this state and the nature of the network dynamics in other regimes are described. Second, a synchronous excitatory stimulus to a fraction of the neurons results in a strong activity response that easily dominates the network dynamics. And third, due to that activity response an embedding of a divergent-convergent feed-forward subnetwork (as in synfire chains) does not naturally lead to a stable propagation of synchronous activity in the subnetwork; this is in contrast to our earlier findings in isolated subnetworks of that type. Possible mechanisms for stabilizing the interplay of volleys of synchronous spikes and network dynamics by specific learning rules or generalizations of the subnetworks are discussed.     

Graded,Dynamically Routable Information Processing with Synfire-Gated Synfire Chains

Coherent neural spiking and local field potentials are believed to be signatures of the binding and transfer of information in the brain. Coherent activity has now been measured experimentally in many regions of mammalian cortex. Recently experimental evidence has been presented suggesting that neural information is encoded and transferred in packets, i.e., in stereotypical, correlated spiking patterns of neural activity. Due to their relevance to coherent spiking, synfire chains are one of the main theoretical constructs that have been appealed to in order to describe coherent spiking and information transfer phenomena. However, for some time, it has been known that synchronous activity in feedforward networks asymptotically either approaches an attractor with fixed waveform and amplitude, or fails to propagate. This has limited the classical synfire chain’s ability to explain graded neuronal responses. Recently, we have shown that pulse-gated synfire chains are capable of propagating graded information coded in mean population current or firing rate amplitudes. In particular, we showed that it is possible to use one synfire chain to provide gating pulses and a second, pulse-gated synfire chain to propagate graded information. We called these circuits synfire-gated synfire chains (SGSCs). Here, we present SGSCs in which graded information can rapidly cascade through a neural circuit, and show a correspondence between this type of transfer and a mean-field model in which gating pulses overlap in time. We show that SGSCs are robust in the presence of variability in population size, pulse timing and synaptic strength. Finally, we demonstrate the computational capabilities of SGSC-based information coding by implementing a self-contained, spike-based, modular neural circuit that is triggered by streaming input, processes the input, then makes a decision based on the processed information and shuts itself down.     

Compositionality of arm movements can be realized by propagating synchrony

We present a biologically plausible spiking neuronal network model of free monkey scribbling that reproduces experimental findings on cortical activity and the properties of the scribbling trajectory. The model is based on the idea that synfire chains can encode movement primitives. Here, we map the propagation of activity in a chain to a linearly evolving preferred velocity, which results in parabolic segments that fulfill the two-thirds power law. Connections between chains that match the final velocity of one encoded primitive to the initial velocity of the next allow the composition of random sequences of primitives with smooth transitions. The model provides an explanation for the segmentation of the trajectory and the experimentally observed deviations of the trajectory from the parabolic shape at primitive transition sites. Furthermore, the model predicts low frequency oscillations (<10 Hz) of the motor cortex local field potential during ongoing movements and increasing firing rates of non-specific motor cortex neurons before movement onset.     

Spatial and temporal stochastic interaction in neuronal assemblies

Summary The observation of various types of spatio-temporal correlations in spike-patterns of multiple cortical neurons has shifted attention from rate coding paradigms to computational processes based on the precise timing of spikes in neuronal ensembles. In the present work we develop the notion of “spatial” and “temporal interaction” which provides measures for statistical dependences in coupled stochastic processes like multiple unit spike trains. We show that the classical Willshaw network and Abeles’ synfire chain model both reveal a moderate spatial interaction, but only the synfire chain model reveals a positive temporal interaction, too. Systems that maximize temporal interaction are shown to be almost deterministic globally, but posses almost unpredictable firing behavior on the single unit level.     

A review of cell assemblies

Since the cell assembly (CA) was hypothesised, it has gained substantial support and is believed to be the neural basis of psychological concepts. A CA is a relatively small set of connected neurons, that through neural firing can sustain activation without stimulus from outside the CA, and is formed by learning. Extensive evidence from multiple single unit recording and other techniques provides support for the existence of CAs that have these properties, and that their neurons also spike with some degree of synchrony. Since the evidence is so broad and deep, the review concludes that CAs are all but certain. A model of CAs is introduced that is informal, but is broad enough to include, e.g. synfire chains, without including, e.g. holographic reduced representation. CAs are found in most cortical areas and in some sub-cortical areas, they are involved in psychological tasks including categorisation, short-term memory and long-term memory, and are central to other tasks including working memory. There is currently insufficient evidence to conclude that CAs are the neural basis of all concepts. A range of models have been used to simulate CA behaviour including associative memory and more process- oriented tasks such as natural language parsing. Questions involving CAs, e.g. memory persistence, CAs’ complex interactions with brain waves and learning, remain unanswered. CA research involves a wide range of disciplines including biology and psychology, and this paper reviews literature directly related to the CA, providing a basis of discussion for this interdisciplinary community on this important topic. Hopefully, this discussion will lead to more formal and accurate models of CAs that are better linked to neuropsychological data.     

ASSET: Analysis of Sequences of Synchronous Events in Massively Parallel Spike Trains

With the ability to observe the activity from large numbers of neurons simultaneously using modern recording technologies, the chance to identify sub-networks involved in coordinated processing increases. Sequences of synchronous spike events (SSEs) constitute one type of such coordinated spiking that propagates activity in a temporally precise manner. The synfire chain was proposed as one potential model for such network processing. Previous work introduced a method for visualization of SSEs in massively parallel spike trains, based on an intersection matrix that contains in each entry the degree of overlap of active neurons in two corresponding time bins. Repeated SSEs are reflected in the matrix as diagonal structures of high overlap values. The method as such, however, leaves the task of identifying these diagonal structures to visual inspection rather than to a quantitative analysis. Here we present ASSET (Analysis of Sequences of Synchronous EvenTs), an improved, fully automated method which determines diagonal structures in the intersection matrix by a robust mathematical procedure. The method consists of a sequence of steps that i) assess which entries in the matrix potentially belong to a diagonal structure, ii) cluster these entries into individual diagonal structures and iii) determine the neurons composing the associated SSEs. We employ parallel point processes generated by stochastic simulations as test data to demonstrate the performance of the method under a wide range of realistic scenarios, including different types of non-stationarity of the spiking activity and different correlation structures. Finally, the ability of the method to discover SSEs is demonstrated on complex data from large network simulations with embedded synfire chains. Thus, ASSET represents an effective and efficient tool to analyze massively parallel spike data for temporal sequences of synchronous activity.     

New attractor states for synchronous activity in synfire chains with excitatory and inhibitory coupling

 In a feedforward network of integrate-and-fire neurons, where the firing of each layer is synchronous (synfire chain), the final firing state of the network converges to two attractor states: either a full activation or complete fading of the tailing layers. In this article, we analyze various modes of pattern propagation in a synfire chain with random connection weights and delta-type postsynaptic currents. We predict analytically that when the input is fully synchronized and the network is noise free, varying the characteristics of the weights distribution would result in modes of behavior that are different from those described in the literature. These are convergence to fixed points, limit cycles, multiple periodic, and possibly chaotic dynamics. We checked our analytic results by computer simulation of the network, and showed that the above results can be generalized when the input is asynchronous and neurons are spontaneously active at low rates. Received: 27 July 2001 / Accepted in revised form: 23 October 2001     

A neurocomputational model for optimal temporal processing

Humans can estimate the duration of intervals of time, and psychophysical experiments show that these estimations are subject to timing errors. According to standard theories of timing, these errors increase linearly with the interval to be estimated (Weber's law), and both at longer and shorter intervals, deviations from linearity are reported. This is not easily reconciled with the accumulation of neuronal noise, which would only lead to an increase with the square root of the interval. Here, we offer a neuronal model which explains the form of the error function as a result of a constrained optimization process. The model consists of a number of synfire chains with different transmission times, which project onto a set of readout neurons. We show that an increase in the transmission time corresponds to a superlinear increase of the timing errors. Under the assumption of a fixed chain length, the experimentally observed error function emerges from optimal selection of chains for each given interval. Furthermore, we show how this optimal selection could be implemented by competitive spike-timing dependent plasticity in the connections from the chains to the readout network, and discuss implications of our model on selective temporal learning and possible neural architectures of interval timing.     

Stochastic emergence of repeating cortical motifs in spontaneous membrane potential fluctuations in vivo

It was recently discovered that subthreshold membrane potential fluctuations of cortical neurons can precisely repeat during spontaneous activity, seconds to minutes apart, both in brain slices and in anesthetized animals. These repeats, also called cortical motifs, were suggested to reflect a replay of sequential neuronal firing patterns. We searched for motifs in spontaneous activity, recorded from the rat barrel cortex and from the cat striate cortex of anesthetized animals, and found numerous repeating patterns of high similarity and repetition rates. To test their significance, various statistics were compared between physiological data and three different types of stochastic surrogate data that preserve dynamical characteristics of the recorded data. We found no evidence for the existence of deterministically generated cortical motifs. Rather, the stochastic properties of cortical motifs suggest that they appear by chance, as a result of the constraints imposed by the coarse dynamics of subthreshold ongoing activity.     

Response variability in the mammalian auditory cortex: an objection to feature detection?

Research strategy in the auditory system has tended to parallel that in the visual system, where neurons have been shown to respond selectively to specific stimulus parameters. Auditory neurons have been shown to be sensitive to changes in acoustic parameters, but only rarely have neurons been reported that respond exclusively to only one biologically significant sound. Even at higher levels of the auditory system very few cells have been found that could be described as "vocalization detectors." In addition, variability in responses to artificial sounds have been reported for auditory cortical neurons similar to the response variability that has been reported in the visual system. Recent evidence indicates that the responses of auditory cortical neurons to species-specific vocalizations can also be labile, varying in both strength and selectivity. This is especially true of the secondary auditory cortex. This variability, coupled with the lack of extreme specificity in the secondary auditory cortex, suggests that secondary cortical neurons are not well suited for the role of "vocalization detectors."     

System principles in the organization of neurons carrying out cortical inhibition

Some new data on neuronal and synaptic organization of sensorimotor cortical area in cat are obtained by a complex of morphological and electrophysiological methods. These data permit considering that direct afferent inhibition is ensured by thalamo-cortical neurons and neurons forming the callosal and association links. The recurrent and lateral inhibition are structurally realized through the ascending recurrent axon collaterals of pyramidal neurons forming links either with short-axon or with long-axon interneurons. Cortico-thalamic (cortico-fugal) inhibition may be performed either via descending cortico-thalamic neurons or via cortico-cortical ipsi- and contralateral neurons. The above mentioned neuronal chains may be considered as structural elements of more complex neuronal sets which ensure the inhibition at the cortical inputs, outputs and intracortically.     

Storing structured sparse memories in a multi-modular cortical network model

We investigate the properties of recently proposed “shotgun” sampling approach for the common inputs problem in the functional estimation of neuronal connectivity. We study the asymptotic correctness, the speed of convergence, and the data size requirements of such an approach. We show that the shotgun approach can be expected to allow the inference of complete connectivity matrix in large neuronal populations under some rather general conditions. However, we find that the posterior error of the shotgun connectivity estimator grows quickly with the size of unobserved neuronal populations, the square of average connectivity strength, and the square of observation sparseness. This implies that the shotgun connectivity estimation will require significantly larger amounts of neuronal activity data whenever the number of neurons in observed neuronal populations remains small. We present a numerical approach for solving the shotgun estimation problem in general settings and use it to demonstrate the shotgun connectivity inference in the examples of simulated synfire and weakly coupled cortical neuronal networks.