Modeling Compositionality by Dynamic Binding of Synfire Chains

This paper examines the feasibility of manifesting compositionality by a system of synfire chains. Compositionality is the ability to construct mental representations, hierarchically, in terms of parts and their relations. We show that synfire chains may synchronize their waves when a few orderly cross links are available. We propose that synchronization among synfire chains can be used for binding component into a whole. Such synchronization is shown both for detailed simulations, and by numerical analysis of the propagation of a wave along a synfire chain. We show that global inhibition may prevent spurious synchronization among synfire chains. We further show that selecting which synfire chains may synchronize to which others may be improved by including inhibitory neurons in the synfire pools. Finally we show that in a hierarchical system of synfire chains, a part-binding problem may be resolved, and that such a system readily demonstrates the property of priming. We compare the properties of our system with the general requirements for neural networks that demonstrate compositionality.     

Synconset Waves and Chains: Spiking Onsets in Synchronous Populations Predict and Are Predicted by Network Structure

Synfire waves are propagating spike packets in synfire chains, which are feedforward chains embedded in random networks. Although synfire waves have proved to be effective quantification for network activity with clear relations to network structure, their utilities are largely limited to feedforward networks with low background activity. To overcome these shortcomings, we describe a novel generalisation of synfire waves, and define ‘synconset wave’ as a cascade of first spikes within a synchronisation event. Synconset waves would occur in ‘synconset chains’, which are feedforward chains embedded in possibly heavily recurrent networks with heavy background activity. We probed the utility of synconset waves using simulation of single compartment neuron network models with biophysically realistic conductances, and demonstrated that the spread of synconset waves directly follows from the network connectivity matrix and is modulated by top-down inputs and the resultant oscillations. Such synconset profiles lend intuitive insights into network organisation in terms of connection probabilities between various network regions rather than an adjacency matrix. To test this intuition, we develop a Bayesian likelihood function that quantifies the probability that an observed synfire wave was caused by a given network. Further, we demonstrate it''s utility in the inverse problem of identifying the network that caused a given synfire wave. This method was effective even in highly subsampled networks where only a small subset of neurons were accessible, thus showing it''s utility in experimental estimation of connectomes in real neuronal-networks. Together, we propose synconset chains/waves as an effective framework for understanding the impact of network structure on function, and as a step towards developing physiology-driven network identification methods. Finally, as synconset chains extend the utilities of synfire chains to arbitrary networks, we suggest utilities of our framework to several aspects of network physiology including cell assemblies, population codes, and oscillatory synchrony.     

High-capacity embedding of synfire chains in a cortical network model

Synfire chains, sequences of pools linked by feedforward connections, support the propagation of precisely timed spike sequences, or synfire waves. An important question remains, how synfire chains can efficiently be embedded in cortical architecture. We present a model of synfire chain embedding in a cortical scale recurrent network using conductance-based synapses, balanced chains, and variable transmission delays. The network attains substantially higher embedding capacities than previous spiking neuron models and allows all its connections to be used for embedding. The number of waves in the model is regulated by recurrent background noise. We computationally explore the embedding capacity limit, and use a mean field analysis to describe the equilibrium state. Simulations confirm the mean field analysis over broad ranges of pool sizes and connectivity levels; the number of pools embedded in the system trades off against the firing rate and the number of waves. An optimal inhibition level balances the conflicting requirements of stable synfire propagation and limited response to background noise. A simplified analysis shows that the present conductance-based synapses achieve higher contrast between the responses to synfire input and background noise compared to current-based synapses, while regulation of wave numbers is traced to the use of variable transmission delays.     

Two’s company,three (or more) is a simplex

As a candidate mechanism of neural representation, large numbers of synfire chains can efficiently be embedded in a balanced recurrent cortical network model. Here we study a model in which multiple synfire chains of variable strength are randomly coupled together to form a recurrent system. The system can be implemented both as a large-scale network of integrate-and-fire neurons and as a reduced model. The latter has binary-state pools as basic units but is otherwise isomorphic to the large-scale model, and provides an efficient tool for studying its behavior. Both the large-scale system and its reduced counterpart are able to sustain ongoing endogenous activity in the form of synfire waves, the proliferation of which is regulated by negative feedback caused by collateral noise. Within this equilibrium, diverse repertoires of ongoing activity are observed, including meta-stability and multiple steady states. These states arise in concert with an effective connectivity structure (ECS). The ECS admits a family of effective connectivity graphs (ECGs), parametrized by the mean global activity level. Of these graphs, the strongly connected components and their associated out-components account to a large extent for the observed steady states of the system. These results imply a notion of dynamic effective connectivity as governing neural computation with synfire chains, and related forms of cortical circuitry with complex topologies.     

Development of neural circuitry for precise temporal sequences through spontaneous activity, axon remodeling, and synaptic plasticity

Temporally precise sequences of neuronal spikes that span hundreds of milliseconds are observed in many brain areas, including songbird premotor nucleus, cat visual cortex, and primary motor cortex. Synfire chains-networks in which groups of neurons are connected via excitatory synapses into a unidirectional chain-are thought to underlie the generation of such sequences. It is unknown, however, how synfire chains can form in local neural circuits, especially for long chains. Here, we show through computer simulation that long synfire chains can develop through spike-time dependent synaptic plasticity and axon remodeling-the pruning of prolific weak connections that follows the emergence of a finite number of strong connections. The formation process begins with a random network. A subset of neurons, called training neurons, intermittently receive superthreshold external input. Gradually, a synfire chain emerges through a recruiting process, in which neurons within the network connect to the tail of the chain started by the training neurons. The model is robust to varying parameters, as well as natural events like neuronal turnover and massive lesions. Our model suggests that long synfire chain can form during the development through self-organization, and axon remodeling, ubiquitous in developing neural circuits, is essential in the process.     

Graded,Dynamically Routable Information Processing with Synfire-Gated Synfire Chains

Coherent neural spiking and local field potentials are believed to be signatures of the binding and transfer of information in the brain. Coherent activity has now been measured experimentally in many regions of mammalian cortex. Recently experimental evidence has been presented suggesting that neural information is encoded and transferred in packets, i.e., in stereotypical, correlated spiking patterns of neural activity. Due to their relevance to coherent spiking, synfire chains are one of the main theoretical constructs that have been appealed to in order to describe coherent spiking and information transfer phenomena. However, for some time, it has been known that synchronous activity in feedforward networks asymptotically either approaches an attractor with fixed waveform and amplitude, or fails to propagate. This has limited the classical synfire chain’s ability to explain graded neuronal responses. Recently, we have shown that pulse-gated synfire chains are capable of propagating graded information coded in mean population current or firing rate amplitudes. In particular, we showed that it is possible to use one synfire chain to provide gating pulses and a second, pulse-gated synfire chain to propagate graded information. We called these circuits synfire-gated synfire chains (SGSCs). Here, we present SGSCs in which graded information can rapidly cascade through a neural circuit, and show a correspondence between this type of transfer and a mean-field model in which gating pulses overlap in time. We show that SGSCs are robust in the presence of variability in population size, pulse timing and synaptic strength. Finally, we demonstrate the computational capabilities of SGSC-based information coding by implementing a self-contained, spike-based, modular neural circuit that is triggered by streaming input, processes the input, then makes a decision based on the processed information and shuts itself down.     

Synchronization without oscillatory neurons

Experimental results suggest that neurons in the cortex synchronize their action potentials on the millisecond time scale. More importantly this binding expresses functional relationships between the neurons. A model of neuronal interactions is proposed in which simultaneous discharges of neurons develop through specialized synaptic circuits. As an important prerequisite for this synchronization it is demonstrated that SynFire chains, generating different levels of excitation, propagate their activity waves at distinct velocities. Two chains were coupled by excitatory synapses and their activity was initiated at different times. Due to synaptic interactions, activity in the earlier-initiated chain accelerates propagation in the other chain until the two activity waves are synchronized. Compared with several neural network models with oscillatory units, physiologically more plausible neurons are simulated. It is still under debate whether neurons in the cortex show oscillatory dischargesper se. In particular, a high rate of noise relative to very weak synaptic gains cannot impair our results in the neural network simulations.     

Spatial and temporal stochastic interaction in neuronal assemblies

Summary The observation of various types of spatio-temporal correlations in spike-patterns of multiple cortical neurons has shifted attention from rate coding paradigms to computational processes based on the precise timing of spikes in neuronal ensembles. In the present work we develop the notion of “spatial” and “temporal interaction” which provides measures for statistical dependences in coupled stochastic processes like multiple unit spike trains. We show that the classical Willshaw network and Abeles’ synfire chain model both reveal a moderate spatial interaction, but only the synfire chain model reveals a positive temporal interaction, too. Systems that maximize temporal interaction are shown to be almost deterministic globally, but posses almost unpredictable firing behavior on the single unit level.     

ASSET: Analysis of Sequences of Synchronous Events in Massively Parallel Spike Trains

With the ability to observe the activity from large numbers of neurons simultaneously using modern recording technologies, the chance to identify sub-networks involved in coordinated processing increases. Sequences of synchronous spike events (SSEs) constitute one type of such coordinated spiking that propagates activity in a temporally precise manner. The synfire chain was proposed as one potential model for such network processing. Previous work introduced a method for visualization of SSEs in massively parallel spike trains, based on an intersection matrix that contains in each entry the degree of overlap of active neurons in two corresponding time bins. Repeated SSEs are reflected in the matrix as diagonal structures of high overlap values. The method as such, however, leaves the task of identifying these diagonal structures to visual inspection rather than to a quantitative analysis. Here we present ASSET (Analysis of Sequences of Synchronous EvenTs), an improved, fully automated method which determines diagonal structures in the intersection matrix by a robust mathematical procedure. The method consists of a sequence of steps that i) assess which entries in the matrix potentially belong to a diagonal structure, ii) cluster these entries into individual diagonal structures and iii) determine the neurons composing the associated SSEs. We employ parallel point processes generated by stochastic simulations as test data to demonstrate the performance of the method under a wide range of realistic scenarios, including different types of non-stationarity of the spiking activity and different correlation structures. Finally, the ability of the method to discover SSEs is demonstrated on complex data from large network simulations with embedded synfire chains. Thus, ASSET represents an effective and efficient tool to analyze massively parallel spike data for temporal sequences of synchronous activity.     

Language models based on Hebbian cell assemblies.

This paper demonstrates how associative neural networks as standard models for Hebbian cell assemblies can be extended to implement language processes in large-scale brain simulations. To this end the classical auto- and hetero-associative paradigms of attractor nets and synfire chains (SFCs) are combined and complemented by conditioned associations as a third principle which allows for the implementation of complex graph-like transition structures between assemblies. We show example simulations of a multiple area network for object-naming, which categorises objects in a visual hierarchy and generates different specific syntactic motor sequences ("words") in response. The formation of cell assemblies due to ongoing plasticity in a multiple area network for word learning is studied afterwards. Simulations show how assemblies can form by means of percolating activity across auditory and motor-related language areas, a process supported by rhythmic, synchronized propagating waves through the network. Simulations further reproduce differences in own EEG&MEG experiments between responses to word- versus non-word stimuli in human subjects.     

Wave-Processing of Long-Scale Information by Neuronal Chains

Investigation of mechanisms of information handling in neural assemblies involved in computational and cognitive tasks is a challenging problem. Synergetic cooperation of neurons in time domain, through synchronization of firing of multiple spatially distant neurons, has been widely spread as the main paradigm. Complementary, the brain may also employ information coding and processing in spatial dimension. Then, the result of computation depends also on the spatial distribution of long-scale information. The latter bi-dimensional alternative is notably less explored in the literature. Here, we propose and theoretically illustrate a concept of spatiotemporal representation and processing of long-scale information in laminar neural structures. We argue that relevant information may be hidden in self-sustained traveling waves of neuronal activity and then their nonlinear interaction yields efficient wave-processing of spatiotemporal information. Using as a testbed a chain of FitzHugh-Nagumo neurons, we show that the wave-processing can be achieved by incorporating into the single-neuron dynamics an additional voltage-gated membrane current. This local mechanism provides a chain of such neurons with new emergent network properties. In particular, nonlinear waves as a carrier of long-scale information exhibit a variety of functionally different regimes of interaction: from complete or asymmetric annihilation to transparent crossing. Thus neuronal chains can work as computational units performing different operations over spatiotemporal information. Exploiting complexity resonance these composite units can discard stimuli of too high or too low frequencies, while selectively compress those in the natural frequency range. We also show how neuronal chains can contextually interpret raw wave information. The same stimulus can be processed differently or identically according to the context set by a periodic wave train injected at the opposite end of the chain.     

Interaction between spiral and paced waves in cardiac tissue

For prevention of lethal arrhythmias, patients at risk receive implantable cardioverter-defibrillators, which use high-frequency antitachycardia pacing (ATP) to convert tachycardias to a normal rhythm. One of the suggested ATP mechanisms involves paced-induced drift of rotating waves followed by their collision with the boundary of excitable tissue. This study provides direct experimental evidence of this mechanism. In monolayers of neonatal rat cardiomyocytes in which rotating waves of activity were initiated by premature stimuli, we used the Ca(2+)-sensitive indicator fluo 4 to observe propagating wave patterns. The interaction of the spiral tip with a paced wave was then monitored at a high spatial resolution. In the course of the experiments, we observed spiral wave pinning to local heterogeneities within the myocyte layer. High-frequency pacing led, in a majority of cases, to successful termination of spiral activity. Our data show that 1) stable spiral waves in cardiac monolayers tend to be pinned to local heterogeneities or areas of altered conduction, 2) overdrive pacing can shift a rotating wave from its original site, and 3) the wave break, formed as a result of interaction between the spiral tip and a paced wave front, moves by a paced-induced drift mechanism to an area where it may become unstable or collide with a boundary. The data were complemented by numerical simulations, which was used to further analyze experimentally observed behavior.     

A neurocomputational model for optimal temporal processing

Humans can estimate the duration of intervals of time, and psychophysical experiments show that these estimations are subject to timing errors. According to standard theories of timing, these errors increase linearly with the interval to be estimated (Weber's law), and both at longer and shorter intervals, deviations from linearity are reported. This is not easily reconciled with the accumulation of neuronal noise, which would only lead to an increase with the square root of the interval. Here, we offer a neuronal model which explains the form of the error function as a result of a constrained optimization process. The model consists of a number of synfire chains with different transmission times, which project onto a set of readout neurons. We show that an increase in the transmission time corresponds to a superlinear increase of the timing errors. Under the assumption of a fixed chain length, the experimentally observed error function emerges from optimal selection of chains for each given interval. Furthermore, we show how this optimal selection could be implemented by competitive spike-timing dependent plasticity in the connections from the chains to the readout network, and discuss implications of our model on selective temporal learning and possible neural architectures of interval timing.     

Compositionality of arm movements can be realized by propagating synchrony

We present a biologically plausible spiking neuronal network model of free monkey scribbling that reproduces experimental findings on cortical activity and the properties of the scribbling trajectory. The model is based on the idea that synfire chains can encode movement primitives. Here, we map the propagation of activity in a chain to a linearly evolving preferred velocity, which results in parabolic segments that fulfill the two-thirds power law. Connections between chains that match the final velocity of one encoded primitive to the initial velocity of the next allow the composition of random sequences of primitives with smooth transitions. The model provides an explanation for the segmentation of the trajectory and the experimentally observed deviations of the trajectory from the parabolic shape at primitive transition sites. Furthermore, the model predicts low frequency oscillations (<10 Hz) of the motor cortex local field potential during ongoing movements and increasing firing rates of non-specific motor cortex neurons before movement onset.     

A thalamo-cortical neural mass model for the simulation of brain rhythms during sleep

Cortico-thalamic interactions are known to play a pivotal role in many brain phenomena, including sleep, attention, memory consolidation and rhythm generation. Hence, simple mathematical models that can simulate the dialogue between the cortex and the thalamus, at a mesoscopic level, have a great cognitive value. In the present work we describe a neural mass model of a cortico-thalamic module, based on neurophysiological mechanisms. The model includes two thalamic populations (a thalamo-cortical relay cell population, TCR, and its related thalamic reticular nucleus, TRN), and a cortical column consisting of four connected populations (pyramidal neurons, excitatory interneurons, inhibitory interneurons with slow and fast kinetics). Moreover, thalamic neurons exhibit two firing modes: bursting and tonic. Finally, cortical synapses among pyramidal neurons incorporate a disfacilitation mechanism following prolonged activity. Simulations show that the model is able to mimic the different patterns of rhythmic activity in cortical and thalamic neurons (beta and alpha waves, spindles, delta waves, K-complexes, slow sleep waves) and their progressive changes from wakefulness to deep sleep, by just acting on modulatory inputs. Moreover, simulations performed by providing short sensory inputs to the TCR show that brain rhythms during sleep preserve the cortex from external perturbations, still allowing a high cortical activity necessary to drive synaptic plasticity and memory consolidation. In perspective, the present model may be used within larger cortico-thalamic networks, to gain a deeper understanding of mechanisms beneath synaptic changes during sleep, to investigate the specific role of brain rhythms, and to explore cortical synchronization achieved via thalamic influences.     

A Comparative Study of Early Afterdepolarization-Mediated Fibrillation in Two Mathematical Models for Human Ventricular Cells

Early afterdepolarizations (EADs), which are abnormal oscillations of the membrane potential at the plateau phase of an action potential, are implicated in the development of cardiac arrhythmias like Torsade de Pointes. We carry out extensive numerical simulations of the TP06 and ORd mathematical models for human ventricular cells with EADs. We investigate the different regimes in both these models, namely, the parameter regimes where they exhibit (1) a normal action potential (AP) with no EADs, (2) an AP with EADs, and (3) an AP with EADs that does not go back to the resting potential. We also study the dependence of EADs on the rate of at which we pace a cell, with the specific goal of elucidating EADs that are induced by slow or fast rate pacing. In our simulations in two- and three-dimensional domains, in the presence of EADs, we find the following wave types: (A) waves driven by the fast sodium current and the L-type calcium current (Na-Ca-mediated waves); (B) waves driven only by the L-type calcium current (Ca-mediated waves); (C) phase waves, which are pseudo-travelling waves. Furthermore, we compare the wave patterns of the various wave-types (Na-Ca-mediated, Ca-mediated, and phase waves) in both these models. We find that the two models produce qualitatively similar results in terms of exhibiting Na-Ca-mediated wave patterns that are more chaotic than those for the Ca-mediated and phase waves. However, there are quantitative differences in the wave patterns of each wave type. The Na-Ca-mediated waves in the ORd model show short-lived spirals but the TP06 model does not. The TP06 model supports more Ca-mediated spirals than those in the ORd model, and the TP06 model exhibits more phase-wave patterns than does the ORd model.     

Time representing cortical activities: two models inspired by prefrontal persistent activity

 Timing information in the range of seconds is significantly correlated with our behavior. There is growing interest in the cognitive behaviors that rely on perception, comparison, or generation of timing. However, little is known about the neural mechanisms underlying such behaviors. Here we model two different neural mechanisms to represent timing information in the range of seconds. In one model, a recurrent network of bistable spiking neurons shows a quasistable state that is initiated by a brief input and typically lasts for a few to several seconds. The duration of this quasistable activity may be regarded as the neural representation of internal time obeying a psychophysical law of time recognition. Another model uses synfire chains to provide the timing information necessary for predicting the times of anticipated events. In this model, the neurons projected to by multiple synfire chains are conditioned to fire synchronously at the times when an external event (GO signal) is expected. The conditioning is accomplished by spike-timing-dependent plasticity. The two models are inspired by the prefrontal activities of the monkeys engaging in different timing-information-related tasks. Thus, this cortical region may provide the timing information required for organizing various behaviors. Received: 12 March 2002 / Accepted in revised form: 26 November 2002 / Published online: 28 March 2003 Correspondence to: T. Fukai (e-mail: tfukai@eng.tamagawa.ac.jp, Tel.: +81-42-7398434, Fax: +81-42-7397135) Acknowledgements. K. Kitano was supported by Japan Society for the Promotion of Science.     

Modeling AFM-induced PEVK extension and the reversible unfolding of Ig/FNIII domains in single and multiple titin molecules

Molecular elasticity is associated with a select number of polypeptides and proteins, such as titin, Lustrin A, silk fibroin, and spider silk dragline protein. In the case of titin, the globular (Ig) and non-globular (PEVK) regions act as extensible springs under stretch; however, their unfolding behavior and force extension characteristics are different. Using our time-dependent macroscopic method for simulating AFM-induced titin Ig domain unfolding and refolding, we simulate the extension and relaxation of hypothetical titin chains containing Ig domains and a PEVK region. Two different models are explored: 1) a series-linked WLC expression that treats the PEVK region as a distinct entropic spring, and 2) a summation of N single WLC expressions that simulates the extension and release of a discrete number of parallel titin chains containing constant or variable amounts of PEVK. In addition to these simulations, we also modeled the extension of a hypothetical PEVK domain using a linear Hooke's spring model to account for "enthalpic" contributions to PEVK elasticity. We find that the modified WLC simulations feature chain length compensation, Ig domain unfolding/refolding, and force-extension behavior that more closely approximate AFM, laser tweezer, and immunolocalization experimental data. In addition, our simulations reveal the following: 1) PEVK extension overlaps with the onset of Ig domain unfolding, and 2) variations in PEVK content within a titin chain ensemble lead to elastic diversity within that ensemble.