Conditions for noise reduction and stable encoding of spatial structure by cortical neural networks

Cortical circuits have been proposed to encode information by forming stable spatially structured attractors. Experimentally in the primary somatosensory cortex of the monkey, temporally invariant stimuli lead to spatially structured activity patterns. The purpose of this work is to study a recurrent cortical neural network model with lateral inhibition and examine what effect additive random noise has on the networks' ability to form stable spatially structured representations of the stimulus pattern. We show numerically that this network performs edge enhancement and forms statistically stationary, spatially structured responses when the lateral inhibition is of moderate strength. We then derive analytical conditions on the connectivity matrix that ensure stochasticly stable encoding of the stimulus spatial structure by the network. For stimuli whose strength falls in the near linear region of the sigmoid, we are able to give explicit conditions on the eigenvalues of the connection matrix. Finally, we prove that a network with a connection matrix, where the total excitation and inhibition impinging upon a neural unit are nearly balanced, will yield stable spatial attractor responses. Received: 16 October 1998 / Accepted in revised form: 25 November 1999     

Ansätze zur quantitativen Analyse der Nachrichtenaufnahme und -verarbeitung in biologischen Rezeptoren

The flux equilibrium theory, used for interpretating active and passive ion transport, can explain the generation of receptor potentials. In a model, driving forces and velocity coefficients are represented by the parameters of electric circuits. From these membrane models ionic fluxes can be calculated quantitatively on the basis of transport equations. These equations are derived from the theory of irreversible thermodynamic processes. Receptor models allow a simulation and prediction of the bioelectric potentials which were recorded by other authors in neuro-physiological experiments under various stimulus conditions. The information capacity of a single receptor channel is determined by the ionic flux and the stimulus parameters. In combination with the network of neuron models, receptor models can be used in a perception. The problems of on-off-activation and lateral inhibition were investigated with such a network.     

A neural network model for texture discrimination

A model of texture discrimination in visual cortex was built using a feedforward network with lateral interactions among relatively realistic spiking neural elements. The elements have various membrane currents, equilibrium potentials and time constants, with action potentials and synapses. The model is derived from the modified programs of MacGregor (1987). Gabor-like filters are applied to overlapping regions in the original image; the neural network with lateral excitatory and inhibitory interactions then compares and adjusts the Gabor amplitudes in order to produce the actual texture discrimination. Finally, a combination layer selects and groups various representations in the output of the network to form the final transformed image material. We show that both texture segmentation and detection of texture boundaries can be represented in the firing activity of such a network for a wide variety of synthetic to natural images. Performance details depend most strongly on the global balance of strengths of the excitatory and inhibitory lateral interconnections. The spatial distribution of lateral connective strengths has relatively little effect. Detailed temporal firing activities of single elements in the lateral connected network were examined under various stimulus conditions. Results show (as in area 17 of cortex) that a single element's response to image features local to its receptive field can be altered by changes in the global context.     

Parallel processing by a homogeneous group of coupled model neurons can enhance, reduce and generate signal correlations

 Correlated activities have been proposed as correlates of flexible association and assembly coding. We addressed the basic question of how signal correlations on parallel pathways are enhanced, reduced and generated by homogeneous groups of coupled neurons, and how this depends on the input activities and their interactions with internal coupling processes. For this we simulated a fully connected group of identical impulse-coded neurons with dynamic input and threshold processes and additive or multiplicative lateral coupling. Input signals were Gaussian white noise (GWN), completely independent or partially correlated on a subgroup of the parallel inputs. We show that in states of high average spike rates input-output correlations were weak while the network could generate correlated activities of stochastic, oscillatory and rhythmic bursting types depending exclusively on lateral coupling strength. In states of low average spike rates input-output correlations were high and the network could effectively enhance or reduce differences in spatial correlation applied to its parallel inputs. The correlation differences were more pronounced with multiplicative lateral coupling than with the additive interactions commonly used. As the different modes of correlation processing emerged already by global changes in the average spike rate and lateral coupling strength, we assume that in real cortical circuits changes in correlational processing may also be induced by unspecific modulations of activation and lateral coupling. Received: 11 December 1995 / Accepted in revised form: 29 November 1996     

Background-activity-dependent properties of a network model for working memory that incorporates cellular bistability

In models of working memory, transient stimuli are encoded by feature-selective persistent neural activity. Network models of working memory are also implicitly bistable. In the absence of a brief stimulus, only spontaneous, low-level, and presumably nonpatterned neural activity is seen. In many working-memory models, local recurrent excitation combined with long-range inhibition (Mexican hat coupling) can result in a network-induced, spatially localized persistent activity or “bump state” that coexists with a stable uniform state. There is now renewed interest in the concept that individual neurons might have some intrinsic ability to sustain persistent activity without recurrent network interactions. A recent visuospatial working-memory model (Camperi and Wang 1998) incorporates both intrinsic bistability of individual neurons within a firing rate network model and a single population of neurons on a ring with lateral inhibitory coupling. We have explored this model in more detail and have characterized the response properties with changes in background synaptic input I_o and stimulus width. We find that only a small range of I_o yields a working-memory-like coexistence of bump and uniform solutions that are both stable. There is a rather larger range where only the bump solution is stable that might correspond instead to a feature-selective long-term memory. Such a network therefore requires careful tuning to exhibit working-memory-like function. Interestingly, where bumps and uniform stable states coexist, we find a continuous family of stable bumps representing stimulus width. Thus, in the range of parameters corresponding to working memory, the model is capable of capturing a two-parameter family of stimulus features including both orientation and width.     

Spatiotemporal burst coding for extracting features of spatiotemporally varying stimuli

Encoding features of spatiotemporally varying stimuli is quite important for understanding the neural mechanisms of various sensory coding. Temporal coding can encode features of time-varying stimulus, and population coding with temporal coding is adequate for encoding spatiotemporal correlation of stimulus features into spatiotemporal activity of neurons. However, little is known about how spatiotemporal features of stimulus are encoded by spatiotemporal property of neural activity. To address this issue, we propose here a population coding with burst spikes, called here spatiotemporal burst (STB) coding. In STB coding, the temporal variation of stimuli is encoded by the precise onset timing of burst spike, and the spatiotemporal correlation of stimuli is emphasized by one specific aspect of burst firing, or spike packet followed by silent interval. To show concretely the role of STB coding, we study the electrosensory system of a weakly electric fish. Weakly electric fish must perceive the information about an object nearby by analyzing spatiotemporal modulations of electric field around it. On the basis of well-characterized circuitry, we constructed a neural network model of the electrosensory system. Here we show that STB coding encodes well the information of object distance and size by extracting the spatiotemporal correlation of the distorted electric field. The burst activity of electrosensory neurons is also affected by feedback signals through synaptic plasticity. We show that the control of burst activity caused by the synaptic plasticity leads to extracting the stimulus features depending on the stimulus context. Our results suggest that sensory systems use burst spikes as a unit of sensory coding in order to extract spatiotemporal features of stimuli from spatially distributed stimuli.     

A locally inhibited lateral neural network: on the fundamental features of a network consisting of neurons with a restricted range of interaction

A network model that consists of neurons with a restricted range of interaction is presented. The neurons are connected mutually by inhibition weights. The inhibition of the whole network can be controlled by the range of interaction of a neuron. By this local inhibition mechanism, the present network can produce patterns with a small activity from input patterns with various large activities. Moreover, it is shown in simulation that the network has attractors for input patterns. The appearance of attractors is caused by the local interaction of neurons. Thus, we expect that the network not only works as a kind of filter, but also as a memory device for storing the produced patterns. In the present paper, the fundamental features and behavior of the network are studied by using a simple network structure and a simple rule of interaction of neurons. In particular, the relation between the interaction range of a neuron and the activity of input-output patterns is shown in simulation. Furthermore, the limit of the␣transformation and the size of basin are studied numerically. Received: 5 January 1995 / Accepted in revised form: 13 November 1997     

Differential parallel processing of olfactory information in the honeybee, Apis mellifera L.

Two distinct neuronal pathways connect the first olfactory neuropil, the antennal lobe, with higher integration areas, such as the mushroom bodies, via antennal lobe projection neurons. Intracellular recordings were used to address the question whether neuroanatomical features affect odor-coding properties. We found that neurons in the median antennocerebral tract code odors by latency differences or specific inhibitory phases in combination with excitatory phases, have a more specific activity profile for different odors and convey the information with a delay. The neurons of the lateral antennocerebral tract code odors by spike rate differences, have a broader activity profile for different odors, and convey the information quickly. Thus, rather preliminary information about the olfactory stimulus first reaches the mushroom bodies and the lateral horn via neurons of the lateral antennocerebral tract and subsequently odor information becomes more specified by activities of neurons of the median antennocerebral tract. We conclude that this neuroanatomical feature is not related to the distinction between different odors, but rather reflects a dual coding of the same odor stimuli by two different neuronal strategies focusing different properties of the same stimulus.     

Feedback Inhibition and Throughput Properties of an Integrate-and-Fire-or-Burst Network Model of Retinogeniculate Transmission

Computational modeling has played an important role in the dissection of the biophysical basis of rhythmic oscillations in thalamus that are associated with sleep and certain forms of epilepsy. In contrast, the dynamic filter properties of thalamic relay nuclei during states of arousal are not well understood. Here we present a modeling and simulation study of the throughput properties of the visually driven dorsal lateral geniculate nucleus (dLGN) in the presence of feedback inhibition from the perigeniculate nucleus (PGN). We employ thalamocortical (TC) and thalamic reticular (RE) versions of a minimal integrate-and-fire-or-burst type model and a one-dimensional, two-layered network architecture. Potassium leakage conductances control the neuromodulatory state of the network and eliminate rhythmic bursting in the presence of spontaneous input (i.e., wake up the network). The aroused dLGN/PGN network model is subsequently stimulated by spatially homogeneous spontaneous retinal input or spatio-temporally patterned input consistent with the activity of X-type retinal ganglion cells during full-field or drifting grating visual stimulation. The throughput properties of this visually-driven dLGN/PGN network model are characterized and quantified as a function of stimulus parameters such as contrast, temporal frequency, and spatial frequency. During low-frequency oscillatory full-field stimulation, feedback inhibition from RE neurons often leads to TC neuron burst responses, while at high frequency tonic responses dominate. Depending on the average rate of stimulation, contrast level, and temporal frequency of modulation, the TC and RE cell bursts may or may not be phase-locked to the visual stimulus. During drifting-grating stimulation, phase-locked bursts often occur for sufficiently high contrast so long as the spatial period of the grating is not small compared to the synaptic footprint length, i.e., the spatial scale of the network connectivity.     

Hippocampal rhythm generation: Gamma-related theta-frequency resonance in CA3 interneurons

 During different behavioral states different population activities are present in the hippocampal formation. These activities are not independent: sharp waves often occur together with high-frequency ripples, and gamma-frequency activity is usually superimposed on theta oscillations. There is both experimental and theoretical evidence supporting the notion that gamma oscillation is generated intrahippocampally, but there is no generally accepted view about the origin of theta waves. Precise timing of population bursts of pyramidal cells may be due to a synchronized external drive. Membrane potential oscillations recorded in the septum are unlikely to fulfill this purpose because they are not coherent enough. We investigated the prospects of an intrahippocampal mechanism supplying pyramidal cells with theta frequency periodic inhibition, by studying a model of a network of hippocampal inhibitory interneurons. As shown previously, interneurons are capable of generating synchronized gamma-frequency action potential oscillations. Exciting the neurons by periodic current injection, the system could either be entrained in an oscillation with the frequency of the inducing current or exhibit in-phase periodic changes at the frequency of single cell (and network) activity. Simulations that used spatially inhomogeneous stimulus currents showed anti-phase frequency changes across cells, which resulted in a periodic decrease in the synchrony of the network. As this periodic change in synchrony occurred in the theta frequency range, our network should be able to exhibit the theta-frequency weakening of inhibition of pyramidal cells, thus offering a possible mechanism for intrahippocampal theta generation. Received: 23 February 2000 / Accepted in revised form: 30 June 2000     

Parallel versus sequential processing of relational stimulus structures

Organisms are often faced with sets of stimuli bearing specifiable relationships to each other. Experimental data suggest that even animals not suspected of being particularly rational can solve problems involving consistent linear relationships. We examine the information processing required to cope with these and related stimulus structures from a theoretical point of view. We show that both a parallel processing neural network model and a serially processing Turing machine model require minimal complexities to process linear hierarchical structures. When dealing with other relational stimulus structures, the models need differing, greater minimal complexities. Siemann and Delius (1994) report experimental results indicating that both pigeons and humans appear to operate according to the parallel, neural network model we propose here. Further experiments likely to be diagnostic are proposed.     

State-space analysis of time-varying higher-order spike correlation for multiple neural spike train data

Precise spike coordination between the spiking activities of multiple neurons is suggested as an indication of coordinated network activity in active cell assemblies. Spike correlation analysis aims to identify such cooperative network activity by detecting excess spike synchrony in simultaneously recorded multiple neural spike sequences. Cooperative activity is expected to organize dynamically during behavior and cognition; therefore currently available analysis techniques must be extended to enable the estimation of multiple time-varying spike interactions between neurons simultaneously. In particular, new methods must take advantage of the simultaneous observations of multiple neurons by addressing their higher-order dependencies, which cannot be revealed by pairwise analyses alone. In this paper, we develop a method for estimating time-varying spike interactions by means of a state-space analysis. Discretized parallel spike sequences are modeled as multi-variate binary processes using a log-linear model that provides a well-defined measure of higher-order spike correlation in an information geometry framework. We construct a recursive Bayesian filter/smoother for the extraction of spike interaction parameters. This method can simultaneously estimate the dynamic pairwise spike interactions of multiple single neurons, thereby extending the Ising/spin-glass model analysis of multiple neural spike train data to a nonstationary analysis. Furthermore, the method can estimate dynamic higher-order spike interactions. To validate the inclusion of the higher-order terms in the model, we construct an approximation method to assess the goodness-of-fit to spike data. In addition, we formulate a test method for the presence of higher-order spike correlation even in nonstationary spike data, e.g., data from awake behaving animals. The utility of the proposed methods is tested using simulated spike data with known underlying correlation dynamics. Finally, we apply the methods to neural spike data simultaneously recorded from the motor cortex of an awake monkey and demonstrate that the higher-order spike correlation organizes dynamically in relation to a behavioral demand.     

Information transmission in oscillatory neural activity

Periodic neural activity not locked to the stimulus or to motor responses is usually ignored. Here, we present new tools for modeling and quantifying the information transmission based on periodic neural activity that occurs with quasi-random phase relative to the stimulus. We propose a model to reproduce characteristic features of oscillatory spike trains, such as histograms of inter-spike intervals and phase locking of spikes to an oscillatory influence. The proposed model is based on an inhomogeneous Gamma process governed by a density function that is a product of the usual stimulus-dependent rate and a quasi-periodic function. Further, we present an analysis method generalizing the direct method (Rieke et al. in Spikes: exploring the neural code. MIT Press, Cambridge, 1999; Brenner et al. in Neural Comput 12(7):1531-1552, 2000) to assess the information content in such data. We demonstrate these tools on recordings from relay cells in the lateral geniculate nucleus of the cat.     

Coherency and connectivity in oscillating neural networks: linear partialization analysis

 This paper studies the relation between the functional synaptic connections between two artificial neural networks and the correlation of their spiking activities. The model neurons had realistic non-oscillatory dynamic properties and the networks showed oscillatory behavior as a result of their internal synaptic connectivity. We found that both excitation and inhibition cause phase locking of the oscillating activities. When the two networks excite each other the oscillations synchronize with zero phase lag, whereas mutual inhibition between the networks resulted in an anti-phase (half period phase difference) synchronization. Correlations between the activities of the two networks can also be caused by correlated external inputs driving the systems (common input). Our analysis shows that when the networks exhibit oscillatory behavior and the rate of the common input is smaller than a characteristic network oscillator frequency, the cross-correlation functions between the activities of two systems still carry information about the mutual synaptic connectivity. This information can be retrieved with linear partialization, removing the influence of the common input. We further explored the network responses to periodic external input. We found that when the input is of a frequency smaller than a certain threshold, the network responds with bursts at the same frequency as the input. Above the threshold, the network responds with a fraction of the input frequency. This frequency threshold, characterizing the oscillatory properties of the network, is also found to determine the limit to which linear partialization works. Received: 20 October 1995 / Accepted in revised form: 20 May 1996     

Connectivity and dynamics in the olfactory bulb

Dendrodendritic interactions between excitatory mitral cells and inhibitory granule cells in the olfactory bulb create a dense interaction network, reorganizing sensory representations of odors and, consequently, perception. Large-scale computational models are needed for revealing how the collective behavior of this network emerges from its global architecture. We propose an approach where we summarize anatomical information through dendritic geometry and density distributions which we use to calculate the connection probability between mitral and granule cells, while capturing activity patterns of each cell type in the neural dynamical systems theory of Izhikevich. In this way, we generate an efficient, anatomically and physiologically realistic large-scale model of the olfactory bulb network. Our model reproduces known connectivity between sister vs. non-sister mitral cells; measured patterns of lateral inhibition; and theta, beta, and gamma oscillations. The model in turn predicts testable relationships between network structure and several functional properties, including lateral inhibition, odor pattern decorrelation, and LFP oscillation frequency. We use the model to explore the influence of cortex on the olfactory bulb, demonstrating possible mechanisms by which cortical feedback to mitral cells or granule cells can influence bulbar activity, as well as how neurogenesis can improve bulbar decorrelation without requiring cell death. Our methodology provides a tractable tool for other researchers.     

Effects of long-term visual experience on responses of distinct classes of single units in inferior temporal cortex

As a precursor to the selection of a stimulus for gaze and attention, a midbrain network categorizes stimuli into "strongest" and "others." The categorization tracks flexibly, in real time, the absolute strength of the strongest stimulus. In this study, we take a first-principles approach to computations that are essential for such categorization. We demonstrate that classical feedforward lateral inhibition cannot produce flexible categorization. However, circuits in which the strength of lateral inhibition varies with the relative strength of competing stimuli categorize successfully. One particular implementation--reciprocal inhibition of feedforward lateral inhibition--is structurally the simplest, and it outperforms others in flexibly categorizing rapidly and reliably. Strong predictions of this anatomically supported circuit model are validated by neural responses measured in the owl midbrain. The results demonstrate the extraordinary power of a remarkably simple, neurally grounded circuit motif in producing flexible categorization, a computation fundamental to attention, perception, and decision making.     

Stepping out of the box: information processing in the neural networks of the basal ganglia

The Albin-DeLong 'box and arrow' model has long been the accepted standard model for the basal ganglia network. However, advances in physiological and anatomical research have enabled a more detailed neural network approach. Recent computational models hold that the basal ganglia use reinforcement signals and local competitive learning rules to reduce the dimensionality of sparse cortical information. These models predict a steady-state situation with diminished efficacy of lateral inhibition and low synchronization. In this framework, Parkinson's disease can be characterized as a persistent state of negative reinforcement, inefficient dimensionality reduction, and abnormally synchronized basal ganglia activity.     

A contribution of synaptic plasticity in the basal ganglia to processing of visual information

The mechanism of involvement of the basal ganglia in processing of visual information on the basis of dopamine-dependent modulation of efficacy of synaptic transmission in interconnected parallel associative and limbic loops (cortex--basal ganglia--thalamus--cortex) is proposed. Each loop consists of one of the visual or prefrontal cortical areas connected with the thalamic nucleus and corresponding loci in different nuclei of the basal ganglia. Circulation of activity in such a loop provides reentrance of information into the thalamus and neocortex. Dopamine releasing in response to a visual stimulus oppositely modulates the efficacy of "strong" and "weak" corticostriatal inputs. Subsequent reorganization of activity in the loop leads to a disinhibition (inhibition) of activity of those cortical neurons that were "strongly" ("weakly)" excited by the visual stimulus simultaneously with activation of dopaminergic cells. A selected neuronal pattern in each cortical area represents a property of the visual stimulus processed by this area. Excitation of dopaminergic cells by the visual stimulus via the superior colliculi requires parallel activation of a disinhibitory input to the superior colliculi via the thalamus and a "direct" pathway through the basal ganglia. The prefrontal cortex excited by the visual stimulus via the mediodorsal thalamic nucleus performs a top-down control over the dopaminergic cell activity, supervising simultaneous dopamine release in different striatal loci and thus promotes the interconnected selection of neuronal representations of individual properties of the visual stimulus and their binding in an integrated image.