Scene segmentation by spike synchronization in reciprocally connected visual areas. I. Local effects of cortical feedback

 To investigate scene segmentation in the visual system we present a model of two reciprocally connected visual areas using spiking neurons. Area P corresponds to the orientation-selective subsystem of the primary visual cortex, while the central visual area C is modeled as associative memory representing stimulus objects according to Hebbian learning. Without feedback from area C, a single stimulus results in relatively slow and irregular activity, synchronized only for neighboring patches (slow state), while in the complete model activity is faster with an enlarged synchronization range (fast state). When presenting a superposition of several stimulus objects, scene segmentation happens on a time scale of hundreds of milliseconds by alternating epochs of the slow and fast states, where neurons representing the same object are simultaneously in the fast state. Correlation analysis reveals synchronization on different time scales as found in experiments (designated as tower, castle, and hill peaks). On the fast time scale (tower peaks, gamma frequency range), recordings from two sites coding either different or the same object lead to correlograms that are either flat or exhibit oscillatory modulations with a central peak. This is in agreement with experimental findings, whereas standard phase-coding models would predict shifted peaks in the case of different objects. Received: 22 August 2001 / Accepted in revised form: 8 April 2002     

Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales

 We present further simulation results of the model of two reciprocally connected visual areas proposed in the first paper [Knoblauch and Palm (2002) Biol Cybern 87:151–167]. One area corresponds to the orientation–selective subsystem of the primary visual cortex, the other is modeled as an associative memory representing stimulus objects according to Hebbian learning. We examine the scene-segmentation capability of our model on larger time and space scales, and relate it to experimental findings. Scene segmentation is achieved by attention switching on a time-scale longer than the gamma range. We find that the time-scale can vary depending on habituation parameters in the range of tens to hundreds of milliseconds. The switching process can be related to findings concerning attention and biased competition, and we reproduce experimental poststimulus time histograms (PSTHs) of single neurons under different stimulus and attentional conditions. In a larger variant the model exhibits traveling waves of activity on both slow and fast time-scales, with properties similar to those found in experiments. An apparent weakness of our standard model is the tendency to produce anti-phase correlations for fast activity from the two areas. Increasing the inter-areal delays in our model produces alternations of in-phase and anti-phase oscillations. The experimentally observed in-phase correlations can most naturally be obtained by the involvement of both fast and slow inter-areal connections; e.g., by two axon populations corresponding to fast-conducting myelinated and slow-conducting unmyelinated axons. Received: 22 August 2001 / Accepted in revised form: 8 April 2002     

A model for feature linking via collective oscillations in the primary visual cortex

A neural network model for explaining experimentally observed neuronal responses in cat primary visual cortex is proposed. In our model, the basic functional unit is an orientation column which is represented by a large homogeneous population of neurons modeled as integrate-and-fire type excitable elements. The orientation column exhibits spontaneous collective oscillations in activity in response to suitable visual stimuli. Such oscillations are caused by mutual synchronization among the neurons within the column. Numerical simulation for various stimulus patterns shows that as a result of activity correlations between different columns, the amplitude and the phase of the oscillation in each column depend strongly on the global feature of the stimulus pattern. These results satisfactorily account for experimental observations.     

Oscillatory network with self-organized dynamical connections for synchronization-based image segmentation

An oscillatory network of columnar architecture located in 3D spatial lattice was recently designed by the authors as oscillatory model of the brain visual cortex. Single network oscillator is a relaxational neural oscillator with internal dynamics tunable by visual image characteristics - local brightness and elementary bar orientation. It is able to demonstrate either activity state (stable undamped oscillations) or "silence" (quickly damped oscillations). Self-organized nonlocal dynamical connections of oscillators depend on oscillator activity levels and orientations of cortical receptive fields. Network performance consists in transfer into a state of clusterized synchronization. At current stage grey-level image segmentation tasks are carried out by 2D oscillatory network, obtained as a limit version of the source model. Due to supplemented network coupling strength control the 2D reduced network provides synchronization-based image segmentation. New results on segmentation of brightness and texture images presented in the paper demonstrate accurate network performance and informative visualization of segmentation results, inherent in the model.     

Binding by temporal structure in multiple feature domains of an oscillatory neuronal network

An important step in visual processing is the segregation of objects in a visual scene from one another and from the embedding background. According to current theories of visual neuroscience, the different features of a particular object are represented by cells which are spatially distributed across multiple visual areas in the brain. The segregation of an object therefore requires the unique identification and integration of the pertaining cells which have to be “bound” into one assembly coding for the object in question. Several authors have suggested that such a binding of cells could be achieved by the selective synchronization of temporally structured responses of the neurons activated by features of the same stimulus. This concept has recently gained support by the observation of stimulus-dependent oscillatory activity in the visual system of the cat, pigeon and monkey. Furthermore, experimental evidence has been found for the formation and segregation of synchronously active cell assemblies representing different stimuli in the visual field. In this study, we investigate temporally structured activity in networks with single and multiple feature domains. As a first step, we examine the formation and segregation of cell assemblies by synchronizing and desynchronizing connections within a single feature module. We then demonstrate that distributed assemblies can be appropriately bound in a network comprising three modules selective for stimulus disparity, orientation and colour, respectively. In this context, we address the principal problem of segregating assemblies representing spatially overlapping stimuli in a distributed architecture. Using synchronizing as well as desynchronizing mechanisms, our simulations demonstrate that the binding problem can be solved by temporally correlated responses of cells which are distributed across multiple feature modules. Received: 25 March 1993/Accepted in revised form: 8 September 1993     

Binding and segmentation of multiple objects through neural oscillators inhibited by contour information

 Temporal correlation of neuronal activity has been suggested as a criterion for multiple object recognition. In this work, a two-dimensional network of simplified Wilson-Cowan oscillators is used to manage the binding and segmentation problem of a visual scene according to the connectedness Gestalt criterion. Binding is achieved via original coupling terms that link excitatory units to both excitatory and inhibitory units of adjacent neurons. These local coupling terms are time independent, i.e., they do not require Hebbian learning during the simulations. Segmentation is realized by a two-layer processing of the visual image. The first layer extracts all object contours from the image by means of “retinal cells” with an “on-center” receptive field. Information on contour is used to selectively inhibit Wilson-Cowan oscillators in the second layer, thus realizing a strong separation among neurons in different objects. Accidental synchronism between oscillations in different objects is prevented with the use of a global inhibitor, i.e., a global neuron that computes the overall activity in the Wilson-Cowan network and sends back an inhibitory signal. Simulations performed in a 50×50 neural grid with 21 different visual scenes (containing up to eight objects + background) with random initial conditions demonstrate that the network can correctly segment objects in almost 100% of cases using a single set of parameters, i.e., without the need to adjust parameters from one visual scene to the next. The network is robust with reference to dynamical noise superimposed on oscillatory neurons. Moreover, the network can segment both black objects on white background and vice versa and is able to deal with the problem of “fragmentation.” The main limitation of the network is its sensitivity to static noise superimposed on the objects. Overcoming this problem requires implementation of more robust mechanisms for contour enhancement in the first layer in agreement with mechanisms actually realized in the visual cortex. Received: 25 October 2001 / Accepted: 26 February 2003 / Published online: 20 May 2003 Correspondence to: Mauro Ursino (e-mail: mursino@deis.unibo.it, Tel.: +39-051-2093008, Fax: +39-051-2093073)     

Visual gamma oscillations: waves,correlations, and other phenomena,including comparison with experimental data

Mean-field theory of brain dynamics is applied to explain the properties of gamma (?30 Hz) oscillations of cortical activity often seen during vision experiments. It is shown that mm-scale patchy connections in the primary visual cortex can support collective gamma oscillations with the correct frequencies and spatial structure, even when driven by uncorrelated inputs. This occurs via resonances associated with the the periodic modulation of the network connections, rather than being due to single-cell properties alone. Near-resonant gamma waves are shown to obey the Schrödinger equation, which enables techniques and insights from quantum theory to be used in exploring these classical oscillations. Resulting predictions for gamma responses to stimuli account in a unified way for a wide range of experimental results, including why oscillations and zero-lag synchrony are associated, and variations in correlation functions with time delay, intercellular distance, and stimulus features. They also imply that gamma oscillations may enable a form of frequency multiplexing of neural signals. Most importantly, it is shown that correlations reproduce experimental results that show maximal correlations between cells that respond to related features, but little correlation with other cells, an effect that has been argued to be associated with segmentation of a scene into separate objects. Consistency with infill of missing contours and increase in response with length of bar-shaped stimuli are discussed. Background correlations expected in the absence of stimulation are also calculated and shown to be consistent in form with experimental measurements and similar to stimulus-induced correlations in structure. Finally, possible links of gamma instabilities to certain classes of photically induced seizures and visual hallucinations are discussed.     

A model of stimulus-specific neural assemblies in the insect antennal lobe

It has been proposed that synchronized neural assemblies in the antennal lobe of insects encode the identity of olfactory stimuli. In response to an odor, some projection neurons exhibit synchronous firing, phase-locked to the oscillations of the field potential, whereas others do not. Experimental data indicate that neural synchronization and field oscillations are induced by fast GABA(A)-type inhibition, but it remains unclear how desynchronization occurs. We hypothesize that slow inhibition plays a key role in desynchronizing projection neurons. Because synaptic noise is believed to be the dominant factor that limits neuronal reliability, we consider a computational model of the antennal lobe in which a population of oscillatory neurons interact through unreliable GABA(A) and GABA(B) inhibitory synapses. From theoretical analysis and extensive computer simulations, we show that transmission failures at slow GABA(B) synapses make the neural response unpredictable. Depending on the balance between GABA(A) and GABA(B) inputs, particular neurons may either synchronize or desynchronize. These findings suggest a wiring scheme that triggers stimulus-specific synchronized assemblies. Inhibitory connections are set by Hebbian learning and selectively activated by stimulus patterns to form a spiking associative memory whose storage capacity is comparable to that of classical binary-coded models. We conclude that fast inhibition acts in concert with slow inhibition to reformat the glomerular input into odor-specific synchronized neural assemblies.     

Spatio-temporal brain mapping of motion-onset VEPs combined with fMRI and retinotopic maps

Neuroimaging studies have identified several motion-sensitive visual areas in the human brain, but the time course of their activation cannot be measured with these techniques. In the present study, we combined electrophysiological and neuroimaging methods (including retinotopic brain mapping) to determine the spatio-temporal profile of motion-onset visual evoked potentials for slow and fast motion stimuli and to localize its neural generators. We found that cortical activity initiates in the primary visual area (V1) for slow stimuli, peaking 100 ms after the onset of motion. Subsequently, activity in the mid-temporal motion-sensitive areas, MT+, peaked at 120 ms, followed by peaks in activity in the more dorsal area, V3A, at 160 ms and the lateral occipital complex at 180 ms. Approximately 250 ms after stimulus onset, activity fast motion stimuli was predominant in area V6 along the parieto-occipital sulcus. Finally, at 350 ms (100 ms after the motion offset) brain activity was visible again in area V1. For fast motion stimuli, the spatio-temporal brain pattern was similar, except that the first activity was detected at 70 ms in area MT+. Comparing functional magnetic resonance data for slow vs. fast motion, we found signs of slow-fast motion stimulus topography along the posterior brain in at least three cortical regions (MT+, V3A and LOR).     

Feature-based attentional modulations in the absence of direct visual stimulation

When faced with a crowded visual scene, observers must selectively attend to behaviorally relevant objects to avoid sensory overload. Often this selection process is guided by prior knowledge of a target-defining feature (e.g., the color red when looking for an apple), which enhances the firing rate of visual neurons that are selective for the attended feature. Here, we used functional magnetic resonance imaging and a pattern classification algorithm to predict the attentional state of human observers as they monitored a visual feature (one of two directions of motion). We find that feature-specific attention effects spread across the visual field-even to regions of the scene that do not contain a stimulus. This spread of feature-based attention to empty regions of space may facilitate the perception of behaviorally relevant stimuli by increasing sensitivity to attended features at all locations in the visual field.     

Functional implications of temporal structure in primate cortical information processing

Processing of information in the cerebral cortex of primates is characterized by distributed representations and processing in neuronal assemblies rather than by detector neurons, cardinal cells or command neurons. Responses of individual neurons in sensory cortical areas contain limited and ambiguous information on common features of the natural environment which is disambiguated by comparison with the responses of other, related neurons. Distributed representations are also capable to represent the enormous complexity and variability of the natural environment by the large number of possible combinations of neurons that can engage in the representation of a stimulus or other content. A critical problem of distributed representation and processing is the superposition of several assemblies activated at the same time since interpretation and processing of a population code requires that the responses related to a single representation can be identified and distinguished from other, related activity. A possible mechanism which tags related responses is the synchronization of neuronal responses of the same assembly with a precision in the millisecond range. This mechanism also supports the separate processing of distributed activity and dynamic assembly formation. Experimental evidence from electrophysiological investigations of non-human primates and human subjects shows that synchronous activity can be found in visual, auditory and motor areas of the cortex. Simultaneous recordings of neurons in the visual cortex indicate that individual neurons synchronize their activity with each other, if they respond to the same stimulus but not if they are part of different assemblies representing different contents. Furthermore, evidence for synchronous activity related to perception, expectation, memory, and attention has been observed.     

Oscillations and slow patterning in the antennal lobe

Odor presentation generates both fast oscillations and slow patterning in the spiking activity of the projection neurons (PNs) in the antennal lobe (AL) of locusts, moths and bees. Experimental results indicate that the oscillations are the result of the interaction between the PNs and the inhibitory local neurons (LNs) in the AL; e.g., blocking inhibition by application of GABA-receptor antagonists abolishes these oscillations. The slow patterning, on the other hand, was shown to be somewhat resistant to such blockage. In a H-H model, we reproduce both the oscillations and the slow patterning. As previously suggested, the oscillations are the result of the interaction between the PNs and LNs. We suggest that calcium and calcium-dependent potassium channels (found in PNs of bees and moths) are sufficient to account for the slow patterning resistant to the application of GABA-receptor antagonists. The intrinsic bursting property of the PNs, resulting from these additional modeled currents, give rise to another network feature that was seen experimentally in locusts: A relatively small increase in the number of additional generated PN action potentials when LN input is blocked. Consequently, the major effect of network inhibition is to redistribute the action potentials of the PNs from bursting to one action potential per cycle of the oscillations. Action Editor: Christiane Linster     

Dynamics of Sparsely Connected Networks of Excitatory and Inhibitory Spiking Neurons

The dynamics of networks of sparsely connected excitatory and inhibitory integrate-and-fire neurons are studied analytically. The analysis reveals a rich repertoire of states, including synchronous states in which neurons fire regularly; asynchronous states with stationary global activity and very irregular individual cell activity; and states in which the global activity oscillates but individual cells fire irregularly, typically at rates lower than the global oscillation frequency. The network can switch between these states, provided the external frequency, or the balance between excitation and inhibition, is varied. Two types of network oscillations are observed. In the fast oscillatory state, the network frequency is almost fully controlled by the synaptic time scale. In the slow oscillatory state, the network frequency depends mostly on the membrane time constant. Finite size effects in the asynchronous state are also discussed.     

A dual-mode dynamic model of the human accommodation system

The function of the accommodation system is to provide a clear retinal image of objects in the visual scene. The system was previously thought to be under simple continuous (i.e., single mode of operation) feedback control, but recent research has shown that it is under discontinuous (i.e., two stimulus-dependent modes of operation) feedback control by means of fast and slow processes. A model using MATLAB/SIMULINK was developed to simulate this dual-mode behavior. It consists of fast and slow components in a feedback loop. The fast component responds to step target disparity with an open-loop movement to nearly reach the desired level, and then the slow component uses closed-loop feedback to reduce the residual error to an acceptable small level. For slow ramps, the slow component provides smooth tracking of the stimulus, whereas for fast ramps, the fast component provides accurate staircase-like step responses. Simulation of this model using a variety of stimuli, including pulse, step, ramp, and sinusoid, showed good agreement with experimental results. Thus, this represents the first dynamic model of accommodation that can accurately simulate the complex dual-mode behavior seen experimentally. The biological significance of this model is that it can be used to quantitatively analyze clinical deficits such as amblyopia and accommodative insufficiency.     

Causal Inference for Spatial Constancy across Saccades

Our ability to interact with the environment hinges on creating a stable visual world despite the continuous changes in retinal input. To achieve visual stability, the brain must distinguish the retinal image shifts caused by eye movements and shifts due to movements of the visual scene. This process appears not to be flawless: during saccades, we often fail to detect whether visual objects remain stable or move, which is called saccadic suppression of displacement (SSD). How does the brain evaluate the memorized information of the presaccadic scene and the actual visual feedback of the postsaccadic visual scene in the computations for visual stability? Using a SSD task, we test how participants localize the presaccadic position of the fixation target, the saccade target or a peripheral non-foveated target that was displaced parallel or orthogonal during a horizontal saccade, and subsequently viewed for three different durations. Results showed different localization errors of the three targets, depending on the viewing time of the postsaccadic stimulus and its spatial separation from the presaccadic location. We modeled the data through a Bayesian causal inference mechanism, in which at the trial level an optimal mixing of two possible strategies, integration vs. separation of the presaccadic memory and the postsaccadic sensory signals, is applied. Fits of this model generally outperformed other plausible decision strategies for producing SSD. Our findings suggest that humans exploit a Bayesian inference process with two causal structures to mediate visual stability.     

Dynamic representation of odours by oscillating neural assemblies

Stimulus evoked oscillatory synchronization of neural assemblies has been most clearly documented in the olfactory and visual systems. Recent results with the olfactory system of locusts show that information about odour identity is contained in spatial and temporal aspects of an oscillatory population response. This suggests that brain oscillations may reflect a common reference for messages encoded in time. Although stimulus-evoked oscillatory phenomena are reliable, their roles in perception, memory and pattern recognition remain to be demonstrated. Using honey bees, we demonstrated that odour encoding involves, as in locusts, the oscillatory synchronization of assemblies of neurons, and that this synchronization is, here also, selectively abolished by the GABA receptor antagonist picrotoxin. In collaboration with Dr Brian Smith's laboratory, we showed, using a behavioural learning paradigm, that picrotoxin-induced desynchronization impairs the discrimination of molecularly similar odourants, but not that of dissimilar odours. It appears, therefore, that oscillatory synchronization of neuronal assemblies is relevant, and essential for fine odour discrimination. Finally, experiments with locust mushroom body neurons, two synapses downstream from the antennal lobe, indicate that their responses to odours become less specific when antennal lobe neurons are desynchronized by picrotoxin injection. These results suggest that oscillatory synchronization and the kind of temporal encoding it affords provide an additional dimension by which the brain can segment spatially overlapping stimulus representations.     

Adaptive surround modulation in cortical area MT

Visual motion perception relies on two opposing operations: integration and segmentation. Integration overcomes motion ambiguity in the visual image by spatial pooling of motion signals, whereas segmentation identifies differences between adjacent moving objects. For visual motion area MT, previous investigations have reported that stimuli in the receptive field surround, which do not elicit a response when presented alone, can nevertheless modulate responses to stimuli in the receptive field center. The directional tuning of this "surround modulation" has been found to be mainly antagonistic and hence consistent with segmentation. Here, we report that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Both types of modulation were delayed relative to response onset. Our results suggest that the dominance of antagonistic modulation in previous MT studies was due to stimulus choice and that segmentation and integration are achieved, in part, via adaptive surround modulation.     

Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention

The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention.     

Neural coding of 3D features of objects for hand action in the parietal cortex of the monkey.

In our previous studies of hand manipulation task-related neurons, we found many neurons of the parietal association cortex which responded to the sight of three-dimensional (3D) objects. Most of the task-related neurons in the AIP area (the lateral bank of the anterior intraparietal sulcus) were visually responsive and half of them responded to objects for manipulation. Most of these neurons were selective for the 3D features of the objects. More recently, we have found binocular visual neurons in the lateral bank of the caudal intraparietal sulcus (c-IPS area) that preferentially respond to a luminous bar or place at a particular orientation in space. We studied the responses of axis-orientation selective (AOS) neurons and surface-orientation selective (SOS) neurons in this area with stimuli presented on a 3D computer graphics display. The AOS neurons showed a stronger response to elongated stimuli and showed tuning to the orientation of the longitudinal axis. Many of them preferred a tilted stimulus in depth and appeared to be sensitive to orientation disparity and/or width disparity. The SOS neurons showed a stronger response to a flat than to an elongated stimulus and showed tuning to the 3D orientation of the surface. Their responses increased with the width or length of the stimulus. A considerable number of SOS neurons responded to a square in a random dot stereogram and were tuned to orientation in depth, suggesting their sensitivity to the gradient of disparity. We also found several SOS neurons that responded to a square with tilted or slanted contours, suggesting their sensitivity to orientation disparity and/or width disparity. Area c-IPS is likely to send visual signals of the 3D features of an object to area AIP for the visual guidance of hand actions.