Synchronization of neuronal assemblies in reciprocally connected cortical areas

Summary To investigate scene segmentation in the visual system we present a model of two reciprocally connected visual areas comprising spiking neurons. The peripheral area P is modeled similar to the primary visual cortex, while the central area C is modeled as an associative memory representing stimulus objects according to Hebbian learning. Without feedback from area C, spikes corresponding to stimulus representations in P are synchronized only locally (slow state). Feedback from C can induce fast oscillations and an increase of synchronization ranges (fast state). Presenting a superposition of several stimulus objects, scene segmentation happens on a time scale of hundreds of milliseconds by alternating epochs of the slow and fast state, where neurons representing the same object are simultaneously in the fast state. We relate our simulation results to various phenomena observed in neurophysiological experiments, such as stimulus-dependent synchronization of fast oscillations, synchronization on different time scales, ongoing activity, and attention-dependent neural activity.     

Scene segmentation by spike synchronization in reciprocally connected visual areas. II. Global assemblies and synchronization on larger space and time scales

 We present further simulation results of the model of two reciprocally connected visual areas proposed in the first paper [Knoblauch and Palm (2002) Biol Cybern 87:151–167]. One area corresponds to the orientation–selective subsystem of the primary visual cortex, the other is modeled as an associative memory representing stimulus objects according to Hebbian learning. We examine the scene-segmentation capability of our model on larger time and space scales, and relate it to experimental findings. Scene segmentation is achieved by attention switching on a time-scale longer than the gamma range. We find that the time-scale can vary depending on habituation parameters in the range of tens to hundreds of milliseconds. The switching process can be related to findings concerning attention and biased competition, and we reproduce experimental poststimulus time histograms (PSTHs) of single neurons under different stimulus and attentional conditions. In a larger variant the model exhibits traveling waves of activity on both slow and fast time-scales, with properties similar to those found in experiments. An apparent weakness of our standard model is the tendency to produce anti-phase correlations for fast activity from the two areas. Increasing the inter-areal delays in our model produces alternations of in-phase and anti-phase oscillations. The experimentally observed in-phase correlations can most naturally be obtained by the involvement of both fast and slow inter-areal connections; e.g., by two axon populations corresponding to fast-conducting myelinated and slow-conducting unmyelinated axons. Received: 22 August 2001 / Accepted in revised form: 8 April 2002     

Binding by temporal structure in multiple feature domains of an oscillatory neuronal network

An important step in visual processing is the segregation of objects in a visual scene from one another and from the embedding background. According to current theories of visual neuroscience, the different features of a particular object are represented by cells which are spatially distributed across multiple visual areas in the brain. The segregation of an object therefore requires the unique identification and integration of the pertaining cells which have to be “bound” into one assembly coding for the object in question. Several authors have suggested that such a binding of cells could be achieved by the selective synchronization of temporally structured responses of the neurons activated by features of the same stimulus. This concept has recently gained support by the observation of stimulus-dependent oscillatory activity in the visual system of the cat, pigeon and monkey. Furthermore, experimental evidence has been found for the formation and segregation of synchronously active cell assemblies representing different stimuli in the visual field. In this study, we investigate temporally structured activity in networks with single and multiple feature domains. As a first step, we examine the formation and segregation of cell assemblies by synchronizing and desynchronizing connections within a single feature module. We then demonstrate that distributed assemblies can be appropriately bound in a network comprising three modules selective for stimulus disparity, orientation and colour, respectively. In this context, we address the principal problem of segregating assemblies representing spatially overlapping stimuli in a distributed architecture. Using synchronizing as well as desynchronizing mechanisms, our simulations demonstrate that the binding problem can be solved by temporally correlated responses of cells which are distributed across multiple feature modules. Received: 25 March 1993/Accepted in revised form: 8 September 1993     

Binding and segmentation of multiple objects through neural oscillators inhibited by contour information

 Temporal correlation of neuronal activity has been suggested as a criterion for multiple object recognition. In this work, a two-dimensional network of simplified Wilson-Cowan oscillators is used to manage the binding and segmentation problem of a visual scene according to the connectedness Gestalt criterion. Binding is achieved via original coupling terms that link excitatory units to both excitatory and inhibitory units of adjacent neurons. These local coupling terms are time independent, i.e., they do not require Hebbian learning during the simulations. Segmentation is realized by a two-layer processing of the visual image. The first layer extracts all object contours from the image by means of “retinal cells” with an “on-center” receptive field. Information on contour is used to selectively inhibit Wilson-Cowan oscillators in the second layer, thus realizing a strong separation among neurons in different objects. Accidental synchronism between oscillations in different objects is prevented with the use of a global inhibitor, i.e., a global neuron that computes the overall activity in the Wilson-Cowan network and sends back an inhibitory signal. Simulations performed in a 50×50 neural grid with 21 different visual scenes (containing up to eight objects + background) with random initial conditions demonstrate that the network can correctly segment objects in almost 100% of cases using a single set of parameters, i.e., without the need to adjust parameters from one visual scene to the next. The network is robust with reference to dynamical noise superimposed on oscillatory neurons. Moreover, the network can segment both black objects on white background and vice versa and is able to deal with the problem of “fragmentation.” The main limitation of the network is its sensitivity to static noise superimposed on the objects. Overcoming this problem requires implementation of more robust mechanisms for contour enhancement in the first layer in agreement with mechanisms actually realized in the visual cortex. Received: 25 October 2001 / Accepted: 26 February 2003 / Published online: 20 May 2003 Correspondence to: Mauro Ursino (e-mail: mursino@deis.unibo.it, Tel.: +39-051-2093008, Fax: +39-051-2093073)     

Globally consistent depth sorting of overlapping 2D surfaces in a model using local recurrent interactions

The human visual system utilizes depth information as a major cue to group together visual items constituting an object and to segregate them from items belonging to other objects in the visual scene. Depth information can be inferred from a variety of different visual cues, such as disparity, occlusions and perspective. Many of these cues provide only local and relative information about the depth of objects. For example, at occlusions, T-junctions indicate the local relative depth precedence of surface patches. However, in order to obtain a globally consistent interpretation of the depth relations between the surfaces and objects in a visual scene, a mechanism is necessary that globally propagates such local and relative information. We present a computational framework in which depth information derived from T-junctions is propagated along surface contours using local recurrent interactions between neighboring neurons. We demonstrate that within this framework a globally consistent depth sorting of overlapping surfaces can be obtained on the basis of local interactions. Unlike previous approaches in which locally restricted cell interactions could merely distinguish between two depths (figure and ground), our model can also represent several intermediate depth positions. Our approach is an extension of a previous model of recurrent V1–V2 interaction for contour processing and illusory contour formation. Based on the contour representation created by this model, a recursive scheme of local interactions subsequently achieves a globally consistent depth sorting of several overlapping surfaces. Within this framework, the induction of illusory contours by the model of recurrent V1–V2 interaction gives rise to the figure-ground segmentation of illusory figures such as a Kanizsa square.     

Object segmentation model: analytical results and biological implications

 A simple, biologically motivated neural network for segmentation of a moving object from a visual scene is presented. The model consists of two parts: an object selection model which employs a scaling approach for receptive field sizes, and a subsequent network implementing a spotlight by means of multiplicative synapses. The network selects one object out of several, segments the rough contour of the object, and encodes the winner object's position with high accuracy. Analytical equations for the performance level of the network, e.g., for the critical distance of two objects above which they are perceived as separate, are derived. The network preferentially chooses the object with the largest angular velocity and the largest angular width. An equation for the velocity and width preferences is presented. Additionally it is shown that for certain neurons of the model, flat receptive fields are more favourable than Gaussian ones. The network exhibits performances similar to those known from amphibians. Various electrophysiological and behavioral results – e.g., the distribution of the diameters of the receptive fields of tectal neurons, of the tongue-projecting salamander Hydromantes italicus and the range of optimal prey velocities for prey catching – can be understood on the basis of the model. Received: 7 December 2000 / Accepted: 13 February 2001     

How Lateral Connections and Spiking Dynamics May Separate Multiple Objects Moving Together

Over successive stages, the ventral visual system of the primate brain develops neurons that respond selectively to particular objects or faces with translation, size and view invariance. The powerful neural representations found in Inferotemporal cortex form a remarkably rapid and robust basis for object recognition which belies the difficulties faced by the system when learning in natural visual environments. A central issue in understanding the process of biological object recognition is how these neurons learn to form separate representations of objects from complex visual scenes composed of multiple objects. We show how a one-layer competitive network comprised of ‘spiking’ neurons is able to learn separate transformation-invariant representations (exemplified by one-dimensional translations) of visual objects that are always seen together moving in lock-step, but separated in space. This is achieved by combining ‘Mexican hat’ functional lateral connectivity with cell firing-rate adaptation to temporally segment input representations of competing stimuli through anti-phase oscillations (perceptual cycles). These spiking dynamics are quickly and reliably generated, enabling selective modification of the feed-forward connections to neurons in the next layer through Spike-Time-Dependent Plasticity (STDP), resulting in separate translation-invariant representations of each stimulus. Variations in key properties of the model are investigated with respect to the network’s ability to develop appropriate input representations and subsequently output representations through STDP. Contrary to earlier rate-coded models of this learning process, this work shows how spiking neural networks may learn about more than one stimulus together without suffering from the ‘superposition catastrophe’. We take these results to suggest that spiking dynamics are key to understanding biological visual object recognition.     

Computational model for perception of objects and motions

Perception of objects and motions in the visual scene is one of the basic problems in the visual system. There exist ‘What’ and ‘Where’ pathways in the superior visual cortex, starting from the simple cells in the primary visual cortex. The former is able to perceive objects such as forms, color, and texture, and the latter perceives ‘where’, for example, velocity and direction of spatial movement of objects. This paper explores brain-like computational architectures of visual information processing. We propose a visual perceptual model and computational mechanism for training the perceptual model. The computational model is a three-layer network. The first layer is the input layer which is used to receive the stimuli from natural environments. The second layer is designed for representing the internal neural information. The connections between the first layer and the second layer, called the receptive fields of neurons, are self-adaptively learned based on principle of sparse neural representation. To this end, we introduce Kullback-Leibler divergence as the measure of independence between neural responses and derive the learning algorithm based on minimizing the cost function. The proposed algorithm is applied to train the basis functions, namely receptive fields, which are localized, oriented, and bandpassed. The resultant receptive fields of neurons in the second layer have the characteristics resembling that of simple cells in the primary visual cortex. Based on these basis functions, we further construct the third layer for perception of what and where in the superior visual cortex. The proposed model is able to perceive objects and their motions with a high accuracy and strong robustness against additive noise. Computer simulation results in the final section show the feasibility of the proposed perceptual model and high efficiency of the learning algorithm.     

Schema design and implementation of the grasp-related mirror neuron system

 Mirror neurons within a monkey's premotor area F5 fire not only when the monkey performs a certain class of actions but also when the monkey observes another monkey (or the experimenter) perform a similar action. It has thus been argued that these neurons are crucial for understanding of actions by others. We offer the hand-state hypothesis as a new explanation of the evolution of this capability: the basic functionality of the F5 mirror system is to elaborate the appropriate feedback – what we call the hand state– for opposition-space based control of manual grasping of an object. Given this functionality, the social role of the F5 mirror system in understanding the actions of others may be seen as an exaptation gained by generalizing from one's own hand to an other's hand. In other words, mirror neurons first evolved to augment the “canonical” F5 neurons (active during self-movement based on observation of an object) by providing visual feedback on “hand state,” relating the shape of the hand to the shape of the object. We then introduce the MNS1 (mirror neuron system 1) model of F5 and related brain regions. The existing Fagg–Arbib–Rizzolatti–Sakata model represents circuitry for visually guided grasping of objects, linking the anterior intraparietal area (AIP) with F5 canonical neurons. The MNS1 model extends the AIP visual pathway by also modeling pathways, directed toward F5 mirror neurons, which match arm–hand trajectories to the affordances and location of a potential target object. We present the basic schemas for the MNS1 model, then aggregate them into three “grand schemas”– visual analysis of hand state, reach and grasp, and the core mirror circuit – for each of which we present a useful implementation (a non-neural visual processing system, a multijoint 3-D kinematics simulator, and a learning neural network, respectively). With this implementation we show how the mirror system may learnto recognize actions already in the repertoire of the F5 canonical neurons. We show that the connectivity pattern of mirror neuron circuitry can be established through training, and that the resultant network can exhibit a range of novel, physiologically interesting behaviors during the process of action recognition. We train the system on the basis of final grasp but then observe the whole time course of mirror neuron activity, yielding predictions for neurophysiological experiments under conditions of spatial perturbation, altered kinematics, and ambiguous grasp execution which highlight the importance of the timingof mirror neuron activity. Received: 6 August 2001 / Accepted in revised form: 5 February 2002     

Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention

The activity of a border ownership selective (BOS) neuron indicates where a foreground object is located relative to its (classical) receptive field (RF). A population of BOS neurons thus provides an important component of perceptual grouping, the organization of the visual scene into objects. In previous theoretical work, it has been suggested that this grouping mechanism is implemented by a population of dedicated grouping (“G”) cells that integrate the activity of the distributed feature cells representing an object and, by feedback, modulate the same cells, thus making them border ownership selective. The feedback modulation by G cells is thought to also provide the mechanism for object-based attention. A recent modeling study showed that modulatory common feedback, implemented by synapses with N-methyl-D-aspartate (NMDA)-type glutamate receptors, accounts for the experimentally observed synchrony in spike trains of BOS neurons and the shape of cross-correlations between them, including its dependence on the attentional state. However, that study was limited to pairs of BOS neurons with consistent border ownership preferences, defined as two neurons tuned to respond to the same visual object, in which attention decreases synchrony. But attention has also been shown to increase synchrony in neurons with inconsistent border ownership selectivity. Here we extend the computational model from the previous study to fully understand these effects of attention. We postulate the existence of a second type of G-cell that represents spatial attention by modulating the activity of all BOS cells in a spatially defined area. Simulations of this model show that a combination of spatial and object-based mechanisms fully accounts for the observed pattern of synchrony between BOS neurons. Our results suggest that modulatory feedback from G-cells may underlie both spatial and object-based attention.     

Changes in reactivity of visual cortical neurons after local photic stimulation of their receptive fields in cats

Recovery cycles of unit responses in the primary visual cortex to local photic stimulation of their receptive fields were studied in unanesthetized, immobilized cats by the paired stimulus method. In most cases the process of recovery of neuronal reactivity did not follow a steady course. Recovery from depression evoked by the first stimulus took place more suddenly in neurons in the central part of the visual field, and initial recovery of activity was more complete than in peripheral neurons. Differences in the synchronization of inhibitory and excitatory inputs to neurons responsible for central and peripheral vision are discussed.Institute of Higher Nervous Activity and Neurophysiology, Academy of Sciences of the USSR, Moscow. Translated from Neirofiziologiya, Vol. 13, No. 3, pp. 233–240, May–June, 1981.     

Object Segmentation from Motion Discontinuities and Temporal Occlusions–A Biologically Inspired Model

Background

Optic flow is an important cue for object detection. Humans are able to perceive objects in a scene using only kinetic boundaries, and can perform the task even when other shape cues are not provided. These kinetic boundaries are characterized by the presence of motion discontinuities in a local neighbourhood. In addition, temporal occlusions appear along the boundaries as the object in front covers the background and the objects that are spatially behind it.

Methodology/Principal Findings

From a technical point of view, the detection of motion boundaries for segmentation based on optic flow is a difficult task. This is due to the problem that flow detected along such boundaries is generally not reliable. We propose a model derived from mechanisms found in visual areas V1, MT, and MSTl of human and primate cortex that achieves robust detection along motion boundaries. It includes two separate mechanisms for both the detection of motion discontinuities and of occlusion regions based on how neurons respond to spatial and temporal contrast, respectively. The mechanisms are embedded in a biologically inspired architecture that integrates information of different model components of the visual processing due to feedback connections. In particular, mutual interactions between the detection of motion discontinuities and temporal occlusions allow a considerable improvement of the kinetic boundary detection.

Conclusions/Significance

A new model is proposed that uses optic flow cues to detect motion discontinuities and object occlusion. We suggest that by combining these results for motion discontinuities and object occlusion, object segmentation within the model can be improved. This idea could also be applied in other models for object segmentation. In addition, we discuss how this model is related to neurophysiological findings. The model was successfully tested both with artificial and real sequences including self and object motion.     

The role of background movement in goldfish vision

In experiments described in the literature objects presented to restrained goldfish failed to induce eye movements like fixation and/or tracking. We show here that eye movements can be induced only if the background (visual surround) is not stationary relative to the fish but moving. We investigated the influence of background motion on eye movements in the range of angular velocities of 5–20° s⁻¹. The response to presentation of an object is a transient shift in mean horizontal eye position which lasts for some 10 s. If an object is presented in front of the fish the eyes move in a direction such that it is seen more or less symmetrically by both eyes. If it is presented at ±70° from the fish's long axis the eye on the side of the object moves in the direction that the object falls more centrally on its retina. During these object induced eye responses the typical optokinetic nystagmus of amplitude of some 5° with alternating fast and slow phases is maintained, and the eye velocity during the slow phase is not modified by presentation of the object. Presenting an object in front of stationary or moving backgrounds leads to transient suppression of respiration which shows habituation to repeated object presentations. Accepted: 14 April 2000     

Feature-based attentional modulations in the absence of direct visual stimulation

When faced with a crowded visual scene, observers must selectively attend to behaviorally relevant objects to avoid sensory overload. Often this selection process is guided by prior knowledge of a target-defining feature (e.g., the color red when looking for an apple), which enhances the firing rate of visual neurons that are selective for the attended feature. Here, we used functional magnetic resonance imaging and a pattern classification algorithm to predict the attentional state of human observers as they monitored a visual feature (one of two directions of motion). We find that feature-specific attention effects spread across the visual field-even to regions of the scene that do not contain a stimulus. This spread of feature-based attention to empty regions of space may facilitate the perception of behaviorally relevant stimuli by increasing sensitivity to attended features at all locations in the visual field.     

Proton n.m.r. of ferrodoxin from Clostridium pasteurianum

Proton magnetic resonance spectra at 500 MHz are reported for the oxidized and reduced forms of the 2[4Fe-4S]-ferredoxin from Clostridium pasteurianum. The reduced protein showed additional peaks in the 10–60 ppm region, which were previously unobserved, and there were significant differences between oxidized and reduced states in the whole region. The electron exchange rate in partially reduced ferredoxin is slow on the n.m.r. time scale when reduced with sodium dithionite, but fast when zinc reduced methyl viologen is used as reducing agent. We explain the difference between fast and slow exchange as being due to the different chemical properties of the two reducing agents.     

Perception of successive brief objects as a function of stimulus onset asynchrony: model experiments based on two-stage synchronization of neuronal oscillators

Recently we introduced a new version of the perceptual retouch model incorporating two interactive binding operations—binding features for objects and binding the bound feature-objects with a large scale oscillatory system that acts as a mediary for the perceptual information to reach consciousness-level representation. The relative level of synchronized firing of the neurons representing the features of an object obtained after the second-stage synchronizing modulation is used as the equivalent of conscious perception of the corresponding object. Here, this model is used for simulating interaction of two successive featured objects as a function of stimulus onset asynchrony (SOA). Model output reproduces typical results of mutual masking—with shortest and longest SOAs first and second object correct perception rate is comparable while with intermediate SOAs second object dominates over the first one. Additionally, with shortest SOAs misbinding of features to form illusory objects is simulated by the model.     

Sensory information in local field potentials and spikes from visual and auditory cortices: time scales and frequency bands

Studies analyzing sensory cortical processing or trying to decode brain activity often rely on a combination of different electrophysiological signals, such as local field potentials (LFPs) and spiking activity. Understanding the relation between these signals and sensory stimuli and between different components of these signals is hence of great interest. We here provide an analysis of LFPs and spiking activity recorded from visual and auditory cortex during stimulation with natural stimuli. In particular, we focus on the time scales on which different components of these signals are informative about the stimulus, and on the dependencies between different components of these signals. Addressing the first question, we find that stimulus information in low frequency bands (<12 Hz) is high, regardless of whether their energy is computed at the scale of milliseconds or seconds. Stimulus information in higher bands (>50 Hz), in contrast, is scale dependent, and is larger when the energy is averaged over several hundreds of milliseconds. Indeed, combined analysis of signal reliability and information revealed that the energy of slow LFP fluctuations is well related to the stimulus even when considering individual or few cycles, while the energy of fast LFP oscillations carries information only when averaged over many cycles. Addressing the second question, we find that stimulus information in different LFP bands, and in different LFP bands and spiking activity, is largely independent regardless of time scale or sensory system. Taken together, these findings suggest that different LFP bands represent dynamic natural stimuli on distinct time scales and together provide a potentially rich source of information for sensory processing or decoding brain activity.     

Two dynamic processes for the induction of long-term potentiation in hippocampal CA1 neurons

This work sets out to investigate fast and slow dynamic processes and how they effect the induction of long-term potentiation (LTP). Functionally, the fast process will work as a time window to take a spatial coincidence among various inputs projected to the hippocampus, and the slow process will work as a temporal integrator of a sequence of dynamic events. Firstly, the two factors were studied using a “burst” stimulus and a “long-interval patterns” stimulus. Secondly, we propose that, for the induction of LTP, there are two dynamic processes, fast and slow, which are productively activated by bursts and long-interval patterns. The model parameters, a time constant of short dynamics and one of long dynamics, were determined by fitting the values obtained from model simulation to the experimental data. A molecular factor or cellular factors with these two time constants are likely to be induced in LTP induction. Received: 3 November 1997 / Accepted in revised form: 18 August 1999