Computations in the early visual cortex.

This paper reviews some of the recent neurophysiological studies that explore the variety of visual computations in the early visual cortex in relation to geometric inference, i.e. the inference of contours, surfaces and shapes. It attempts to draw connections between ideas from computational vision and findings from awake primate electrophysiology. In the classical feed-forward, modular view of visual processing, the early visual areas (LGN, V1 and V2) are modules that serve to extract local features, while higher extrastriate areas are responsible for shape inference and invariant object recognition. However, recent findings in primate early visual systems reveal that the computations in the early visual cortex are rather complex and dynamic, as well as interactive and plastic, subject to influence from global context, higher order perceptual inference, task requirement and behavioral experience. The evidence argues that the early visual cortex does not merely participate in the first stage of visual processing, but is involved in many levels of visual computation.     

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.     

A Network Model of Motion Processing in Area MT of Primates

A simple and biologically plausible model is proposed to simulatethe visual motion processing taking place in the middle temporal (MT) areaof the visual cortex in the primate brain. The model is ahierarchical neural network composed of multiple competitive learninglayers. The input layer of the network simulates the neurons in the primaryvisual cortex (V1), which are sensitive to the orientation and motionvelocity of the visual stimuli, and the middle and output layers of thenetwork simulate the component MT and pattern MT neurons, which areselectively responsive to local and global motions, respectively. Thenetwork model was tested with various simulated motion patterns (random dotsof different direction correlations, transparent motion, grating and plaidpatterns, and so on). The response properties of the model closely resemblemany of the known features of the MT neurons found neurophysiologically.These results show that the sophisticated response behaviors of the MTneurons can emerge naturally from some very simple models, such as acompetitive learning network.     

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition performance even in brief presentations, and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations, such as the amount of noise, the number of neural recording sites, and the number of trials, and computational limitations, such as the complexity of the decoding classifier and the number of classifier training examples. In this work, we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of “kernel analysis” that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT, and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.     

Modeling the Emergence of Whisker Direction Maps in Rat Barrel Cortex

Based on measuring responses to rat whiskers as they are mechanically stimulated, one recent study suggests that barrel-related areas in layer 2/3 rat primary somatosensory cortex (S1) contain a pinwheel map of whisker motion directions. Because this map is reminiscent of topographic organization for visual direction in primary visual cortex (V1) of higher mammals, we asked whether the S1 pinwheels could be explained by an input-driven developmental process as is often suggested for V1. We developed a computational model to capture how whisker stimuli are conveyed to supragranular S1, and simulate lateral cortical interactions using an established self-organizing algorithm. Inputs to the model each represent the deflection of a subset of 25 whiskers as they are contacted by a moving stimulus object. The subset of deflected whiskers corresponds with the shape of the stimulus, and the deflection direction corresponds with the movement direction of the stimulus. If these two features of the inputs are correlated during the training of the model, a somatotopically aligned map of direction emerges for each whisker in S1. Predictions of the model that are immediately testable include (1) that somatotopic pinwheel maps of whisker direction exist in adult layer 2/3 barrel cortex for every large whisker on the rat''s face, even peripheral whiskers; and (2) in the adult, neurons with similar directional tuning are interconnected by a network of horizontal connections, spanning distances of many whisker representations. We also propose specific experiments for testing the predictions of the model by manipulating patterns of whisker inputs experienced during early development. The results suggest that similar intracortical mechanisms guide the development of primate V1 and rat S1.     

One picture is worth at least a million neurons

How many neurons participate in the representation of a single visual image? Answering this question is critical for constraining biologically inspired models of object recognition, which vary greatly in their assumptions from few "grandmother cells" to numerous neurons in widely distributed networks. Functional imaging techniques, such as fMRI, provide an opportunity to explore this issue, since they allow the simultaneous detection of the entire neuronal population responding to each stimulus. Several studies have shown that fMRI BOLD signal is approximately proportional to neuronal activity. However, since it provides an indirect measure of this activity, obtaining a realistic estimate of the number of activated neurons requires several intervening steps. Here, we used the extensive knowledge of primate V1 to yield a conservative estimate of the ratio between hemodynamic response and neuronal firing. This ratio was then used, in addition to several cautious assumptions, to assess the number of neurons responding to a single-object image in the entire visual cortex and particularly in object-related areas. Our results show that at least a million neurons in object-related cortex and about two hundred million neurons in the entire visual cortex are involved in the representation of a single-object image.     

How Lateral Connections and Spiking Dynamics May Separate Multiple Objects Moving Together

Over successive stages, the ventral visual system of the primate brain develops neurons that respond selectively to particular objects or faces with translation, size and view invariance. The powerful neural representations found in Inferotemporal cortex form a remarkably rapid and robust basis for object recognition which belies the difficulties faced by the system when learning in natural visual environments. A central issue in understanding the process of biological object recognition is how these neurons learn to form separate representations of objects from complex visual scenes composed of multiple objects. We show how a one-layer competitive network comprised of ‘spiking’ neurons is able to learn separate transformation-invariant representations (exemplified by one-dimensional translations) of visual objects that are always seen together moving in lock-step, but separated in space. This is achieved by combining ‘Mexican hat’ functional lateral connectivity with cell firing-rate adaptation to temporally segment input representations of competing stimuli through anti-phase oscillations (perceptual cycles). These spiking dynamics are quickly and reliably generated, enabling selective modification of the feed-forward connections to neurons in the next layer through Spike-Time-Dependent Plasticity (STDP), resulting in separate translation-invariant representations of each stimulus. Variations in key properties of the model are investigated with respect to the network’s ability to develop appropriate input representations and subsequently output representations through STDP. Contrary to earlier rate-coded models of this learning process, this work shows how spiking neural networks may learn about more than one stimulus together without suffering from the ‘superposition catastrophe’. We take these results to suggest that spiking dynamics are key to understanding biological visual object recognition.     

A light and electron microscopic study of GAT-1-positive cells in the cerebral cortex of man and monkey

Summary Specimens of human cerebral cortex were obtained during neurosurgical operations and studied by immunocytochemistry and electron microscopy, using antibodies to the GABA transporter GAT-1. Cortical material from macaque monkeys was prepared similarly. Large numbers of GAT-1-positive non-pyramidal neurons were observed in layers I, II, V, and VI of the cortex. Electron microscopy also showed that the GAT-1-positive axon terminals formed symmetrical and not asymmetrical synapses, suggesting that they were the terminals of non-pyramidal neurons. Processes of cells in the walls of blood vessels were also labelled. We conclude that GAT-1 is present in cell bodies and axon terminals of non-pyramidal neurons, and a population of mural cells in blood vessels, in the primate cerebral cortex.     

Multisynaptic Inputs from the Medial Temporal Lobe to V4 in Macaques

Retrograde transsynaptic transport of rabies virus was employed to undertake the top-down projections from the medial temporal lobe (MTL) to visual area V4 of the occipitotemporal visual pathway in Japanese monkeys (Macaca fuscata). On day 3 after rabies injections into V4, neuronal labeling was observed prominently in the temporal lobe areas that have direct connections with V4, including area TF of the parahippocampal cortex. Furthermore, conspicuous neuron labeling appeared disynaptically in area TH of the parahippocampal cortex, and areas 35 and 36 of the perirhinal cortex. The labeled neurons were located predominantly in deep layers. On day 4 after the rabies injections, labeled neurons were found in the hippocampal formation, along with massive labeling in the parahippocampal and perirhinal cortices. In the hippocampal formation, the densest neuron labeling was seen in layer 5 of the entorhinal cortex, and a small but certain number of neurons were labeled in other regions, such as the subicular complex and CA1 and CA3 of the hippocampus proper. The present results indicate that V4 receives major input from the hippocampus proper via the entorhinal cortex, as well as “short-cut” pathways that bypass the entorhinal cortex. These multisynaptic pathways may define an anatomical basis for hippocampal-cortical interactions involving lower visual areas. The multisynaptic input from the MTL to V4 is likely to provide mnemonic information about object recognition that is accomplished through the occipitotemporal pathway.     

An fMRI study of the selective activation of human extrastriate form vision areas by radial and concentric gratings

The ventral form vision pathway of the primate brain comprises a sequence of areas that include V1, V2, V4 and the inferior temporal cortex (IT) [1]. Although contour extraction in the V1 area and responses to complex images, such as faces, in the IT have been studied extensively, much less is known about shape extraction at intermediate cortical levels such as V4. Here, we used functional magnetic resonance imaging (fMRI) to demonstrate that the human V4 is more strongly activated by concentric and radial patterns than by conventional sinusoidal gratings. This is consistent with global pooling of local V1 orientations to extract concentric and radial shape information in V4. Furthermore, concentric patterns were found to be effective in activating the fusiform face area. These findings support recent psychophysical [2,3] and physiological [4,5] data indicating that analysis of concentric and radial structure represents an important aspect of processing at intermediate levels of form vision.     

Extrastriate feedback to primary visual cortex in primates: a quantitative analysis of connectivity.

Knowledge-based or top-down influences on primary visual cortex (area V1) are believed to originate from information conveyed by extrastriate feedback axon connections. Understanding how this information is communicated to area V1 neurons relies in part on elucidating the quantitative as well as the qualitative nature of extrastriate pathway connectivity. A quantitative analysis of the connectivity based on anatomical data regarding the feedback pathway from extrastriate area V2 to area V1 in macaque monkey suggests (i) a total of around ten million or more area V2 axons project to area V1; (ii) the mean number of synaptic inputs from area V2 per upper-layer pyramidal cell in area V1 is less than 6% of all excitatory inputs; and (iii) the mean degree of convergence of area V2 afferents may be high, perhaps more than 100 afferent axons per cell. These results are consistent with empirical observations of the density of radial myelinated axons present in the upper layers in macaque area V1 and the proportion of excitatory extrastriate feedback synaptic inputs onto upper-layer neurons in rat visual cortex. Thus, in primate area V1, extrastriate feedback synapses onto upper-layer cells may, like geniculocortical afferent synapses onto layer IVC neurons, form only a small percentage of the total excitatory synaptic input.     

Neurons in area V4 of the macaque translate attended visual features into behaviorally relevant categories

Neural processing at most stages of the primate visual system is modulated by selective attention, such that behaviorally relevant information is emphasized at the expenses of irrelevant, potentially distracting information. The form of attention best understood at the cellular level is when stimuli at a given location in the visual field must be selected (space-based attention). In contrast, fewer single-unit recording studies have so far explored the cellular mechanisms of attention operating on individual stimulus features, specifically when one feature (e.g., color) of an object must guide behavioral responses while a second feature (e.g., shape) of the same object is potentially interfering and therefore must be ignored. Here we show that activity of neurons in macaque area V4 can underlie the selection of elemental object features and their "translation" into a categorical format that can directly contribute to the control of the animal's behavior.     

How does the cerebral cortex work? Learning, attention, and grouping by the laminar circuits of visual cortex

The organization of neocortex into layers is one of its most salient anatomical features. These layers include circuits that form functional columns in cortical maps. A major unsolved problem concerns how bottom-up, top-down, and horizontal interactions are organized within cortical layers to generate adaptive behaviors. This article models how these interactions help visual cortex to realize: (i) the binding process whereby cortex groups distributed data into coherent object representations; (ii) the attentional process whereby cortex selectively processes important events; and (iii) the developmental and learning processes whereby cortex shapes its circuits to match environmental constraints. New computational ideas about feedback systems suggest how neocortex develops and learns in a stable way, and why top-down attention requires converging bottom-up inputs to fully activate cortical cells, whereas perceptual groupings do not.     

基于初级视觉皮层的同步振荡目标检测模型

提出一种基于初级视觉皮层的目标检测模型,该模型只采用方位选择性细胞和皮层内水平连接等Ｖ１基本单元,它以链码表示的目标轮廓作为知识,允许该知识以时间脉冲的形式控制Ｖ１区内神经细胞的动态活动,使与知识轮廓形状相符合的轮廓内的细胞进入同步振荡状态,实现对视野中特定目标轮廓的识别。计算机仿真结果表明,在较高级皮层的“知识”控制之下,初级视觉皮层结构上实现简单的目标检测是可行的。     

Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

Predicting functional properties of visual cortex from an evolutionary scaling law

The number of neurons in the primary visual cortex (V1) is, across primate species, related to the number of neurons in the visual thalamus (the lateral geniculate nucleus [LGN]) by a power law with an exponent of 3/2. This evolutionary scaling law is explained by a simple relation according to which the fineness of resolution in cortex is related to the number of neurons in the area of cortex used to process the information from a single point of light (the point-spread area). The same theory provides a link between two functional properties of the visual cortex, the areal cortical magnification factor (ACMF) and the receptive field (RF) area.     

Contour saliency in primary visual cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.     

Visual field maps in human cortex

Much of the visual cortex is organized into visual field maps: nearby neurons have receptive fields at nearby locations in the image. Mammalian species generally have multiple visual field maps with each species having similar, but not identical, maps. The introduction of functional magnetic resonance imaging made it possible to identify visual field maps in human cortex, including several near (1) medial occipital (V1,V2,V3), (2) lateral occipital (LO-1,LO-2, hMT+), (3) ventral occipital (hV4, VO-1, VO-2), (4) dorsal occipital (V3A, V3B), and (5) posterior parietal cortex (IPS-0 to IPS-4). Evidence is accumulating for additional maps, including some in the frontal lobe. Cortical maps are arranged into clusters in which several maps have parallel eccentricity representations, while the angular representations within a cluster alternate in visual field sign. Visual field maps have been linked to functional and perceptual properties of the visual system at various spatial scales, ranging from the level of individual maps to map clusters to dorsal-ventral streams. We survey recent measurements of human visual field maps, describe hypotheses about the function and relationships between maps, and consider methods to improve map measurements and characterize the response properties of neurons comprising these maps.     

On the role of medial geometry in human vision.

A key challenge underlying theories of vision is how the spatially restricted, retinotopically represented feature analysis can be integrated to form abstract, coordinate-free object models. A resolution likely depends on the use of intermediate-level representations which can on the one hand be populated by local features and on the other hand be used as atomic units underlying the formation of, and interaction with, object hypotheses. The precise structure of this intermediate representation derives from the varied requirements of a range of visual tasks which motivate a significant role for incorporating a geometry of visual form. The need to integrate input from features capturing surface properties such as texture, shading, motion, color, etc., as well as from features capturing surface discontinuities such as silhouettes, T-junctions, etc., implies a geometry which captures both regional and boundary aspects. Curves, as a geometric model of boundaries, have been extensively used as an intermediate representation in computational, perceptual, and physiological studies, while the use of the medial axis (MA) has been popular mainly in computer vision as a geometric region-based model of the interior of closed boundaries. We extend the traditional model of the MA to represent images, where each MA segment represents a region of the image which we call a visual fragment. We present a unified theory of perceptual grouping and object recognition where through various sequences of transformations of the MA representation, visual fragments are grouped in various configurations to form object hypotheses, and are related to stored models. The mechanisms underlying both the computation and the transformation of the MA is a lateral wave propagation model. Recent psychophysical experiments depicting contrast sensitivity map peaks at the medial axes of stimuli, and experiments on perceptual filling-in, and brightness induction and modulation, are consistent with both the use of an MA representation and a propagation-based scheme. Also, recent neurophysiological recordings in V1 correlate with the MA hypothesis and a horizontal propagation scheme. This evidence supports a geometric computational paradigm for processing sensory data where both dynamic in-plane propagation and feedforward-feedback connections play an integral role.     

Remembering visual motion: neural correlates of associative plasticity and motion recall in cortical area MT

The pictorial content of visual memories recalled by association is embodied by neuronal activity at the highest processing stages of primate visual cortex. This activity is elicited by top-down signals from the frontal lobe and recapitulates the bottom-up pattern normally obtained by the recalled stimulus. To explore the generality and mechanisms of this phenomenon, we recorded motion-sensitive neurons at an early stage of cortical processing. After monkeys learned to associate directions of motion with static shapes, these neurons exhibited unprecedented selectivity for the shapes. This emergent shape selectivity reflects activation of neurons representing the motion stimuli recalled by association, and it suggests that recall-related activity may be a general feature of neurons in visual cortex.