Invariant visual object recognition: a model, with lighting invariance.

How are invariant representations of objects formed in the visual cortex? We describe a neurophysiological and computational approach which focusses on a feature hierarchy model in which invariant representations can be built by self-organizing learning based on the statistics of the visual input. The model can use temporal continuity in an associative synaptic learning rule with a short term memory trace, and/or it can use spatial continuity in Continuous Transformation learning. The model of visual processing in the ventral cortical stream can build representations of objects that are invariant with respect to translation, view, size, and in this paper we show also lighting. The model has been extended to provide an account of invariant representations in the dorsal visual system of the global motion produced by objects such as looming, rotation, and object-based movement. The model has been extended to incorporate top-down feedback connections to model the control of attention by biased competition in for example spatial and object search tasks. The model has also been extended to account for how the visual system can select single objects in complex visual scenes, and how multiple objects can be represented in a scene.     

How Lateral Connections and Spiking Dynamics May Separate Multiple Objects Moving Together

Over successive stages, the ventral visual system of the primate brain develops neurons that respond selectively to particular objects or faces with translation, size and view invariance. The powerful neural representations found in Inferotemporal cortex form a remarkably rapid and robust basis for object recognition which belies the difficulties faced by the system when learning in natural visual environments. A central issue in understanding the process of biological object recognition is how these neurons learn to form separate representations of objects from complex visual scenes composed of multiple objects. We show how a one-layer competitive network comprised of ‘spiking’ neurons is able to learn separate transformation-invariant representations (exemplified by one-dimensional translations) of visual objects that are always seen together moving in lock-step, but separated in space. This is achieved by combining ‘Mexican hat’ functional lateral connectivity with cell firing-rate adaptation to temporally segment input representations of competing stimuli through anti-phase oscillations (perceptual cycles). These spiking dynamics are quickly and reliably generated, enabling selective modification of the feed-forward connections to neurons in the next layer through Spike-Time-Dependent Plasticity (STDP), resulting in separate translation-invariant representations of each stimulus. Variations in key properties of the model are investigated with respect to the network’s ability to develop appropriate input representations and subsequently output representations through STDP. Contrary to earlier rate-coded models of this learning process, this work shows how spiking neural networks may learn about more than one stimulus together without suffering from the ‘superposition catastrophe’. We take these results to suggest that spiking dynamics are key to understanding biological visual object recognition.     

Circadian rhythms in Limulus photoreceptors. II. Quantum bumps

The light response of the lateral eye of the horseshoe crab, Limulus polyphemus, increases at night, while the frequency of spontaneous discrete fluctuations of its photoreceptor membrane potential (quantum bumps) decreases. These changes are controlled by a circadian clock in the brain, which transmits activity to the eye via efferent optic nerve fibers (Barlow, R. B., S. J. Bolanski, and M. L Brachman. 1977. Science. 197:86-89). Here we report the results of experiments in which we recorded from single Limulus photoreceptors in vivo for several days and studied in detail changes in their physiological and membrane properties. We found that: (a) The shape of (voltage) quantum bumps changes with the time of day. At night, spontaneous bumps and bumps evoked by dim light are prolonged. The return of the membrane potential to its resting level is delayed, but the rise time of the bump is unaffected. On average, the area under a bump is 2.4 times greater at night than during the day. (b) The rate of spontaneous bumps decreases at night by roughly a factor of 3, but their amplitude distribution remains unchanged. (c) The resting potential and resistance of the photoreceptor membrane do not change with the time of day. (d) the relationship between injected current and impulse rate of the second order neuron, the eccentric cell, also remains unchanged with the time of day. Thus the efferent input from the brain to the retina modulates some of the membrane properties of photoreceptor cells. Our findings suggest that the efferent input acts on ionic channels in the membrane to increase the sensitivity of the photoreceptor to light.     

Subcortical discrimination of unperceived objects during binocular rivalry

Rapid identification of behaviorally relevant objects is important for survival. In humans, the neural computations for visually discriminating complex objects involve inferior temporal cortex (IT). However, less detailed but faster form processing may also occur in a phylogenetically older subcortical visual system that terminates in the amygdala. We used binocular rivalry to present stimuli without conscious awareness, thereby eliminating the IT object representation and isolating subcortical visual input to the amygdala. Functional magnetic resonance imaging revealed significant brain activation in the left amygdala but not in object-selective IT in response to unperceived fearful faces compared to unperceived nonface objects. These findings indicate that, for certain behaviorally relevant stimuli, a high-level cortical representation in IT is not required for object discrimination in the amygdala.     

Charles Bonnet Syndrome: Evidence for a Generative Model in the Cortex?

Several theories propose that the cortex implements an internal model to explain, predict, and learn about sensory data, but the nature of this model is unclear. One condition that could be highly informative here is Charles Bonnet syndrome (CBS), where loss of vision leads to complex, vivid visual hallucinations of objects, people, and whole scenes. CBS could be taken as indication that there is a generative model in the brain, specifically one that can synthesise rich, consistent visual representations even in the absence of actual visual input. The processes that lead to CBS are poorly understood. Here, we argue that a model recently introduced in machine learning, the deep Boltzmann machine (DBM), could capture the relevant aspects of (hypothetical) generative processing in the cortex. The DBM carries both the semantics of a probabilistic generative model and of a neural network. The latter allows us to model a concrete neural mechanism that could underlie CBS, namely, homeostatic regulation of neuronal activity. We show that homeostatic plasticity could serve to make the learnt internal model robust against e.g. degradation of sensory input, but overcompensate in the case of CBS, leading to hallucinations. We demonstrate how a wide range of features of CBS can be explained in the model and suggest a potential role for the neuromodulator acetylcholine. This work constitutes the first concrete computational model of CBS and the first application of the DBM as a model in computational neuroscience. Our results lend further credence to the hypothesis of a generative model in the brain.     

Spatially Pooled Contrast Responses Predict Neural and Perceptual Similarity of Naturalistic Image Categories

The visual world is complex and continuously changing. Yet, our brain transforms patterns of light falling on our retina into a coherent percept within a few hundred milliseconds. Possibly, low-level neural responses already carry substantial information to facilitate rapid characterization of the visual input. Here, we computationally estimated low-level contrast responses to computer-generated naturalistic images, and tested whether spatial pooling of these responses could predict image similarity at the neural and behavioral level. Using EEG, we show that statistics derived from pooled responses explain a large amount of variance between single-image evoked potentials (ERPs) in individual subjects. Dissimilarity analysis on multi-electrode ERPs demonstrated that large differences between images in pooled response statistics are predictive of more dissimilar patterns of evoked activity, whereas images with little difference in statistics give rise to highly similar evoked activity patterns. In a separate behavioral experiment, images with large differences in statistics were judged as different categories, whereas images with little differences were confused. These findings suggest that statistics derived from low-level contrast responses can be extracted in early visual processing and can be relevant for rapid judgment of visual similarity. We compared our results with two other, well- known contrast statistics: Fourier power spectra and higher-order properties of contrast distributions (skewness and kurtosis). Interestingly, whereas these statistics allow for accurate image categorization, they do not predict ERP response patterns or behavioral categorization confusions. These converging computational, neural and behavioral results suggest that statistics of pooled contrast responses contain information that corresponds with perceived visual similarity in a rapid, low-level categorization task.     

The neurobiology and behavior of the American water shrew (Sorex palustris)

American water shrews (Sorex palustris) are aggressive predators that dive into streams and ponds to find prey at night. They do not use eyesight for capturing fish or for discriminating shapes. Instead they make use of vibrissae to detect and attack water movements generated by active prey and to detect the form of stationary prey. Tactile investigations are supplemented with underwater sniffing. This remarkable behavior consists of exhalation of air bubbles that spread onto objects and are then re-inhaled. Recordings for ultrasound both above and below water provide no evidence for echolocation or sonar, and presentation of electric fields and anatomical investigations provide no evidence for electroreception. Counts of myelinated fibers show by far the largest volume of sensory information comes from the trigeminal nerve compared to optic and cochlear nerves. This is in turn reflected in the organization of the water shrew’s neocortex, which contains two large somatosensory areas and much smaller visual and auditory areas. The shrew’s small brain with few cortical areas may allow exceptional speed in processing sensory information and producing motor output. Water shrews can accurately attack the source of a water disturbance in only 50 ms, perhaps outpacing any other mammalian predator.     

What we see is how we are: new paradigms in visual research

As most sensory modalities, the visual system needs to deal with very fast changes in the environment. Instead of processing all sensory stimuli, the brain is able to construct a perceptual experience by combining selected sensory input with an ongoing internal activity. Thus, the study of visual perception needs to be approached by examining not only the physical properties of stimuli, but also the brain's ongoing dynamical states onto which these perturbations are imposed. At least three different models account for this internal dynamics. One model is based on cardinal cells where the activity of few cells by itself constitutes the neuronal correlate of perception, while a second model is based on a population coding that states that the neuronal correlate of perception requires distributed activity throughout many areas of the brain. A third proposition, known as the temporal correlation hypothesis states that the distributed neuronal populations that correlate with perception, are also defined by synchronization of the activity on a millisecond time scale. This would serve to encode contextual information by defining relations between the features of visual objects. If temporal properties of neural activity are important to establish the neural mechanisms of perception, then the study of appropriate dynamical stimuli should be instrumental to determine how these systems operate. The use of natural stimuli and natural behaviors such as free viewing, which features fast changes of internal brain states as seen by motor markers, is proposed as a new experimental paradigm to study visual perception.     

Distance and shape: perception of the 3-dimensional world by weakly electric fish.

Weakly electric fish orient at night in complete darkness by employing their active electrolocation system. They emit short electric signals and perceive the consequences of these emissions with epidermal electroreceptors. Objects are detected by analyzing the electric images which they project onto the animal's electroreceptive skin surface. This process corresponds to similar processes during vision, where visual images are cast onto the retinas of eyes. Behavioral experiments have shown that electric fish can measure the distance of objects during active electrolocation, thus possessing three-dimensional depth perception of their surroundings. The fundamental mechanism for distance determination differs from stereopsis used during vision by two-eyed animals, but resembles some supplementary mechanisms for distance deduction in humans. Weakly electric fish can also perceive the three-dimensional shape of objects. The fish can learn to identify certain objects and discriminate them from all other objects. In addition, they spontaneously categorize objects according to their shapes and not according to object size or material properties. There is good evidence that some fundamental types of perceptional invariances during visual object recognition in humans are also found in electric fish during active electrolocation. These include size invariance (maybe including size constancy), rotational invariance, and translational invariance. The mechanisms of shape detection during electrolocation are still unknown, and their discoveries require additional experiments.     

Effects of visual experience on the representation of objects in the prefrontal cortex

The perception and recognition of objects are improved by experience. Here, we show that monkeys' ability to recognize degraded objects was improved by several days of practice with these objects. This improvement was reflected in the activity of neurons in the prefrontal (PF) cortex, a brain region critical for a wide range of visual behaviors. Familiar objects activated fewer neurons than did novel objects, but these neurons were more narrowly tuned, and the object representation was more resistant to the effects of degradation, after experience. These results demonstrate a neural correlate of visual learning in the PF cortex of adult monkeys.     

A multiscale dynamic routing circuit for forming size- and position-invariant object representations

We describe a neural model for forming size- and position-invariant representations of visual objects. The model is based on a previously proposed dynamic routing circuit that remaps selected portions of an input array into an object-centered reference frame. Here, we show how a multiscale representation may be incorporated at the input stage of the model, and we describe the control architecture and dynamics for a hierarchical, multistage routing circuit. Specific neurobiological substrates and mechanisms for the model are proposed, and a number of testable predictions are described.     

模拟昆虫视觉-行为抉择的强化学习模型

视觉信息用于行为抉择的过程是一个极其复杂的脑信息处理过程,昆虫或动物对外界环境的学习是以价值来控制的,并可影响其行为抉择,研究这一过程对揭示人类自身脑运行机制有重要意义.文章在郭爱克研究小组果蝇实验提供的生物依据基础上,提出了一种模拟果蝇视觉-行为抉择的神经网络模型.该模型引入了价值和基于价值的强化学习算法,应用于输入视觉图像的强化学习,以此建立果蝇脑内多巴胺和蘑菇体对于抉择判断的价值体系.模拟的结果表明,该模型可以模拟果蝇视觉信息的学习和行为抉择过程,其结果与生物实验相符,同时也为机器人视觉信息控制行为抉择的应用提供了基础.     

Predicting Cortical Dark/Bright Asymmetries from Natural Image Statistics and Early Visual Transforms

The nervous system has evolved in an environment with structure and predictability. One of the ubiquitous principles of sensory systems is the creation of circuits that capitalize on this predictability. Previous work has identified predictable non-uniformities in the distributions of basic visual features in natural images that are relevant to the encoding tasks of the visual system. Here, we report that the well-established statistical distributions of visual features -- such as visual contrast, spatial scale, and depth -- differ between bright and dark image components. Following this analysis, we go on to trace how these differences in natural images translate into different patterns of cortical input that arise from the separate bright (ON) and dark (OFF) pathways originating in the retina. We use models of these early visual pathways to transform natural images into statistical patterns of cortical input. The models include the receptive fields and non-linear response properties of the magnocellular (M) and parvocellular (P) pathways, with their ON and OFF pathway divisions. The results indicate that there are regularities in visual cortical input beyond those that have previously been appreciated from the direct analysis of natural images. In particular, several dark/bright asymmetries provide a potential account for recently discovered asymmetries in how the brain processes visual features, such as violations of classic energy-type models. On the basis of our analysis, we expect that the dark/bright dichotomy in natural images plays a key role in the generation of both cortical and perceptual asymmetries.     

Mere visual experience impacts preference for body shape: evidence from male competitive swimmers

Theoretical and empirical studies suggest that phenotypic averageness is a sign of an individual's high biological quality. The averageness should therefore be preferred in mates. A condition for such preference is the knowledge of average phenotype in the population. It is envisaged that an individual develops a neural template of typical phenotype on the basis of perceptual experience with images of conspecifics, and the template is then used in attractiveness assessments of potential partners. Regrettably, studies supporting this view are lacking. In the present study, adult male competitive swimmers and men who did not partake in swimming assessed the attractiveness of female silhouettes with proportions typical for swimmers or non-swimmers. Because swimmers see other swimmers relatively frequently, we hypothesize that they prefer swimmer-like female silhouettes more strongly than non-swimmers do. The analysis supports this hypothesis, suggesting that mere visual experience shapes a neural template of a silhouette, which subsequently serves as a reference for attractiveness evaluations.     

Active manual control of object views facilitates visual recognition

Active exploration of large-scale environments leads to better learning of spatial layout than does passive observation [1] [2] [3]. But active exploration might also help us to remember the appearance of individual objects in a scene. In fact, when we encounter new objects, we often manipulate them so that they can be seen from a variety of perspectives. We present here the first evidence that active control of the visual input in this way facilitates later recognition of objects. Observers who actively rotated novel, three-dimensional objects on a computer screen later showed more efficient visual recognition than observers who passively viewed the exact same sequence of images of these virtual objects. During active exploration, the observers focused mainly on the 'side' or 'front' views of the objects (see also [4] [5] [6]). The results demonstrate that how an object is represented for later recognition is influenced by whether or not one controls the presentation of visual input during learning.     

Geometric optical investigation of the underwater visual field of aerial animals

The underwater visual field distorted by refraction for aerial animals living near the water surface is investigated by means of geometric optics. The imaging of underwater objects by one and two aerial eyes is studied. The underwater binocular image field is determined for pairs of aerial eyes placed in horizontal and vertical planes. Some possible biooptical consequences of the visual detection of underwater prey and predator by aerial animals are discussed on the basis of the structure of their distorted visual field.     

Binocular rivalry requires visual attention

An interocular conflict arises when different images are presented to each eye at the same spatial location. The visual system resolves this conflict through binocular rivalry: observers consciously perceive spontaneous alternations between the two images. Visual attention is generally important for resolving competition between neural representations. However, given the seemingly spontaneous and automatic nature of binocular rivalry, the role of attention in resolving interocular competition remains unclear. Here we test whether visual attention is necessary to?produce rivalry. Using an EEG frequency-tagging method to track cortical representations of the conflicting images, we show that when attention was diverted away, rivalry stopped. The EEG data further suggested that the neural representations of the dichoptic images combined without attention. Thus, attention is necessary for dichoptic images to be engaged in sustained rivalry and may be generally required for resolving conflicting, potentially ambiguous input and giving a single interpretation access to consciousness.     

Face recognition from a single image per person using deep architecture neural networks

Implementing an accurate face recognition system requires images in different variations, and if our database is large, we suffer from problems such as storing cost and low speed in recognition algorithms. On the other hand, in some applications there is only one image available per person for training recognition model. In this article, we propose a neural network model inspired of bidirectional analysis and synthesis brain network which can learn nonlinear mapping between image space and components space. Using a deep neural network model, we have tried to separate pose components from person ones. After setting apart these components, we can use them to synthesis virtual images of test data in different pose and lighting conditions. These virtual images are used to train neural network classifier. The results showed that training neural classifier with virtual images gives better performance than training classifier with frontal view images.     

Distributed neural plasticity for shape learning in the human visual cortex

Expertise in recognizing objects in cluttered scenes is a critical skill for our interactions in complex environments and is thought to develop with learning. However, the neural implementation of object learning across stages of visual analysis in the human brain remains largely unknown. Using combined psychophysics and functional magnetic resonance imaging (fMRI), we show a link between shape-specific learning in cluttered scenes and distributed neuronal plasticity in the human visual cortex. We report stronger fMRI responses for trained than untrained shapes across early and higher visual areas when observers learned to detect low-salience shapes in noisy backgrounds. However, training with high-salience pop-out targets resulted in lower fMRI responses for trained than untrained shapes in higher occipitotemporal areas. These findings suggest that learning of camouflaged shapes is mediated by increasing neural sensitivity across visual areas to bolster target segmentation and feature integration. In contrast, learning of prominent pop-out shapes is mediated by associations at higher occipitotemporal areas that support sparser coding of the critical features for target recognition. We propose that the human brain learns novel objects in complex scenes by reorganizing shape processing across visual areas, while taking advantage of natural image correlations that determine the distinctiveness of target shapes.