Coding of Visual Object Features and Feature Conjunctions in the Human Brain

Object recognition is achieved through neural mechanisms reliant on the activity of distributed coordinated neural assemblies. In the initial steps of this process, an object''s features are thought to be coded very rapidly in distinct neural assemblies. These features play different functional roles in the recognition process - while colour facilitates recognition, additional contours and edges delay it. Here, we selectively varied the amount and role of object features in an entry-level categorization paradigm and related them to the electrical activity of the human brain. We found that early synchronizations (approx. 100 ms) increased quantitatively when more image features had to be coded, without reflecting their qualitative contribution to the recognition process. Later activity (approx. 200–400 ms) was modulated by the representational role of object features. These findings demonstrate that although early synchronizations may be sufficient for relatively crude discrimination of objects in visual scenes, they cannot support entry-level categorization. This was subserved by later processes of object model selection, which utilized the representational value of object features such as colour or edges to select the appropriate model and achieve identification.     

Evaluation of a shape-based model of human face discrimination using FMRI and behavioral techniques

Understanding the neural mechanisms underlying object recognition is one of the fundamental challenges of visual neuroscience. While neurophysiology experiments have provided evidence for a "simple-to-complex" processing model based on a hierarchy of increasingly complex image features, behavioral and fMRI studies of face processing have been interpreted as incompatible with this account. We present a neurophysiologically plausible, feature-based model that quantitatively accounts for face discrimination characteristics, including face inversion and "configural" effects. The model predicts that face discrimination is based on a sparse representation of units selective for face shapes, without the need to postulate additional, "face-specific" mechanisms. We derive and test predictions that quantitatively link model FFA face neuron tuning, neural adaptation measured in an fMRI rapid adaptation paradigm, and face discrimination performance. The experimental data are in excellent agreement with the model prediction that discrimination performance should asymptote as faces become dissimilar enough to activate different neuronal populations.     

Visual Ecology and Perception of Coloration Patterns by Domestic Chicks

This article suggests how we might understand the way potential predators see coloration patterns used in aposematism and visual mimicry. We start by briefly reviewing work on evolutionary function of eyes and neural mechanisms of vision. Often mechanisms used for achromatic vision are accurately modeled as adaptations for detection and recognition of the generality of optical stimuli, rather than specific stimuli such as biological signals. Colour vision is less well understood, but for photoreceptor spectral sensitivities of birds and hymenopterans there is no evidence for adaptations to species-specific stimuli, such as those of food or mates. Turning to experimental work, we investigate how achromatic and chromatic stimuli are used for object recognition by foraging domestic chicks (Gallus gallus). Chicks use chromatic and achromatic signals in different ways: discrimination of large targets uses (chromatic) colour differences, and chicks remember chromatic signals accurately. However, detection of small targets, and discrimination of visual textures requires achromatic contrast. The different roles of chromatic and achromatic information probably reflect their utility for object recognition in nature. Achromatic (intensity) variation exceeds chromatic variation, and hence is more informative about change in reflectance – for example, object borders, while chromatic signals yield more information about surface reflectance (object colour) under variable illumination. This revised version was published online in July 2006 with corrections to the Cover Date.     

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition performance even in brief presentations, and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations, such as the amount of noise, the number of neural recording sites, and the number of trials, and computational limitations, such as the complexity of the decoding classifier and the number of classifier training examples. In this work, we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of “kernel analysis” that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT, and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.     

Visual saliency and spike timing in the ventral visual pathway.

Visual saliency is a fundamental yet hard to define property of objects or locations in the visual world. In a context where objects and their representations compete to dominate our perception, saliency can be thought of as the "juice" that makes objects win the race. It is often assumed that saliency is extracted and represented in an explicit saliency map, which serves to determine the location of spatial attention at any given time. It is then by drawing attention to a salient object that it can be recognized or categorized. I argue against this classical view that visual "bottom-up" saliency automatically recruits the attentional system prior to object recognition. A number of visual processing tasks are clearly performed too fast for such a costly strategy to be employed. Rather, visual attention could simply act by biasing a saliency-based object recognition system. Under natural conditions of stimulation, saliency can be represented implicitly throughout the ventral visual pathway, independent of any explicit saliency map. At any given level, the most activated cells of the neural population simply represent the most salient locations. The notion of saliency itself grows increasingly complex throughout the system, mostly based on luminance contrast until information reaches visual cortex, gradually incorporating information about features such as orientation or color in primary visual cortex and early extrastriate areas, and finally the identity and behavioral relevance of objects in temporal cortex and beyond. Under these conditions the object that dominates perception, i.e. the object yielding the strongest (or the first) selective neural response, is by definition the one whose features are most "salient"--without the need for any external saliency map. In addition, I suggest that such an implicit representation of saliency can be best encoded in the relative times of the first spikes fired in a given neuronal population. In accordance with our subjective experience that saliency and attention do not modify the appearance of objects, the feed-forward propagation of this first spike wave could serve to trigger saliency-based object recognition outside the realm of awareness, while conscious perceptions could be mediated by the remaining discharges of longer neuronal spike trains.     

Size does not matter: size-invariant echo-acoustic object classification

Echolocating bats can not only extract spatial information from the auditory analysis of their ultrasonic emissions, they can also discriminate, classify and identify the three-dimensional shape of objects reflecting their emissions. Effective object recognition requires the segregation of size and shape information. Previous studies have shown that, like in visual object recognition, bats can transfer an echo-acoustic object discrimination task to objects of different size and that they spontaneously classify scaled versions of virtual echo-acoustic objects according to trained virtual-object standards. The current study aims to bridge the gap between these previous findings using a different class of real objects and a classification—instead of a discrimination paradigm. Echolocating bats (Phyllostomus discolor) were trained to classify an object as either a sphere or an hour-glass shaped object. The bats spontaneously generalised this classification to objects of the same shape. The generalisation cannot be explained based on similarities of the power spectra or temporal structures of the echo-acoustic object images and thus require dedicated neural mechanisms dealing with size-invariant echo-acoustic object analysis. Control experiments with human listeners classifying the echo-acoustic images of the objects confirm the universal validity of auditory size invariance. The current data thus corroborate and extend previous psychophysical evidence for sonar auditory-object normalisation and suggest that the underlying auditory mechanisms following the initial neural extraction of the echo-acoustic images in echolocating bats may be very similar in bats and humans.     

Activation of kinetically distinct synaptic conductances on inhibitory interneurons by electrotonically overlapping afferents

We measured brain activity during mental rotation and object recognition with objects rotated around three different axes. Activity in the superior parietal lobe (SPL) increased proportionally to viewpoint disparity during mental rotation, but not during object recognition. In contrast, the fusiform gyrus was preferentially recruited in a viewpoint-dependent manner in recognition as compared to mental rotation. In addition, independent of the effect of viewpoint, object recognition was associated with ventral areas and mental rotation with dorsal areas. These results indicate that the similar behavioral effects of viewpoint obtained in these two tasks are based on different neural substrates. Such findings call into question the hypothesis that mental rotation is used to compensate for changes in viewpoint during object recognition.     

Neural Correlates of the Perception for Novel Objects

Perception of novel objects is of enormous importance in our lives. People have to perceive or understand novel objects when seeing an original painting, admiring an unconventional construction, and using an inventive device. However, very little is known about neural mechanisms underlying the perception for novel objects. Perception of novel objects relies on the integration of unusual features of novel objects in order to identify what such objects are. In the present study, functional Magnetic Resonance Imaging (MRI) was employed to investigate neural correlates of perception of novel objects. The neuroimaging data on participants engaged in novel object viewing versus ordinary object viewing revealed that perception of novel objects involves significant activation in the left precuneus (Brodmann area 7) and the right visual cortex. The results suggest that the left precuneus is associated with the integration of unusual features of novel objects, while the right visual cortex is sensitive to the detection of such features. Our findings highlight the left precuneus as a crucial component of the neural circuitry underlying perception of novel objects.     

Neural Representation of Ambiguous Visual Objects in the Inferior Temporal Cortex

Inferior temporal (IT) cortex as the final stage of the ventral visual pathway is involved in visual object recognition. In our everyday life we need to recognize visual objects that are degraded by noise. Psychophysical studies have shown that the accuracy and speed of the object recognition decreases as the amount of visual noise increases. However, the neural representation of ambiguous visual objects and the underlying neural mechanisms of such changes in the behavior are not known. Here, by recording the neuronal spiking activity of macaque monkeys’ IT we explored the relationship between stimulus ambiguity and the IT neural activity. We found smaller amplitude, later onset, earlier offset and shorter duration of the response as visual ambiguity increased. All of these modulations were gradual and correlated with the level of stimulus ambiguity. We found that while category selectivity of IT neurons decreased with noise, it was preserved for a large extent of visual ambiguity. This noise tolerance for category selectivity in IT was lost at 60% noise level. Interestingly, while the response of the IT neurons to visual stimuli at 60% noise level was significantly larger than their baseline activity and full (100%) noise, it was not category selective anymore. The latter finding shows a neural representation that signals the presence of visual stimulus without signaling what it is. In general these findings, in the context of a drift diffusion model, explain the neural mechanisms of perceptual accuracy and speed changes in the process of recognizing ambiguous objects.     

The mechanisms of kin discrimination and the evolution of kin recognition in vertebrates: a critical re-evaluation

I re-examine the four most widely proposed mechanisms of kin discrimination among vertebrates and conclude that the current categorization of kin discrimination mechanisms has been counterproductive because it has a hindered a clear understanding of the basic mechanisms by which animals discriminate kin. I suggest that there likely is only one authentic mechanism of kin discrimination and that this mechanism is learning, particularly associative learning and habituation. Observed differences in the way animals discriminate between kin and non-kin are due only to the cues (e.g., individually-distinctive, family-distinctive, or self) that are used, and not to different mechanisms per se. I also consider whether kin discrimination is mediated by specially evolved kin recognition systems, defined as neural mechanisms that allow animals to directly classify conspecifics as either kin or non-kin. A preliminary analysis of vertebrate recognition systems suggests that specialized neural, endocrine, and developmental mechanisms specifically for recognizing kin have not evolved. Rather, kin discrimination results from an extension of other, non-specialized sensory and cognitive abilities of animals, and may be derived from other forms of social recognition, such as individual, group, or species recognition.     

MDA: a psychoactive agent with dual stimulus effects

Rats were trained to discriminate injections of either (+)-amphetamine (1.0 mg/kg) or racemic MDA (1.5 mg/kg) from saline in a two-lever drug discrimination task. After stable discrimination performances (greater than 85%) were attained in each group, stimulus generalization studies were conducted. The amphetamine-stimulus generalized to MDA, but not to the hallucinogenic agent DOM; the MDA-stimulus generalized to both amphetamine and DOM. Taken together with our previous finding that DOM-stimulus generalization occurs to MDA but not to amphetamine, the present study suggests that MDA is capable of producing dual stimulus effects in animals. In addition to these salient features, the results of this study also have an impact on stimulus specificity, and further emphasize the importance of thorough dose-response relationships as related to tests of stimulus generalization.     

Olfactory information processing in the brain: Encoding chemical and temporal features of odors

A fundamental problem in studying the neural mechanisms of odor recognition and discrimination in the olfactory system lies in determining the features or “primitives” of an odor stimulus that are analyzed by glomerular circuits at the first level of processing in the brain. Several recent studies support the idea that it is not simply the molecular features of odors that contain important information, but also the intermittent pattern of their presentation to the olfactory epithelium that helps determine the behavioral response to odor. © 1996 John Wiley & Sons, Inc.     

Early development of visual recognition

The most important ability of the human vision is object recognition, yet it is exactly the less understood aspect of the vision system. Computational models have been helpful in progressing towards an explanation of this obscure cognitive ability, and today it is possible to conceive more refined models, thanks to the new availability of neuroscientific data about the human visual cortex. This work proposes a model of the development of the object recognition capability, under a different perspective with respect to the most common approaches, with a precise theoretical epistemology. It is assumed that the main processing functions involved in recognition are not genetically determined and hardwired in the neural circuits, but are the result of interactions between epigenetic influences and the basic neural plasticity mechanisms. The model is organized in modules related with the main visual biological areas, and is implemented mainly using the LISSOM architecture, a recent self-organizing algorithm closely reflecting the essential behavior of cortical circuits.     

Benefits of hierarchical generalization and storage of the representations of "object-place" associations in the hippocampal subfields (a hypothesis)

We analyzed the results of experimental research of features of processing sensory information in the hippocampus and neocortex available in literature and results of modelling the perception of information in the neocortex. It is noted that "place" fields of neurons become wider, and overlapping of receptive fields increases during upward moving in trisynaptic hippocampal pathway. These effects specify the generalization of the information processed. The results of our analysis allow us to put forward a hypothesis that a hierarchical complication of"object - place" associations occurs during upward propagation of signals through all hippocampal subfields. Complexity of neural representations of "object - place" associations that are formed and permanently stored in the hippocampal areas increases in process of propagation of signals from the entorhinal cortex to the hierarchically higher dentate gyrus, area CA3 and area CA1. Therefore, with the aim to extract information about "object - place" associations with certain details it is necessary to access that hippocampal area in which associations were processed and stored with the required degree of elaboration. By analogy with the neocortex, it is proposed that such processing of information in the hippocampus makes it possible to avoid the combinatorial explosion and provides storing (memory) the associations accumulated during the life. The proposed mechanism can serve as an addition to the known multiple trace theory, which states that the hippocampus is an integrating part of memory trace and is always involved in recall of long-delayed episodes.     

Visual exploration and object recognition by lattice deformation

Mechanisms of explicit object recognition are often difficult to investigate and require stimuli with controlled features whose expression can be manipulated in a precise quantitative fashion. Here, we developed a novel method (called "Dots"), for generating visual stimuli, which is based on the progressive deformation of a regular lattice of dots, driven by local contour information from images of objects. By applying progressively larger deformation to the lattice, the latter conveys progressively more information about the target object. Stimuli generated with the presented method enable a precise control of object-related information content while preserving low-level image statistics, globally, and affecting them only little, locally. We show that such stimuli are useful for investigating object recognition under a naturalistic setting--free visual exploration--enabling a clear dissociation between object detection and explicit recognition. Using the introduced stimuli, we show that top-down modulation induced by previous exposure to target objects can greatly influence perceptual decisions, lowering perceptual thresholds not only for object recognition but also for object detection (visual hysteresis). Visual hysteresis is target-specific, its expression and magnitude depending on the identity of individual objects. Relying on the particular features of dot stimuli and on eye-tracking measurements, we further demonstrate that top-down processes guide visual exploration, controlling how visual information is integrated by successive fixations. Prior knowledge about objects can guide saccades/fixations to sample locations that are supposed to be highly informative, even when the actual information is missing from those locations in the stimulus. The duration of individual fixations is modulated by the novelty and difficulty of the stimulus, likely reflecting cognitive demand.     

Representation of Faces in Longtailed Macaques (Macaca fascicularis)

The experiments described in this study were intended to increase our knowledge about social cognition in primates. Longtailed macaques (Macaca fascicularis) had to discriminate facial drawings of different emotional expressions. A new experimental approach was used. During the experimental sessions social interactions within the group were permitted, but the learning behaviour of individual monkeys was analysed. The procedure consisted of a simultaneous discrimination between four visual patterns under continuous reinforcement. It has implications not only for simple tasks of stimulus discrimination but also for complex problems of internal representations and visual communication. The monkeys learned quickly to discriminate faces of different emotional expressions. This discrimination ability was completely invariant with variations of colour, brightness, size, and rotation. Rotated and inverted faces were recognized perfectly. A preference test for particular features resulted in a graded estimation of particular facial components. Most important for face recognition was the outline, followed by the eye region and the mouth. An asymmetry in recognition of the left and right halves of the face was found. Further tests involving jumbled faces indicated that not only the presence of distinct facial cues but the specific relation of facial features is essential in recognizing faces. The experiment generally confirms that causal mechanisms of social cognition in non-human primates can be studied experimentally. The behavioural results are highly consistent with findings from neurophysiology and research with human subjects.     

Generalization and the evolution of symmetry preferences

Biological displays are often symmetrical, and there is growing evidence that receivers are sensitive to these symmetries. One explanation for the evolution of such sensitivity is that symmetry reflects the quality of the signaller. An alternative is that the sensitivity may arise as a by-product of general properties of biological recognition systems. In line with the latter idea, simulations of the recognition process based on simple, artificial neural networks have suggested that generalization can give rise to preferences for particular symmetrical stimuli. However, it is not clear from these studies exactly how the preferences emerge, and to what extent the results are relevant to biological recognition systems. Here, we employ a different class of recognition models (gradient interaction models) to demonstrate more clearly how generalization can generate a preference for symmetrical variants of a display. We also point out that the predictions of the gradient interaction and network-based models regarding the effects of generalization closely match the results from empirical studies of stimulus control. Our analysis demonstrates that the effects of generalization cannot be ignored when studying the evolution of symmetry preferences and symmetric signals.     

How Can Selection of Biologically Inspired Features Improve the Performance of a Robust Object Recognition Model?

Humans can effectively and swiftly recognize objects in complex natural scenes. This outstanding ability has motivated many computational object recognition models. Most of these models try to emulate the behavior of this remarkable system. The human visual system hierarchically recognizes objects in several processing stages. Along these stages a set of features with increasing complexity is extracted by different parts of visual system. Elementary features like bars and edges are processed in earlier levels of visual pathway and as far as one goes upper in this pathway more complex features will be spotted. It is an important interrogation in the field of visual processing to see which features of an object are selected and represented by the visual cortex. To address this issue, we extended a hierarchical model, which is motivated by biology, for different object recognition tasks. In this model, a set of object parts, named patches, extracted in the intermediate stages. These object parts are used for training procedure in the model and have an important role in object recognition. These patches are selected indiscriminately from different positions of an image and this can lead to the extraction of non-discriminating patches which eventually may reduce the performance. In the proposed model we used an evolutionary algorithm approach to select a set of informative patches. Our reported results indicate that these patches are more informative than usual random patches. We demonstrate the strength of the proposed model on a range of object recognition tasks. The proposed model outperforms the original model in diverse object recognition tasks. It can be seen from the experiments that selected features are generally particular parts of target images. Our results suggest that selected features which are parts of target objects provide an efficient set for robust object recognition.