Neural mechanism of dynamic responses of neurons in inferior temporal cortex in face perception

Understanding the neural mechanisms of object and face recognition is one of the fundamental challenges of visual neuroscience. The neurons in inferior temporal (IT) cortex have been reported to exhibit dynamic responses to face stimuli. However, little is known about how the dynamic properties of IT neurons emerge in the face information processing. To address this issue, we made a model of IT cortex, which performs face perception via an interaction between different IT networks. The model was based on the face information processed by three resolution maps in early visual areas. The network model of IT cortex consists of four kinds of networks, in which the information about a whole face is combined with the information about its face parts and their arrangements. We show here that the learning of face stimuli makes the functional connections between these IT networks, causing a high spike correlation of IT neuron pairs. A dynamic property of subthreshold membrane potential of IT neuron, produced by Hodgkin–Huxley model, enables the coordination of temporal information without changing the firing rate, providing the basis of the mechanism underlying face perception. We show also that the hierarchical processing of face information allows IT cortex to perform a “coarse-to-fine” processing of face information. The results presented here seem to be compatible with experimental data about dynamic properties of IT neurons.     

Preferred spatial frequencies for human face processing are associated with optimal class discrimination in the machine

Psychophysical studies suggest that humans preferentially use a narrow band of low spatial frequencies for face recognition. Here we asked whether artificial face recognition systems have an improved recognition performance at the same spatial frequencies as humans. To this end, we estimated recognition performance over a large database of face images by computing three discriminability measures: Fisher Linear Discriminant Analysis, Non-Parametric Discriminant Analysis, and Mutual Information. In order to address frequency dependence, discriminabilities were measured as a function of (filtered) image size. All three measures revealed a maximum at the same image sizes, where the spatial frequency content corresponds to the psychophysical found frequencies. Our results therefore support the notion that the critical band of spatial frequencies for face recognition in humans and machines follows from inherent properties of face images, and that the use of these frequencies is associated with optimal face recognition performance.     

Face pictures reduce behavioural, autonomic, endocrine and neural indices of stress and fear in sheep

Faces are highly emotive stimuli and we find smiling or familiar faces both attractive and comforting, even as young babies. Do other species with sophisticated face recognition skills, such as sheep, also respond to the emotional significance of familiar faces? We report that when sheep experience social isolation, the sight of familiar sheep face pictures compared with those of goats or inverted triangles significantly reduces behavioural (activity and protest vocalizations), autonomic (heart rate) and endocrine (cortisol and adrenaline) indices of stress. They also increase mRNA expression of activity-dependent genes (c-fos and zif/268) in brain regions specialized for processing faces (temporal and medial frontal cortices and basolateral amygdala) and for emotional control (orbitofrontal and cingulate cortex), and reduce their expression in regions associated with stress responses (hypothalamic paraventricular nucleus) and fear (central and lateral amygdala). Effects on face recognition, emotional control and fear centres are restricted to the right brain hemisphere. Results provide evidence that face pictures may be useful for relieving stress caused by unavoidable social isolation in sheep, and possibly other animal species, including humans. The finding that sheep, like humans, appear to have a right brain hemisphere involvement in the control of negative emotional experiences also suggests that functional lateralization of brain emotion systems may be a general feature in mammals.     

Successful Decoding of Famous Faces in the Fusiform Face Area

What are the neural mechanisms of face recognition? It is believed that the network of face-selective areas, which spans the occipital, temporal, and frontal cortices, is important in face recognition. A number of previous studies indeed reported that face identity could be discriminated based on patterns of multivoxel activity in the fusiform face area and the anterior temporal lobe. However, given the difficulty in localizing the face-selective area in the anterior temporal lobe, its role in face recognition is still unknown. Furthermore, previous studies limited their analysis to occipito-temporal regions without testing identity decoding in more anterior face-selective regions, such as the amygdala and prefrontal cortex. In the current high-resolution functional Magnetic Resonance Imaging study, we systematically examined the decoding of the identity of famous faces in the temporo-frontal network of face-selective and adjacent non-face-selective regions. A special focus has been put on the face-area in the anterior temporal lobe, which was reliably localized using an optimized scanning protocol. We found that face-identity could be discriminated above chance level only in the fusiform face area. Our results corroborate the role of the fusiform face area in face recognition. Future studies are needed to further explore the role of the more recently discovered anterior face-selective areas in face recognition.     

Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical visual areas.

The ways in which information about faces is represented and stored in the temporal lobe visual areas of primates, as shown by recordings from single neurons in macaques, are considered. Some neurons that respond primarily to faces are found in the cortex in the anterior part of the superior temporal sulcus (in which neurons are especially likely to be tuned to facial expression and to face movement involved in gesture), and in the TE areas more ventrally forming the inferior temporal gyrus (in which neurons are more likely to have responses related to the identity of faces). Quantitative studies of the responses of the neurons that respond differently to the faces of different individuals show that information about the identity of the individual is represented by the responses of a population of neurons, that is, ensemble encoding rather than 'grandmother cell' encoding is used. It is argued that this type of tuning is a delicate compromise between very fine tuning, which has the advantage of low interference in neuronal network operations but the disadvantage of losing the useful properties (such as generalization, completion and graceful degradation) of storage in neuronal networks, and broad tuning, which has the advantage of allowing these properties of neuronal networks to be realized but the disadvantage of leading to interference between the different memories stored in an associative network. There is evidence that the responses of some of these neurons are altered by experience so that new stimuli become incorporated in the network. It is shown that the representation that is built in temporal cortical areas shows considerable invariance for size, contrast, spatial frequency and translation. Thus the representation is in a form which is particularly useful for storage and as an output from the visual system. It is also shown that one of the representations that is built is object based, which is suitable for recognition and as an input to associative memory, and that another is viewer centred, which is appropriate for conveying information about gesture. Ways are considered in which such cortical representations might be built by competitive self-organization aided by back projections in the multi-stage cortical processing hierarchy which has convergence from stage to stage.     

Organization and functions of cells responsive to faces in the temporal cortex.

Cells selectively responsive to the face have been found in several visual sub-areas of temporal cortex in the macaque brain. These include the lateral and ventral surfaces of inferior temporal cortex and the upper bank, lower bank and fundus of the superior temporal sulcus (STS). Cells in the different regions may contribute in different ways to the processing of the facial image. Within the upper bank of the STS different populations of cells are selective for different views of the face and head. These cells occur in functionally discrete patches (3-5 mm across) within the STS cortex. Studies of output connections from the STS also reveal a modular anatomical organization of repeating 3-5 mm patches connected to the parietal cortex, an area thought to be involved in spatial awareness and in the control of attention. The properties of some cells suggest a role in the discrimination of heads from other objects, and in the recognition of familiar individuals. The selectivity for view suggests that the neural operations underlying face or head recognition rely on parallel analyses of different characteristic views of the head, the outputs of these view-specific analyses being subsequently combined to support view-independent (object-centred) recognition. An alternative functional interpretation of the sensitivity to head view is that the cells enable an analysis of 'social attention', i.e. they signal where other individuals are directing their attention. A cell maximally responsive to the left profile thus provides a signal that the attention (of another individual) is directed to the observer's left. Such information is useful for analysing social interactions between other individuals.(ABSTRACT TRUNCATED AT 250 WORDS)     

Human face recognition in sheep: lack of configurational coding and right hemisphere advantage

Face recognition in sheep is qualitatively similar to that in humans in terms of its left visual field bias, and the effects of expertise and configural coding. The current study was designed to determine whether such effects are species specific by investigating the case of sheep recognising humans. It was found that the sheep could identify human faces and while they showed a small inversion-induced decline in discriminatory performance, this was significantly less than seen with sheep faces. In other aspects, there were qualitative differences with human face recognition compared with conspecific recognition. In contrast with sheep faces there was no left visual field advantage in the recognition of human faces and the internal features were not used at all as visual cues. The data suggest that these sheep, whilst being extensively exposed to interactions with humans, were unable to identify them with all the same 'expert' methods as were used to discriminate other sheep. This suggests that different neural systems may, to some extent, be used for recognition of sheep as opposed to human faces. The relative contribution to differential neural processing of the faces of the different species and the role of expertise are discussed.     

Evidence Accumulation and Choice Maintenance Are Dissociated in Human Perceptual Decision Making

Perceptual decision making in monkeys relies on decision neurons, which accumulate evidence and maintain choices until a response is given. In humans, several brain regions have been proposed to accumulate evidence, but it is unknown if these regions also maintain choices. To test if accumulator regions in humans also maintain decisions we compared delayed and self-paced responses during a face/house discrimination decision making task. Computational modeling and fMRI results revealed dissociated processes of evidence accumulation and decision maintenance, with potential accumulator activations found in the dorsomedial prefrontal cortex, right inferior frontal gyrus and bilateral insula. Potential maintenance activation spanned the frontal pole, temporal gyri, precuneus and the lateral occipital and frontal orbital cortices. Results of a quantitative reverse inference meta-analysis performed to differentiate the functions associated with the identified regions did not narrow down potential accumulation regions, but suggested that response-maintenance might rely on a verbalization of the response.     

Human hippocampal neurons predict how well word pairs will be remembered

What is the neuronal basis for whether an experience is recalled or forgotten? In contrast to recognition, recall is difficult to study in nonhuman primates and rarely is accessible at the single neuron level in humans. We recorded 128 medial temporal lobe (MTL) neurons in patients implanted with intracranial microelectrodes while they encoded and recalled word paired associates. Neurons in the amygdala, entorhinal cortex, and hippocampus showed altered activity during encoding (9%), recall (22%), and both task phases (23%). The responses of hippocampal neurons during encoding predicted whether or not subjects later remembered the pairs successfully. Entorhinal cortex neuronal activity during retrieval was correlated with recall success. These data provide support at the single neuron level for MTL contributions to encoding and retrieval, while also suggesting there may be differences in the level of contribution of MTL regions to these memory processes.     

Constructive Autoassociative Neural Network for Facial Recognition

Autoassociative artificial neural networks have been used in many different computer vision applications. However, it is difficult to define the most suitable neural network architecture because this definition is based on previous knowledge and depends on the problem domain. To address this problem, we propose a constructive autoassociative neural network called CANet (Constructive Autoassociative Neural Network). CANet integrates the concepts of receptive fields and autoassociative memory in a dynamic architecture that changes the configuration of the receptive fields by adding new neurons in the hidden layer, while a pruning algorithm removes neurons from the output layer. Neurons in the CANet output layer present lateral inhibitory connections that improve the recognition rate. Experiments in face recognition and facial expression recognition show that the CANet outperforms other methods presented in the literature.     

Music in Our Ears: The Biological Bases of Musical Timbre Perception

Timbre is the attribute of sound that allows humans and other animals to distinguish among different sound sources. Studies based on psychophysical judgments of musical timbre, ecological analyses of sound''s physical characteristics as well as machine learning approaches have all suggested that timbre is a multifaceted attribute that invokes both spectral and temporal sound features. Here, we explored the neural underpinnings of musical timbre. We used a neuro-computational framework based on spectro-temporal receptive fields, recorded from over a thousand neurons in the mammalian primary auditory cortex as well as from simulated cortical neurons, augmented with a nonlinear classifier. The model was able to perform robust instrument classification irrespective of pitch and playing style, with an accuracy of 98.7%. Using the same front end, the model was also able to reproduce perceptual distance judgments between timbres as perceived by human listeners. The study demonstrates that joint spectro-temporal features, such as those observed in the mammalian primary auditory cortex, are critical to provide the rich-enough representation necessary to account for perceptual judgments of timbre by human listeners, as well as recognition of musical instruments.     

Effects of reversible cold block of face primary somatosensory cortex on orofacial movements and related face primary motor cortex neuronal activity

Our previous studies have revealed that face primary somatosensory cortex (SI) as well as face primary motor cortex (MI) play important roles in the control of orofacial movements in awake monkeys, and that both face MI and face SI neurons may have an orofacial mechanoreceptive field and show activity related to orofacial movements. Since it is possible that the movement-related activity of face MI neurons could reflect movement-generated orofacial afferent inputs projecting to face MI via face SI, the present study used reversible cold block-induced inactivation of the monkey's face SI to determine if face MI neuronal activity related to a trained tongue-protrusion task, chewing or swallowing was dependent on the functional integrity of the ipsilateral face SI and if inactivation of face SI affects orofacial movements. The effects of face SI cold block were tested on chewing, swallowing and/or task-related activity of 73 face MI neurons. Both task and chewing and/or swallowing-related activity of most face MI neurons was independent of the functional integrity of the ipsilateral face SI since SI cold block affected the movement-related activity in approximately 25% of the neurons. Similarly, unilateral cold block of SI had very limited effects on the performance of the task and chewing, and no effect on the performance of swallowing. These findings suggest that movement-induced reafferentation via face SI may not be a significant factor in accounting for the activity of the majority of ipsilateral face MI neurons related to trained movements, chewing and swallowing.     

Spatiotopic coding and remapping in humans

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.     

Less is more: how reduced activity reflects stronger recognition

The mechanisms of recognition involve reductions of activity in the medial temporal lobe. This preview discusses recent fMRI and MEG data from Gonsalves et al. (this issue of Neuron) that provide some of the strongest evidence to date demonstrating that reduced medial temporal activity is correlated with stronger recognition of items in humans. This result provides an important test of theories of recognition memory function based on previous neuroimaging and unit recording data.     

A quantitative morphometric comparative analysis of the primate temporal lobe

Given their importance in language comprehension, the human temporal lobes and/or some of their component structures might be expected to be larger than allometric predictions for a nonhuman anthropoid brain of human size. Whole brain, T1-weighted MRI scans were collected from 44 living anthropoid primates spanning 11 species. Easyvision software (Philips Medical Systems, The Netherlands) was used to measure the volume of the entire brain, the temporal lobes, the superior temporal gyri, and the temporal lobe white matter. The surface areas of both the entire temporal lobe and the superior temporal gyrus were also measured, as was temporal cortical gyrification.Allometric regressions of temporal lobe structures on brain volume consistently showed apes and monkeys to scale along different trajectories, with the monkeys typically lying at a higher elevation than the apes. Within the temporal lobe, overall volume, surface area, and white matter volume were significantly larger in humans than predicted by the ape regression lines. The largest departure from allometry in humans was for the temporal lobe white matter volume which, in addition to being significantly larger than predicted for brain size, was also significantly larger than predicted for temporal lobe volume. Among the nonhuman primate sample, Cebus have small temporal lobes for their brain size, and Macaca and Papio have large superior temporal gyri for their brain size. The observed departures from allometry might reflect neurobiological adaptations supporting species-specific communication in both humans and old world monkeys.     

Recognizing Age-Separated Face Images: Humans and Machines

Humans utilize facial appearance, gender, expression, aging pattern, and other ancillary information to recognize individuals. It is interesting to observe how humans perceive facial age. Analyzing these properties can help in understanding the phenomenon of facial aging and incorporating the findings can help in designing effective algorithms. Such a study has two components - facial age estimation and age-separated face recognition. Age estimation involves predicting the age of an individual given his/her facial image. On the other hand, age-separated face recognition consists of recognizing an individual given his/her age-separated images. In this research, we investigate which facial cues are utilized by humans for estimating the age of people belonging to various age groups along with analyzing the effect of one''s gender, age, and ethnicity on age estimation skills. We also analyze how various facial regions such as binocular and mouth regions influence age estimation and recognition capabilities. Finally, we propose an age-invariant face recognition algorithm that incorporates the knowledge learned from these observations. Key observations of our research are: (1) the age group of newborns and toddlers is easiest to estimate, (2) gender and ethnicity do not affect the judgment of age group estimation, (3) face as a global feature, is essential to achieve good performance in age-separated face recognition, and (4) the proposed algorithm yields improved recognition performance compared to existing algorithms and also outperforms a commercial system in the young image as probe scenario.     

Face Recognition and the Social Individual

Face recognition depends upon the uniqueness of each human face. This is accomplished by the patterns formed by the unique relationship among face features. Unique face-patterns are produced by the intrusion of random factors into the process of biological growth and development. Processes are described which enable a unique face-pattern to be represented as a percept in the visual sensory system. The components of the face recognition system are analyzed as is the manner in which the precept is connected through microcircuits to a memory file so that the history of a perceiver’s encounters with a familiar face enables the perceiver to access a memory store that is a record of the outcome of past encounters with the perceived. The importance of the face recognition system in enabling humans to individuate members the social group is discussed, as well as the importance of face recognition in the development of the individual’s social identity and ability to be a collaborative member of the social groups to which it belongs. The role of prosopagnosia—the inability to recognize familiar faces—in furthering an understanding of the face recognition system is examined, as is its importance in demonstrating the crucial nature of face recognition in human social functions. It is proposed that human face recognition is not a unique phenomenon but is an elaboration of processes existing in nonhuman primates as well as in lower animals.     

Effects of reversible cold block of face primary somatosensory cortex on orofacial movements and related face primary motor cortex neuronal activity

Our previous studies have revealed that face primary somatosensory cortex (SI) as well as face primary motor cortex (MI) play important roles in the control of orofacial movements in awake monkeys, and that both face MI and face SI neurons may have an orofacial mechanoreceptive field and show activity related to orofacial movements. Since it is possible that the movement-related activity of face MI neurons could reflect movement-generated orofacial afferent inputs projecting to face MI via face SI, the present study used reversible cold block-induced inactivation of the monkey's face SI to determine if face MI neuronal activity related to a trained tongue-protrusion task, chewing or swallowing was dependent on the functional integrity of the ipsilateral face SI and if inactivation of face SI affects orofacial movements. The effects of face SI cold block were tested on chewing, swallowing and/or task-related activity of 73 face MI neurons. Both task and chewing and/or swallowing-related activity of most face MI neurons was independent of the functional integrity of the ipsilateral face SI since SI cold block affected the movement-related activity in approximately 25% of the neurons. Similarly, unilateral cold block of SI had very limited effects on the performance of the task and chewing, and no effect on the performance of swallowing. These findings suggest that movement-induced reafferentation via face SI may not be a significant factor in accounting for the activity of the majority of ipsilateral face MI neurons related to trained movements, chewing and swallowing.     

Dynamic representation of odours by oscillating neural assemblies

Stimulus evoked oscillatory synchronization of neural assemblies has been most clearly documented in the olfactory and visual systems. Recent results with the olfactory system of locusts show that information about odour identity is contained in spatial and temporal aspects of an oscillatory population response. This suggests that brain oscillations may reflect a common reference for messages encoded in time. Although stimulus-evoked oscillatory phenomena are reliable, their roles in perception, memory and pattern recognition remain to be demonstrated. Using honey bees, we demonstrated that odour encoding involves, as in locusts, the oscillatory synchronization of assemblies of neurons, and that this synchronization is, here also, selectively abolished by the GABA receptor antagonist picrotoxin. In collaboration with Dr Brian Smith's laboratory, we showed, using a behavioural learning paradigm, that picrotoxin-induced desynchronization impairs the discrimination of molecularly similar odourants, but not that of dissimilar odours. It appears, therefore, that oscillatory synchronization of neuronal assemblies is relevant, and essential for fine odour discrimination. Finally, experiments with locust mushroom body neurons, two synapses downstream from the antennal lobe, indicate that their responses to odours become less specific when antennal lobe neurons are desynchronized by picrotoxin injection. These results suggest that oscillatory synchronization and the kind of temporal encoding it affords provide an additional dimension by which the brain can segment spatially overlapping stimulus representations.     

Object recognition: holistic representations in the monkey brain

Cognitive-psychological and neuropsychological studies suggest that the human brain processes facial information in a distinct manner, relying on mechanisms that are anatomically and functionally different from those underlying the recognition of other objects. Face recognition, for instance, can be disrupted selectively as a result of localized brain damage, and relies strongly on holistic information rather than on the mere processing of local features. Similarly, in the non-human primate, distinct neocortical and limbic structures have cell populations responding specifically to face stimuli and only weakly to other visual patterns. Moreover, such cells tend to respond to the entire configuration of a face rather than to individual facial features. But are faces the only objects represented in this way? Here I present some evidence suggesting that at least one aspect of facial processing, the processing of holistic information, may be employed by the primate brain when recognizing any arbitrary homogeneous class of even artificial objects, which the monkey has to individually learn, remember, and recognize again and again from among a large number of distractors sharing a number of common features with the target. Acquiring such an expertise can induce configurational selectivity in the response of neurons in the visual system. Our findings suggests that regarding their neural encoding faces are unlikely to be 'special', but they rather are the default 'special class' of the primate visual system.