A head view-invariant representation of gaze direction in anterior superior temporal sulcus

Humans show a remarkable ability to discriminate others' gaze direction, even though a given direction can be conveyed by many physically dissimilar configurations of different eye positions and head views. For example, eye contact can be signaled by a rightward glance in a left-turned head or by direct gaze in a front-facing head. Such acute gaze discrimination implies considerable perceptual invariance. Previous human research found that superior temporal sulcus (STS) responds preferentially to gaze shifts [1], but the underlying representation that supports such general responsiveness remains poorly understood. Using multivariate pattern analysis (MVPA) of human functional magnetic resonance imaging (fMRI) data, we tested whether STS contains a higher-order, head view-invariant code for gaze direction. The results revealed a finely graded gaze direction code in right anterior STS that was invariant to head view and physical image features. Further analyses revealed similar gaze effects in left anterior STS and precuneus. Our results suggest that anterior STS codes the direction of another's attention regardless of how this information is conveyed and demonstrate how high-level face areas carry out fine-grained, perceptually relevant discrimination through invariance to other face features.     

Separate coding of different gaze directions in the superior temporal sulcus and inferior parietal lobule

Electrophysiological recording in the anterior superior temporal sulcus (STS) of monkeys has demonstrated separate cell populations responsive to direct and averted gaze. Human functional imaging has demonstrated posterior STS activation in gaze processing, particularly in coding the intentions conveyed by gaze, but to date has provided no evidence of dissociable coding of different gaze directions. Because the spatial resolution typical of group-based fMRI studies (approximately 6-10 mm) exceeds the size of cellular patches sensitive to different facial characteristics (1-4 mm in monkeys), a more sensitive technique may be required. We therefore used fMRI adaptation, which is considered to offer superior resolution, to investigate whether the human anterior STS contains representations of different gaze directions, as suggested by non-human primate research. Subjects viewed probe faces gazing left, directly ahead, or right. Adapting to leftward gaze produced a reduction in BOLD response to left relative to right (and direct) gaze probes in the anterior STS and inferior parietal cortex; rightward gaze adaptation produced a corresponding reduction to right gaze probes. Consistent with these findings, averted gaze in the adapted direction was misidentified as direct. Our study provides the first human evidence of dissociable neural systems for left and right gaze.     

fMRI of the face-processing network in the ventral temporal lobe of awake and anesthetized macaques

The primate brain features specialized areas devoted to processing of faces, which human imaging studies?localized in the superior temporal sulcus (STS) and ventral temporal cortex. Studies in macaque monkeys, in contrast, revealed face selectivity predominantly in the STS. While this discrepancy could result from a true species difference, it may simply be the consequence of technical difficulties in obtaining high-quality MR images from the ventral temporal lobe. By using an optimized fMRI protocol we here report face-selective areas in ventral TE, the parahippocampal cortex, the entorhinal cortex, and the hippocampus of awake macaques, in addition to those already known in the STS. Notably, the face-selective activation of these memory-related areas was observed although the animals were passively viewing and it was preserved even under anesthesia. These results point to similarly extensive cortical networks for face processing in humans and monkeys and highlight potential homologs of the human fusiform face area.     

Face recognition.

The study of face-selective neurons in the monkey temporal lobe, and face recognition deficits in humans after brain damage have both become very active fields of investigation. Face-selective neurons appear to be members of ensembles for coding faces rather than individual face detectors or grandmother cells. They reflect the more general role of temporal cortex in pattern recognition. In humans there are a variety of face-processing impairments that result from damage to different areas, and which reflect interference at different levels of processing of the facial image.     

A neural network reflecting decisions about human faces.

Anatomic structures have been linked to the mnemonic component of working memory, but the neural network underlying associated decision processes remains elusive. Here we present an event-related functional magnetic resonance imaging study that measured activity during the decision period of a delayed face recognition task. A double dissociation of activity between anterior cingulate cortex (ACC), and a network including left fusiform face area (FFA) and left dorsolateral prefrontal cortex (DLPFC), reflected whether a probe face matched the remembered face at the time of decision. Greater activity in the left FFA and left DLPFC correlated with probe faces that matched the remembered face; in contrast, activity in ACC was greater when the probe face did not match the remembered face. These results support a model where frontal regions act in concert with stimulus-specific temporal structures to make recognition decisions about visual stimuli.     

Facial-expression and gaze-selective responses in the monkey amygdala

The social behavior of both human and nonhuman primates relies on specializations for the recognition of individuals, their facial expressions, and their direction of gaze. A broad network of cortical and subcortical structures has been implicated in face processing, yet it is unclear whether co-occurring dimensions of face stimuli, such as expression and direction of gaze, are processed jointly or independently by anatomically and functionally segregated neural structures. Awake macaques were presented with a set of monkey faces displaying aggressive, neutral, and appeasing expressions with head and eyes either averted or directed. BOLD responses to these faces as compared to Fourier-phase-scrambled images revealed widespread activation of the superior temporal sulcus and inferotemporal cortex and included activity in the amygdala. The different dimensions of the face stimuli elicited distinct activation patterns among the amygdaloid nuclei. The basolateral amygdala, including the lateral, basal, and accessory basal nuclei, produced a stronger response for threatening than appeasing expressions. The central nucleus and bed nucleus of the stria terminalis responded more to averted than directed-gaze faces. Independent behavioral measures confirmed that faces with averted gaze were more arousing, suggesting the activity in the central nucleus may be related to attention and arousal.     

The influence of low-frequency left prefrontal repetitive transcranial magnetic stimulation on memory for words but not for faces

Brain imaging studies suggest localization of verbal working memory in the left dorsolateral prefrontal cortex (DLPFC) while face processing and memory is localized in the inferior temporal cortex and other brain areas. The goal of this study was to assess the effect of left DLPFC low-frequency repetitive transcranial magnetic stimulation (rTMS) on verbal recall and face recognition. The study revealed a significant decrease of free recall in word encoding under rTMS (110% of motor threshold, 0.9 Hz) in comparison with sham stimulation (p=0.03), while no significant difference was found with facial memory tests. Our findings support the essential role of the left DLPFC in word but not facial memory and confirm the content specific arrangement of cortical areas involved in semantic memory. As a non-invasive tool, rTMS is useful for cognitive brain mapping and the functional localization of the category specific memory system.     

Patchy organization and asymmetric distribution of the neural correlates of face processing in monkey inferotemporal cortex

BACKGROUND: It is believed that a face-specific system exists within the primate ventral visual pathway that is separate from a domain-general nonface object coding system. In addition, it is believed that hemispheric asymmetry, which was long held to be a distinct feature of the human brain, can be found in the brains of other primates as well. We show here for the first time by way of a functional imaging technique that face- and object-selective neurons form spatially distinct clusters at the cellular level in monkey inferotemporal cortex. We have used a novel functional mapping technique that simultaneously generates two separate activity profiles by exploiting the differential time course of zif268 mRNA and protein expression. RESULTS: We show that neurons activated by face stimulation can be visualized at cellular resolution and distinguished from those activated by nonface complex objects. Our dual-activity maps of face and object selectivity show that face-selective patches of various sizes (mean, 22.30 mm2; std, 32.76 mm2) exist throughout the IT cortex in the context of a large expanse of cortical territory that is responsive to visual objects. CONCLUSIONS: These results add to recent findings that face-selective patches of various sizes exist throughout area IT and provide the first direct anatomical evidence at cellular resolution for a hemispheric asymmetry in favor of the right hemisphere. Together, our results support the notion that human and monkey brains share a similarity in both anatomical organization and distribution of function with respect to high-level visual processing.     

Discriminating power of localized three-dimensional facial morphology

Many genetic syndromes involve a facial gestalt that suggests a preliminary diagnosis to an experienced clinical geneticist even before a clinical examination and genotyping are undertaken. Previously, using visualization and pattern recognition, we showed that dense surface models (DSMs) of full face shape characterize facial dysmorphology in Noonan and in 22q11 deletion syndromes. In this much larger study of 696 individuals, we extend the use of DSMs of the full face to establish accurate discrimination between controls and individuals with Williams, Smith-Magenis, 22q11 deletion, or Noonan syndromes and between individuals with different syndromes in these groups. However, the full power of the DSM approach is demonstrated by the comparable discriminating abilities of localized facial features, such as periorbital, perinasal, and perioral patches, and the correlation of DSM-based predictions and molecular findings. This study demonstrates the potential of face shape models to assist clinical training through visualization, to support clinical diagnosis of affected individuals through pattern recognition, and to enable the objective comparison of individuals sharing other phenotypic or genotypic properties.     

Emotion Separation Is Completed Early and It Depends on Visual Field Presentation

It is now apparent that the visual system reacts to stimuli very fast, with many brain areas activated within 100 ms. It is, however, unclear how much detail is extracted about stimulus properties in the early stages of visual processing. Here, using magnetoencephalography we show that the visual system separates different facial expressions of emotion well within 100 ms after image onset, and that this separation is processed differently depending on where in the visual field the stimulus is presented. Seven right-handed males participated in a face affect recognition experiment in which they viewed happy, fearful and neutral faces. Blocks of images were shown either at the center or in one of the four quadrants of the visual field. For centrally presented faces, the emotions were separated fast, first in the right superior temporal sulcus (STS; 35–48 ms), followed by the right amygdala (57–64 ms) and medial pre-frontal cortex (83–96 ms). For faces presented in the periphery, the emotions were separated first in the ipsilateral amygdala and contralateral STS. We conclude that amygdala and STS likely play a different role in early visual processing, recruiting distinct neural networks for action: the amygdala alerts sub-cortical centers for appropriate autonomic system response for fight or flight decisions, while the STS facilitates more cognitive appraisal of situations and links appropriate cortical sites together. It is then likely that different problems may arise when either network fails to initiate or function properly.     

Perceptual scales of spatial heterogeneity of periphyton for freshwater snails

The hypothesis that individuals recognize their environment as homogeneous when the scales of spatial heterogeneity of resources are smaller than a certain scale, but can distinguish them as patches when they are larger than that scale was tested using freshwater snails ( Physa acuta ) in various distributions in periphyton environments. In a pattern of periphyton distribution in which the size of algal cells was 47 mm, individuals moved significantly more slowly on algal cells than on nonalgal cells. However, in other patterns in which the sizes of algal cells were 23.5 mm and 15.7 mm, the speeds of individuals on algal and nonalgal cells were not significantly different. These results support the hypothesis that individuals use the environment homogeneously when the scales of spatial heterogeneity of resources are smaller than a certain scale, but they can distinguish between patches when the scales are larger.     

Decoding Face Information in Time,Frequency and Space from Direct Intracranial Recordings of the Human Brain

Faces are processed by a neural system with distributed anatomical components, but the roles of these components remain unclear. A dominant theory of face perception postulates independent representations of invariant aspects of faces (e.g., identity) in ventral temporal cortex including the fusiform gyrus, and changeable aspects of faces (e.g., emotion) in lateral temporal cortex including the superior temporal sulcus. Here we recorded neuronal activity directly from the cortical surface in 9 neurosurgical subjects undergoing epilepsy monitoring while they viewed static and dynamic facial expressions. Applying novel decoding analyses to the power spectrogram of electrocorticograms (ECoG) from over 100 contacts in ventral and lateral temporal cortex, we found better representation of both invariant and changeable aspects of faces in ventral than lateral temporal cortex. Critical information for discriminating faces from geometric patterns was carried by power modulations between 50 to 150 Hz. For both static and dynamic face stimuli, we obtained a higher decoding performance in ventral than lateral temporal cortex. For discriminating fearful from happy expressions, critical information was carried by power modulation between 60–150 Hz and below 30 Hz, and again better decoded in ventral than lateral temporal cortex. Task-relevant attention improved decoding accuracy more than10% across a wide frequency range in ventral but not at all in lateral temporal cortex. Spatial searchlight decoding showed that decoding performance was highest around the middle fusiform gyrus. Finally, we found that the right hemisphere, in general, showed superior decoding to the left hemisphere. Taken together, our results challenge the dominant model for independent face representation of invariant and changeable aspects: information about both face attributes was better decoded from a single region in the middle fusiform gyrus.     

Parallel visual motion processing streams for manipulable objects and human movements

We tested the hypothesis that different regions of lateral temporal cortex are specialized for processing different types of visual motion by studying the cortical responses to moving gratings and to humans and manipulable objects (tools and utensils) that were either stationary or moving with natural or artificially generated motions. Segregated responses to human and tool stimuli were observed in both ventral and lateral regions of posterior temporal cortex. Relative to ventral cortex, lateral temporal cortex showed a larger response for moving compared with static humans and tools. Superior temporal cortex preferred human motion, and middle temporal gyrus preferred tool motion. A greater response was observed in STS to articulated compared with unarticulated human motion. Specificity for different types of complex motion (in combination with visual form) may be an organizing principle in lateral temporal cortex.     

Functional Connectivity between Face-Movement and Speech-Intelligibility Areas during Auditory-Only Speech Perception

It has been proposed that internal simulation of the talking face of visually-known speakers facilitates auditory speech recognition. One prediction of this view is that brain areas involved in auditory-only speech comprehension interact with visual face-movement sensitive areas, even under auditory-only listening conditions. Here, we test this hypothesis using connectivity analyses of functional magnetic resonance imaging (fMRI) data. Participants (17 normal participants, 17 developmental prosopagnosics) first learned six speakers via brief voice-face or voice-occupation training (<2 min/speaker). This was followed by an auditory-only speech recognition task and a control task (voice recognition) involving the learned speakers’ voices in the MRI scanner. As hypothesized, we found that, during speech recognition, familiarity with the speaker’s face increased the functional connectivity between the face-movement sensitive posterior superior temporal sulcus (STS) and an anterior STS region that supports auditory speech intelligibility. There was no difference between normal participants and prosopagnosics. This was expected because previous findings have shown that both groups use the face-movement sensitive STS to optimize auditory-only speech comprehension. Overall, the present findings indicate that learned visual information is integrated into the analysis of auditory-only speech and that this integration results from the interaction of task-relevant face-movement and auditory speech-sensitive areas.     

Mechanisms of neural integration in the parietal lobe for visual attention.

The impulse discharges of neurons in the inferior parietal association cortex (area 7) were studied in the alert, behaving rhesus monkey, trained to fixate and follow visual targets. Four classes of cells related to visual or visuomotor function were found. Cells of one of these are sensitive to visual stimuli and have large, contralateral receptive fields with maximal sensitivity in the far temporal quadrants. Cells of the other three classes are related to visuomotor functions: visual fixation, tracking, and saccades. They are neither sensory nor motor in the usual sense for they are activated only by interested fixation of gaze or tracking, or before visually evoked saccadic eye movements. They are not activated during the spontaneous saccades and fixations that the monkey makes while casually exploring his environment. It is hypothesized that the light-sensitive neurons provide the visual input to the visuomotor cells that, in turn, produce a command signal for the direction of visual attention and for shifting the focus of attention from one target to another.     

Successful Decoding of Famous Faces in the Fusiform Face Area

What are the neural mechanisms of face recognition? It is believed that the network of face-selective areas, which spans the occipital, temporal, and frontal cortices, is important in face recognition. A number of previous studies indeed reported that face identity could be discriminated based on patterns of multivoxel activity in the fusiform face area and the anterior temporal lobe. However, given the difficulty in localizing the face-selective area in the anterior temporal lobe, its role in face recognition is still unknown. Furthermore, previous studies limited their analysis to occipito-temporal regions without testing identity decoding in more anterior face-selective regions, such as the amygdala and prefrontal cortex. In the current high-resolution functional Magnetic Resonance Imaging study, we systematically examined the decoding of the identity of famous faces in the temporo-frontal network of face-selective and adjacent non-face-selective regions. A special focus has been put on the face-area in the anterior temporal lobe, which was reliably localized using an optimized scanning protocol. We found that face-identity could be discriminated above chance level only in the fusiform face area. Our results corroborate the role of the fusiform face area in face recognition. Future studies are needed to further explore the role of the more recently discovered anterior face-selective areas in face recognition.     

Neural mechanism of dynamic responses of neurons in inferior temporal cortex in face perception

Understanding the neural mechanisms of object and face recognition is one of the fundamental challenges of visual neuroscience. The neurons in inferior temporal (IT) cortex have been reported to exhibit dynamic responses to face stimuli. However, little is known about how the dynamic properties of IT neurons emerge in the face information processing. To address this issue, we made a model of IT cortex, which performs face perception via an interaction between different IT networks. The model was based on the face information processed by three resolution maps in early visual areas. The network model of IT cortex consists of four kinds of networks, in which the information about a whole face is combined with the information about its face parts and their arrangements. We show here that the learning of face stimuli makes the functional connections between these IT networks, causing a high spike correlation of IT neuron pairs. A dynamic property of subthreshold membrane potential of IT neuron, produced by Hodgkin–Huxley model, enables the coordination of temporal information without changing the firing rate, providing the basis of the mechanism underlying face perception. We show also that the hierarchical processing of face information allows IT cortex to perform a “coarse-to-fine” processing of face information. The results presented here seem to be compatible with experimental data about dynamic properties of IT neurons.     

Explicating the Face Perception Network with White Matter Connectivity

A network of multiple brain regions is recruited in face perception. Our understanding of the functional properties of this network can be facilitated by explicating the structural white matter connections that exist between its functional nodes. We accomplished this using functional MRI (fMRI) in combination with fiber tractography on high angular resolution diffusion weighted imaging data. We identified the three nodes of the core face network: the “occipital face area” (OFA), the “fusiform face area” (mid-fusiform gyrus or mFus), and the superior temporal sulcus (STS). Additionally, a region of the anterior temporal lobe (aIT), implicated as being important for face perception was identified. Our data suggest that we can further divide the OFA into multiple anatomically distinct clusters – a partitioning consistent with several recent neuroimaging results. More generally, structural white matter connectivity within this network revealed: 1) Connectivity between aIT and mFus, and between aIT and occipital regions, consistent with studies implicating this posterior to anterior pathway as critical to normal face processing; 2) Strong connectivity between mFus and each of the occipital face-selective regions, suggesting that these three areas may subserve different functional roles; 3) Almost no connectivity between STS and mFus, or between STS and the other face-selective regions. Overall, our findings suggest a re-evaluation of the “core” face network with respect to what functional areas are or are not included in this network.     

The genitals and gluteal skin are represented lateral to the foot in anterior parietal somatosensory cortex of macaques

Detailed electrophysiological maps of the representations of trunk and adjacent body parts in area 3b and area 1 of somatosensory cortex were obtained in three macaque monkeys (Macaca mulatta and Macaca radiata) of either sex. A total of 211 microelectrode penetrations 250-300 microm apart resulted in 1,190 recording sites. During penetrations deep into the posterior bank of the central sulcus, recordings were made every 300 microm to depths of 6-7 mm until sites unresponsive to somatic stimuli were reached. Cortex was later cut parasagittally and sections were stained for cytochrome oxidase (CO) or Nissl substance. Contrary to expectations from earlier reports, the genitalia were represented lateral to the representations of the foot in cortex along the area 3b/1 border. The gluteal skin including the gluteal pads and the base of the tail were also represented in this section of cortex. Only a small region of cortex was devoted to the genitalia, and neurons in this cortex had receptive fields that were large and typically included skin of the inner thigh and belly. The lower, middle and upper trunk were represented more laterally, followed by the neck, upper head and arm. The receptive fields on the trunk were roughly the same size as those for the middle and lower trunk and slightly smaller on the upper trunk.