The dynamics of visual adaptation to faces

Several recent demonstrations using visual adaptation have revealed high-level aftereffects for complex patterns including faces. While traditional aftereffects involve perceptual distortion of simple attributes such as orientation or colour that are processed early in the visual cortical hierarchy, face adaptation affects perceived identity and expression, which are thought to be products of higher-order processing. And, unlike most simple aftereffects, those involving faces are robust to changes in scale, position and orientation between the adapting and test stimuli. These differences raise the question of how closely related face aftereffects are to traditional ones. Little is known about the build-up and decay of the face aftereffect, and the similarity of these dynamic processes to traditional aftereffects might provide insight into this relationship. We examined the effect of varying the duration of both the adapting and test stimuli on the magnitude of perceived distortions in face identity. We found that, just as with traditional aftereffects, the identity aftereffect grew logarithmically stronger as a function of adaptation time and exponentially weaker as a function of test duration. Even the subtle aspects of these dynamics, such as the power-law relationship between the adapting and test durations, closely resembled that of other aftereffects. These results were obtained with two different sets of face stimuli that differed greatly in their low-level properties. We postulate that the mechanisms governing these shared dynamics may be dissociable from the responses of feature-selective neurons in the early visual cortex.     

Stimulus timing-dependent plasticity in high-level vision

Humans are able to efficiently learn and remember complex visual patterns after only a few seconds of exposure [1]. At a cellular level, such learning is thought to involve changes in synaptic efficacy, which have been linked to the precise timing of action potentials relative to synaptic inputs [2-4]. Previous experiments have tapped into the timing of neural spiking events by using repeated asynchronous presentation of visual stimuli to induce changes in both the tuning properties of visual neurons and the perception of simple stimulus attributes [5, 6]. Here we used a similar approach to investigate potential mechanisms underlying the perceptual learning of face identity, a high-level stimulus property based on the spatial configuration of local features. Periods of stimulus pairing induced a systematic bias in face-identity perception in a manner consistent with the predictions of spike timing-dependent plasticity. The perceptual shifts induced for face identity were tolerant to a 2-fold change in stimulus size, suggesting that they reflected neuronal changes in nonretinotopic areas, and were more than twice as strong as the perceptual shifts induced for low-level visual features. These results support the idea that spike timing-dependent plasticity can rapidly adjust the neural encoding of high-level stimulus attributes [7-11].     

Auditory adaptation in voice perception

Perceptual aftereffects following adaptation to simple stimulus attributes (e.g., motion, color) have been studied for hundreds of years. A striking recent discovery was that adaptation also elicits contrastive aftereffects in visual perception of complex stimuli and faces [1-6]. Here, we show for the first time that adaptation to nonlinguistic information in voices elicits systematic auditory aftereffects. Prior adaptation to male voices causes a voice to be perceived as more female (and vice versa), and these auditory aftereffects were measurable even minutes after adaptation. By contrast, crossmodal adaptation effects were absent, both when male or female first names and when silently articulating male or female faces were used as adaptors. When sinusoidal tones (with frequencies matched to male and female voice fundamental frequencies) were used as adaptors, no aftereffects on voice perception were observed. This excludes explanations for the voice aftereffect in terms of both pitch adaptation and postperceptual adaptation to gender concepts and suggests that contrastive voice-coding mechanisms may routinely influence voice perception. The role of adaptation in calibrating properties of high-level voice representations indicates that adaptation is not confined to vision but is a ubiquitous mechanism in the perception of nonlinguistic social information from both faces and voices.     

The functional impact of mental imagery on conscious perception

Mental imagery has been proposed to contribute to a variety of high-level cognitive functions, including memory encoding and retrieval, navigation, spatial planning, and even social communication and language comprehension. However, it is debated whether mental imagery relies on the same sensory representations as perception, and if so, what functional consequences such an overlap might have on perception itself. We report novel evidence that single instances of imagery can have a pronounced facilitatory influence on subsequent conscious perception. Either seeing or imagining a specific pattern could strongly bias which of two competing stimuli reach awareness during binocular rivalry. Effects of imagery and perception were location and orientation specific, accumulated in strength over time, and survived an intervening visual task lasting several seconds prior to presentation of the rivalry display. Interestingly, effects of imagery differed from those of feature-based attention. The results demonstrate that imagery, in the absence of any incoming visual signals, leads to the formation of a short-term sensory trace that can bias future perception, suggesting a means by which high-level processes that support imagination and memory retrieval may shape low-level sensory representations.     

Cross-Category Adaptation: Objects Produce Gender Adaptation in the Perception of Faces

Adaptation aftereffects have been found for low-level visual features such as colour, motion and shape perception, as well as higher-level features such as gender, race and identity in domains such as faces and biological motion. It is not yet clear if adaptation effects in humans extend beyond this set of higher order features. The aim of this study was to investigate whether objects highly associated with one gender, e.g. high heels for females or electric shavers for males can modulate gender perception of a face. In two separate experiments, we adapted subjects to a series of objects highly associated with one gender and subsequently asked participants to judge the gender of an ambiguous face. Results showed that participants are more likely to perceive an ambiguous face as male after being exposed to objects highly associated to females and vice versa. A gender adaptation aftereffect was obtained despite the adaptor and test stimuli being from different global categories (objects and faces respectively). These findings show that our perception of gender from faces is highly affected by our environment and recent experience. This suggests two possible mechanisms: (a) that perception of the gender associated with an object shares at least some brain areas with those responsible for gender perception of faces and (b) adaptation to gender, which is a high-level concept, can modulate brain areas that are involved in facial gender perception through top-down processes.     

Behavioural and neurophysiological evidence for face identity and face emotion processing in animals

Visual cues from faces provide important social information relating to individual identity, sexual attraction and emotional state. Behavioural and neurophysiological studies on both monkeys and sheep have shown that specialized skills and neural systems for processing these complex cues to guide behaviour have evolved in a number of mammals and are not present exclusively in humans. Indeed, there are remarkable similarities in the ways that faces are processed by the brain in humans and other mammalian species. While human studies with brain imaging and gross neurophysiological recording approaches have revealed global aspects of the face-processing network, they cannot investigate how information is encoded by specific neural networks. Single neuron electrophysiological recording approaches in both monkeys and sheep have, however, provided some insights into the neural encoding principles involved and, particularly, the presence of a remarkable degree of high-level encoding even at the level of a specific face. Recent developments that allow simultaneous recordings to be made from many hundreds of individual neurons are also beginning to reveal evidence for global aspects of a population-based code. This review will summarize what we have learned so far from these animal-based studies about the way the mammalian brain processes the faces and the emotions they can communicate, as well as associated capacities such as how identity and emotion cues are dissociated and how face imagery might be generated. It will also try to highlight what questions and advances in knowledge still challenge us in order to provide a complete understanding of just how brain networks perform this complex and important social recognition task.     

Sex-contingent face after-effects suggest distinct neural populations code male and female faces

Exposure to faces biases perceptions of subsequently viewed faces. Faces similar to those seen previously are judged more normal and attractive than they were prior to exposure. Here we show sex-contingent after-effects following adaptation to eye-spacing (experiment 1), facial identity (experiment 2) and masculinity (experiment 3). Viewing faces of one sex with increased eye-spacing and faces of the other sex with decreased eye-spacing simultaneously induced opposite after-effects for male and female faces (assessed by normality judgments). Viewing faces transformed in identity or masculinity increased preferences for novel faces with characteristics similar to those viewed only when the sex of the faces presented in the adaptation phase and in post-adaptation preference tests were congruent. Because after-effects reflect changes in responses of neural populations that code faces, our findings indicate that distinct neural populations code male and female faces.     

Perceptual grouping of biological motion promotes binocular rivalry

Investigation of perceptual rivalry between conflicting stimuli presented one to each eye can further understanding of the neural underpinnings of conscious visual perception. During rivalry, visual awareness fluctuates between perceptions of the two stimuli. Here, we demonstrate that high-level perceptual grouping can promote rivalry between stimulus pairs that would otherwise be perceived as nonrivalrous. Perceptual grouping was generated with point-light walker stimuli that simulate human motion, visible only as lights placed on the joints. Although such walking figures are unrecognizable when stationary, recognition judgments as complex as gender and identity can accurately be made from animated displays, demonstrating the efficiency with which our visual system can group dynamic local signals into a globally coherent walking figure. We find that point-light walker stimuli presented one to each eye and in different colors and configurations results in strong rivalry. However, rivalry is minimal when the two walkers are split between the eyes or both presented to one eye. This pattern of results suggests that processing animated walker figures promotes rivalry between signals from the two eyes rather than between higher-level representations of the walkers. This leads us to hypothesize that awareness during binocular rivalry involves the integrated activity of high-level perceptual mechanisms in conjunction with lower-level ocular suppression modulated via cortical feedback.     

Atypicalities in Perceptual Adaptation in Autism Do Not Extend to Perceptual Causality

A recent study showed that adaptation to causal events (collisions) in adults caused subsequent events to be less likely perceived as causal. In this study, we examined if a similar negative adaptation effect for perceptual causality occurs in children, both typically developing and with autism. Previous studies have reported diminished adaptation for face identity, facial configuration and gaze direction in children with autism. To test whether diminished adaptive coding extends beyond high-level social stimuli (such as faces) and could be a general property of autistic perception, we developed a child-friendly paradigm for adaptation of perceptual causality. We compared the performance of 22 children with autism with 22 typically developing children, individually matched on age and ability (IQ scores). We found significant and equally robust adaptation aftereffects for perceptual causality in both groups. There were also no differences between the two groups in their attention, as revealed by reaction times and accuracy in a change-detection task. These findings suggest that adaptation to perceptual causality in autism is largely similar to typical development and, further, that diminished adaptive coding might not be a general characteristic of autism at low levels of the perceptual hierarchy, constraining existing theories of adaptation in autism.     

Abnormal adaptive face-coding mechanisms in children with autism spectrum disorder

In low-level vision, exquisite sensitivity to variation in luminance is achieved by adaptive mechanisms that adjust neural sensitivity to the prevailing luminance level. In high-level vision, adaptive mechanisms contribute to our remarkable ability to distinguish thousands of similar faces [1]. A clear example of this sort of adaptive coding is the face-identity aftereffect [2, 3, 4, 5], in which adaptation to a particular face biases perception toward the opposite identity. Here we investigated face adaptation in children with autism spectrum disorder (ASD) by asking them to discriminate between two face identities, with and without prior adaptation to opposite-identity faces. The ASD group discriminated the identities with the same precision as did the age- and ability-matched control group, showing that face identification per se was unimpaired. However, children with ASD showed significantly less adaptation than did their typical peers, with the amount of adaptation correlating significantly with current symptomatology, and face aftereffects of children with elevated symptoms only one third those of controls. These results show that although children with ASD can learn a simple discrimination between two identities, adaptive face-coding mechanisms are severely compromised, offering a new explanation for previously reported face-perception difficulties [6, 7, 8] and possibly for some of the core social deficits in ASD [9, 10].     

Artificial neural network properties associated with wiring patterns in the visual projections of vertebrates and arthropods

We model the functioning of different wiring schemes in visual projections using artificial neural networks and so speculate on selective factors underlying taxonomic variation in neural architecture. We model the high connective overlap of vertebrates (where networks have a dense mesh of connections) and the less overlapping, more modular architecture of arthropods. We also consider natural variation in these basic wiring schemes. Generally, arthropod networks are as efficient or more efficient in functioning compared to vertebrate networks. They do not show the confusion effect (decreasing targeting accuracy with increasing input group size), and they train as well or better. Arthropod networks are, however, generally poorer at reconstructing novel inputs. The ability of vertebrate networks to effectively process novel stimuli could promote behavioral sophistication and drive the evolution of vertebrate wiring schemes. Vertebrate networks with less connective overlap have, surprisingly, similar or superior properties compared to those with high connective overlap. Thus, the partial connective overlap seen in real vertebrate visual projections may be an optimal, evolved solution. Arthropod networks with and without whole-cell neural connections within neural layers have similar properties. This indicates that neural connections mediated by offshoots of single cells (dendrites) may be fundamental to generating the confusion effect.     

Attention driven phantom vision: measuring the sensory strength of attentional templates and their relation to visual mental imagery and aphantasia

When we search for an object in an array or anticipate attending to a future object, we create an ‘attentional template'' of the object. The definitions of attentional templates and visual imagery share many similarities as well as many of the same neural characteristics. However, the phenomenology of these attentional templates and their neural similarities to visual imagery and perception are rarely, if ever discussed. Here, we investigate the relationship between these two forms of non-retinal phantom vision through the use of the binocular rivalry technique, which allows us to measure the sensory strength of attentional templates in the absence of concurrent perceptual stimuli. We find that attentional templates correlate with both feature-based attention and visual imagery. Attentional templates, like imagery, were significantly disrupted by the presence of irrelevant visual stimuli, while feature-based attention was not. We also found that a special population who lack the ability to visualize (aphantasia), showed evidence of feature-based attention when measured using the binocular rivalry paradigm, but not attentional templates. Taken together, these data suggest functional similarities between attentional templates and visual imagery, advancing the theory of visual imagery as a general simulation tool used across cognition.This article is part of the theme issue ‘Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation’.     

Subcortical discrimination of unperceived objects during binocular rivalry

Rapid identification of behaviorally relevant objects is important for survival. In humans, the neural computations for visually discriminating complex objects involve inferior temporal cortex (IT). However, less detailed but faster form processing may also occur in a phylogenetically older subcortical visual system that terminates in the amygdala. We used binocular rivalry to present stimuli without conscious awareness, thereby eliminating the IT object representation and isolating subcortical visual input to the amygdala. Functional magnetic resonance imaging revealed significant brain activation in the left amygdala but not in object-selective IT in response to unperceived fearful faces compared to unperceived nonface objects. These findings indicate that, for certain behaviorally relevant stimuli, a high-level cortical representation in IT is not required for object discrimination in the amygdala.     

Emergence and transmission of visual awareness through optical coding in the brain: A redox molecular hypothesis on visual mental imagery

Does the primary visual cortex mediate consciousness for higher-level stages of information processing by providing an outlet for mental imagery? Evidence based on neural electrical activity is inconclusive as reflected in the “imagery debate” in cognitive science. Neural information and activity, however, also depend on regulated biophoton (optical) signaling. During encoding and retrieval of visual information, regulated electrical (redox) signals of neurons are converted into synchronized biophoton signals by bioluminescent radical processes. That is, visual information may be represented by regulated biophotons of mitochondrial networks in retinotopically organized cytochrome oxidase-rich neural networks within early visual areas. Therefore, we hypothesize that regulated biophotons can generate intrinsic optical representations in the primary visual cortex and then propagate variably degraded versions along cytochrome oxidase pathways during both perception and imagery. Testing this hypothesis requires to establish a methodology for measurement of in vivo and/or in vitro increases of biophoton emission in humans' brain during phosphene inductions by transcranial magnetic stimulation and to compare the decrease in phosphene thresholds during transcranial magnetic stimulation and imagery. Our hypothesis provides a molecular mechanism for the visual buffer and for imagery as the prevalent communication mode (through optical signaling) within the brain. If confirmed empirically, this hypothesis could resolve the imagery debate and the underlying issue of continuity between perception and abstract thought.     

Imagine that: elevated sensory strength of mental imagery in individuals with Parkinson's disease and visual hallucinations

Visual hallucinations occur when our conscious experience does not accurately reflect external reality. However, these dissociations also regularly occur when we imagine the world around us in the absence of visual stimulation. We used two novel behavioural paradigms to objectively measure visual hallucinations and voluntary mental imagery in 19 individuals with Parkinson''s disease (ten with visual hallucinations; nine without) and ten healthy, age-matched controls. We then used this behavioural overlap to interrogate the connectivity both within and between the major attentional control networks using resting-state functional magnetic resonance imaging. Patients with visual hallucinations had elevated mental imagery strength compared with patients without hallucinations and controls. Specifically, the sensory strength of imagery predicted the frequency of visual hallucinations. Together, hallucinations and mental imagery predicted multiple abnormalities in functional connectivity both within and between the attentional control networks, as measured with resting-state functional magnetic resonance imaging. However, the two phenomena were also dissociable at the neural level, with both mental imagery and visual misperceptions associated with specific abnormalities in attentional network connectivity. Our results provide the first evidence of both the shared and unique neural correlates of these two similar, yet distinct phenomena.     

Abnormal FMRI adaptation to unfamiliar faces in a case of developmental prosopamnesia

In rare cases, damage to the temporal lobe causes a selective impairment in the ability to learn new faces, a condition known as prosopamnesia [1]. Here we present the case of an individual with prosopamnesia in the absence of any acquired structural lesion. "C" shows intact processing of simple and complex nonface objects, but her ability to learn new faces is severely impaired. We used a neural marker of perceptual learning known as repetition suppression to examine functioning within C's fusiform face area (FFA), a region of cortex involved in face perception [2]. For comparison, we examined repetition suppression in the scene-selective parahippocampal place area (PPA) [3]. As expected, normal controls showed significant region-specific attenuation of neural activity across repetitions of each stimulus class. C also showed normal attenuation within the PPA to familiar and unfamiliar scenes, and within the FFA to familiar faces. Critically, however, she failed to show any adaptive change within the FFA for repeated unfamiliar faces, despite a face-specific blood-oxygen-dependent response (BOLD) response in her FFA during viewing of face stimuli. Our findings suggest that in developmental prosopamnesia, the FFA cannot maintain stable representations of new faces for subsequent recall or recognition.     

Adding Years to Your Life (or at Least Looking Like It): A Simple Normalization Underlies Adaptation to Facial Age

Adaptation has been widely used to probe how experience shapes the visual encoding of faces, but the pattern of perceptual changes produced by adaptation and the neural mechanisms these imply remain poorly characterized. We explored how adaptation alters the perceived age of faces, a fundamental facial attribute which can uniquely and reliably be scaled by observers. This allowed us to measure how adaptation to one age level affected the full continuum of perceived ages. Participants guessed the ages of faces ranging from 18–89, before or after adapting to a different set of faces composed of younger, older, or middle-aged adults. Adapting to young or old faces induced opposite linear shifts in perceived age that were independent of the model''s age. Specifically, after adapting to younger or older faces, faces of all ages appeared 2 to 3 years older or younger, respectively. In contrast, middle-aged adaptors induced no aftereffects. This pattern suggests that adaptation leads to a simple and uniform renormalization of age perception, and is consistent with a norm-based neural code for the mechanisms mediating the perception of facial age.     

Hallucinogen persisting perception disorder in neuronal networks with adaptation

We study the spatiotemporal dynamics of neuronal networks with spike frequency adaptation. In particular, we compare the effects of adaptation being either a linear or nonlinear function of neural activity. We find that altering parameters controlling the strength of synaptic connections in the network can lead to spatially structured activity suggestive of symptoms of hallucinogen persisting perception disorder (HPPD). First, we study how both networks track spatially homogeneous flickering stimuli, and find input is encoded as continuous at lower flicker frequencies when the network??s synapses exhibit more net excitation. Mainly, we study instabilities of stimulus-driven traveling pulse solutions, representative of visual trailing phenomena common to HPPD patients. Visual trails are reported as discrete afterimages in the wake of a moving input. Thus, we analyze several solutions arising in response to moving inputs in both networks: an ON state, stimulus-locked pulses, and traveling breathers. We find traveling breathers can arise in both networks when an input moves beyond a critical speed. These possible neural substrates of visual trails occur at slower speeds when the modulation of synaptic connectivity is increased.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.