Facial Expression Aftereffect Revealed by Adaption to Emotion-Invisible Dynamic Bubbled Faces

Visual adaptation is a powerful tool to probe the short-term plasticity of the visual system. Adapting to local features such as the oriented lines can distort our judgment of subsequently presented lines, the tilt aftereffect. The tilt aftereffect is believed to be processed at the low-level of the visual cortex, such as V1. Adaptation to faces, on the other hand, can produce significant aftereffects in high-level traits such as identity, expression, and ethnicity. However, whether face adaptation necessitate awareness of face features is debatable. In the current study, we investigated whether facial expression aftereffects (FEAE) can be generated by partially visible faces. We first generated partially visible faces using the bubbles technique, in which the face was seen through randomly positioned circular apertures, and selected the bubbled faces for which the subjects were unable to identify happy or sad expressions. When the subjects adapted to static displays of these partial faces, no significant FEAE was found. However, when the subjects adapted to a dynamic video display of a series of different partial faces, a significant FEAE was observed. In both conditions, subjects could not identify facial expression in the individual adapting faces. These results suggest that our visual system is able to integrate unrecognizable partial faces over a short period of time and that the integrated percept affects our judgment on subsequently presented faces. We conclude that FEAE can be generated by partial face with little facial expression cues, implying that our cognitive system fills-in the missing parts during adaptation, or the subcortical structures are activated by the bubbled faces without conscious recognition of emotion during adaptation.     

Non-Conscious Processing of Motion Coherence Can Boost Conscious Access

Research on the scope and limits of non-conscious vision can advance our understanding of the functional and neural underpinnings of visual awareness. Here we investigated whether distributed local features can be bound, outside of awareness, into coherent patterns. We used continuous flash suppression (CFS) to create interocular suppression, and thus lack of awareness, for a moving dot stimulus that varied in terms of coherence with an overall pattern (radial flow). Our results demonstrate that for radial motion, coherence favors the detection of patterns of moving dots even under interocular suppression. Coherence caused dots to break through the masks more often: this indicates that the visual system was able to integrate low-level motion signals into a coherent pattern outside of visual awareness. In contrast, in an experiment using meaningful or scrambled biological motion we did not observe any increase in the sensitivity of detection for meaningful patterns. Overall, our results are in agreement with previous studies on face processing and with the hypothesis that certain features are spatiotemporally bound into coherent patterns even outside of attention or awareness.     

The dynamics of visual adaptation to faces

Several recent demonstrations using visual adaptation have revealed high-level aftereffects for complex patterns including faces. While traditional aftereffects involve perceptual distortion of simple attributes such as orientation or colour that are processed early in the visual cortical hierarchy, face adaptation affects perceived identity and expression, which are thought to be products of higher-order processing. And, unlike most simple aftereffects, those involving faces are robust to changes in scale, position and orientation between the adapting and test stimuli. These differences raise the question of how closely related face aftereffects are to traditional ones. Little is known about the build-up and decay of the face aftereffect, and the similarity of these dynamic processes to traditional aftereffects might provide insight into this relationship. We examined the effect of varying the duration of both the adapting and test stimuli on the magnitude of perceived distortions in face identity. We found that, just as with traditional aftereffects, the identity aftereffect grew logarithmically stronger as a function of adaptation time and exponentially weaker as a function of test duration. Even the subtle aspects of these dynamics, such as the power-law relationship between the adapting and test durations, closely resembled that of other aftereffects. These results were obtained with two different sets of face stimuli that differed greatly in their low-level properties. We postulate that the mechanisms governing these shared dynamics may be dissociable from the responses of feature-selective neurons in the early visual cortex.     

Interference between Conscious and Unconscious Facial Expression Information

There is ample evidence to show that many types of visual information, including emotional information, could be processed in the absence of visual awareness. For example, it has been shown that masked subliminal facial expressions can induce priming and adaptation effects. However, stimulus made invisible in different ways could be processed to different extent and have differential effects. In this study, we adopted a flanker type behavioral method to investigate whether a flanker rendered invisible through Continuous Flash Suppression (CFS) could induce a congruency effect on the discrimination of a visible target. Specifically, during the experiment, participants judged the expression (either happy or fearful) of a visible face in the presence of a nearby invisible face (with happy or fearful expression). Results show that participants were slower and less accurate in discriminating the expression of the visible face when the expression of the invisible flanker face was incongruent. Thus, facial expression information rendered invisible with CFS and presented a different spatial location could enhance or interfere with consciously processed facial expression information.     

Access to Awareness for Faces during Continuous Flash Suppression Is Not Modulated by Affective Knowledge

It is a controversially debated topic whether stimuli can be analyzed up to the semantic level when they are suppressed from visual awareness during continuous flash suppression (CFS). Here, we investigated whether affective knowledge, i.e., affective biographical information about faces, influences the time it takes for initially invisible faces with neutral expressions to overcome suppression and break into consciousness. To test this, we used negative, positive, and neutral famous faces as well as initially unfamiliar faces, which were associated with negative, positive or neutral biographical information. Affective knowledge influenced ratings of facial expressions, corroborating recent evidence and indicating the success of our affective learning paradigm. Furthermore, we replicated shorter suppression durations for upright than for inverted faces, demonstrating the suitability of our CFS paradigm. However, affective biographical information did not modulate suppression durations for newly learned faces, and even though suppression durations for famous faces were influenced by affective knowledge, these effects did not differ between upright and inverted faces, indicating that they might have been due to low-level visual differences. Thus, we did not obtain unequivocal evidence for genuine influences of affective biographical information on access to visual awareness for faces during CFS.     

Adults' awareness of faces follows newborns' looking preferences

From the first days of life, humans preferentially orient towards upright faces, likely reflecting innate subcortical mechanisms. Here, we show that binocular rivalry can reveal face detection mechanisms in adults that are surprisingly similar to inborn face detection mechanism. We used continuous flash suppression (CFS), a variant of binocular rivalry, to render stimuli invisible at the beginning of each trial and measured the time upright and inverted stimuli needed to overcome such interocular suppression. Critically, specific stimulus properties previously shown to modulate looking preferences in neonates similarly modulated adults' awareness of faces presented during CFS. First, the advantage of upright faces in overcoming CFS was strongly modulated by contrast polarity and direction of illumination. Second, schematic patterns consisting of three dark blobs were suppressed for shorter durations when the arrangement of these blobs respected the face-like configuration of the eyes and the mouth, and this effect was modulated by contrast polarity. No such effects were obtained in a binocular control experiment not involving CFS, suggesting a crucial role for face-sensitive mechanisms operating outside of conscious awareness. These findings indicate that visual awareness of faces in adults is governed by perceptual mechanisms that are sensitive to similar stimulus properties as those modulating newborns' face preferences.     

Visually induced plasticity of auditory spatial perception in macaques

When experiencing spatially disparate visual and auditory stimuli, a common percept is that the sound originates from the location of the visual stimulus, an illusion known as the ventriloquism effect. This illusion can persist for tens of minutes, a phenomenon termed the ventriloquism aftereffect. The underlying neuronal mechanisms of this rapidly induced plasticity remain unclear; indeed, it remains untested whether similar multimodal interactions occur in other species. We therefore tested whether macaque monkeys experience the ventriloquism aftereffect similar to the way humans do. The ability of two monkeys to determine which side of the midline a sound was presented from was tested before and after a period of 20-60 min in which the monkeys experienced either spatially identical or spatially disparate auditory and visual stimuli. In agreement with human studies, the monkeys did experience a shift in their auditory spatial perception in the direction of the spatially disparate visual stimulus, and the aftereffect did not transfer across sounds that differed in frequency by two octaves. These results show that macaque monkeys experience the ventriloquism aftereffect similar to the way humans do in all tested respects, indicating that these multimodal interactions are a basic phenomenon of the central nervous system.     

Feature-based attention modulates orientation-selective responses in human visual cortex

How does feature-based attention modulate neural responses? We used adaptation to quantify the effect of feature-based attention on orientation-selective responses in human visual cortex. Observers were adapted to two superimposed oblique gratings while attending to one grating only. We measured the magnitude of attention-induced orientation-selective adaptation both psychophysically, by the behavioral tilt aftereffect, and physiologically, using fMRI response adaptation. We found evidence for orientation-selective attentional modulation of neuronal responses-a lower fMRI response for the attended than the unattended orientation-in multiple visual areas, and a significant correlation between the magnitude of the tilt aftereffect and that of fMRI response adaptation in V1, the earliest site of orientation coding. These results show that feature-based attention can selectively increase the response of neuronal subpopulations that prefer the attended feature, even when the attended and unattended features are coded in the same visual areas and share the same retinotopic location.     

The Body as a Tool for Anger Awareness—Differential Effects of Angry Facial and Bodily Expressions on Suppression from Awareness

Emotional signals are perceived whether or not we are aware of it. The evidence so far mostly came from studies with facial expressions. Here, we investigated whether the pattern of non-conscious face expression perception is found for whole body expressions. Continuous flash suppression (CFS) was used to measure the time for neutral, fearful, and angry facial or bodily expressions to break from suppression. We observed different suppression time patterns for emotions depending on whether the stimuli were faces or bodies. The suppression time for anger was shortest for bodily expressions, but longest for the facial expressions. This pattern indicates different processing and detection mechanisms for faces and bodies outside awareness, and suggests that awareness mechanisms associated with dorsal structures might play a role in becoming conscious of angry bodily expressions.     

Familiar Face Detection in 180ms

The visual system is tuned for rapid detection of faces, with the fastest choice saccade to a face at 100ms. Familiar faces have a more robust representation than do unfamiliar faces, and are detected faster in the absence of awareness and with reduced attentional resources. Faces of family and close friends become familiar over a protracted period involving learning the unique visual appearance, including a view-invariant representation, as well as person knowledge. We investigated the effect of personal familiarity on the earliest stages of face processing by using a saccadic-choice task to measure how fast familiar face detection can happen. Subjects made correct and reliable saccades to familiar faces when unfamiliar faces were distractors at 180ms—very rapid saccades that are 30 to 70ms earlier than the earliest evoked potential modulated by familiarity. By contrast, accuracy of saccades to unfamiliar faces with familiar faces as distractors did not exceed chance. Saccades to faces with object distractors were even faster (110 to 120 ms) and equivalent for familiar and unfamiliar faces, indicating that familiarity does not affect ultra-rapid saccades. We propose that detectors of diagnostic facial features for familiar faces develop in visual cortices through learning and allow rapid detection that precedes explicit recognition of identity.     

Colour Terms Affect Detection of Colour and Colour-Associated Objects Suppressed from Visual Awareness

The idea that language can affect how we see the world continues to create controversy. A potentially important study in this field has shown that when an object is suppressed from visual awareness using continuous flash suppression (a form of binocular rivalry), detection of the object is differently affected by a preceding word prime depending on whether the prime matches or does not match the object. This may suggest that language can affect early stages of vision. We replicated this paradigm and further investigated whether colour terms likewise influence the detection of colours or colour-associated object images suppressed from visual awareness by continuous flash suppression. This method presents rapidly changing visual noise to one eye while the target stimulus is presented to the other. It has been shown to delay conscious perception of a target for up to several minutes. In Experiment 1 we presented greyscale photos of objects. They were either preceded by a congruent object label, an incongruent label, or white noise. Detection sensitivity (d’) and hit rates were significantly poorer for suppressed objects preceded by an incongruent label compared to a congruent label or noise. In Experiment 2, targets were coloured discs preceded by a colour term. Detection sensitivity was significantly worse for suppressed colour patches preceded by an incongruent colour term as compared to a congruent term or white noise. In Experiment 3 targets were suppressed greyscale object images preceded by an auditory presentation of a colour term. On congruent trials the colour term matched the object’s stereotypical colour and on incongruent trials the colour term mismatched. Detection sensitivity was significantly poorer on incongruent trials than congruent trials. Overall, these findings suggest that colour terms affect awareness of coloured stimuli and colour- associated objects, and provide new evidence for language-perception interaction in the brain.     

Differential roles of distracters in reflexive and memory-based localization

We investigated the effects of spatial and temporal factors on manual localization of a visual target by measuring accuracy, precision, and bias. Spatial factors included manipulation of display as with or without distracters, with invariant or variant distracters, and with near or far distracters, respectively, in Experiments 1, 2, and 3. The target and distracters were of 1degrees dots differing only by luminance parameter; they were presented concurrently for 150 or 1000 ms while observers had to memorize the target location maintaining a fixed gaze. The observers' task was to reproduce the location of the target with a mouse cursor available 150 ms following stimuli offset. Results from all experiments showed that localization performance for a briefly exposed target was as accurate and precise as that for a long exposed target. Moreover, manipulation of spatial factors had no systematic effects on accuracy and precision except that near distracters yielded higher precision. Interestingly, localization performance was unbiased in 150 ms condition when there were distracters in the display, while being biased towards the fovea in 1000 ms condition regardless of their presence or absence. These results suggest a temporal dynamics in dominance-suppression between egocentric and exocentric cues in the construction of memory for location.     

To what extent are emotional visual stimuli processed without attention and awareness?

In the past few years, important contributions have been made to the study of emotional visual perception. Researchers have reported responses to emotional stimuli in the human amygdala under some unattended conditions (i.e. conditions in which the focus of attention was diverted away from the stimuli due to task instructions), during visual masking and during binocular suppression. Taken together, these results reveal the relative degree of autonomy of emotional processing. At the same time, however, important limitations to the notion of complete automaticity have been revealed. Effects of task context and attention have been shown, as well as large inter-subject differences in sensitivity to the detection of masked fearful faces (whereby briefly presented, target fearful faces are immediately followed by a neutral face that 'masks' the initial face). A better understanding of the neural basis of emotional perception and how it relates to visual attention and awareness is likely to require further refinement of the concepts of automaticity and awareness.     

Ignored Faces Produce Figural Face Aftereffects

Following adaptation to faces with contracted (or expanded) internal features, faces previously perceived as normal appear distorted in the opposite direction. This figural face aftereffect suggests face-coding mechanisms adapt to changes in the spatial relations of features and/or the global structure of faces. Here, we investigated whether the figural aftereffect requires spatial attention. Participants ignored a distorted adapting face and performed a highly demanding letter-count task. Before and after adaptation, participants rated the normality of morphed distorted faces ranging from 50% contracted through undistorted to 50% expanded. A robust aftereffect was observed. These results suggest that the figural face aftereffect can occur in the absence of spatial attention, even when the attentional demands of the relevant task are high.     

Independent binocular rivalry processes for motion and form

During binocular rivalry, conflicting monocular images undergo alternating suppression. This study explores rivalry suppression by probing visual sensitivity during rivalry with various probe stimuli. When two faces engage in rivalry, sensitivity to face probes is reduced 4-fold during suppression. Rivaling global motions also rivaled very deeply when probed with a global motion. However, in a surprising finding, sensitivity to face probes is completely unimpaired during global motion rivalry, and motion sensitivity is unimpaired during face rivalry. This suggests that rivalry suppression is localized to the neurons representing the image conflict, which means that probes of a different kind suffer no suppression. Sensibly, this would leave visual processes not involved in rivalry free to function normally.     

Attentional demands of movement observation as tested by a dual task approach

Movement observation (MO) has been shown to activate the motor cortex of the observer as indicated by an increase of corticomotor excitability for muscles involved in the observed actions. Moreover, behavioral work has strongly suggested that this process occurs in a near-automatic manner. Here we further tested this proposal by applying transcranial magnetic stimulation (TMS) when subjects observed how an actor lifted objects of different weights as a single or a dual task. The secondary task was either an auditory discrimination task (experiment 1) or a visual discrimination task (experiment 2). In experiment 1, we found that corticomotor excitability reflected the force requirements indicated in the observed movies (i.e. higher responses when the actor had to apply higher forces). Interestingly, this effect was found irrespective of whether MO was performed as a single or a dual task. By contrast, no such systematic modulations of corticomotor excitability were observed in experiment 2 when visual distracters were present. We conclude that interference effects might arise when MO is performed while competing visual stimuli are present. However, when a secondary task is situated in a different modality, neural responses are in line with the notion that the observers motor system responds in a near-automatic manner. This suggests that MO is a task with very low cognitive demands which might be a valuable supplement for rehabilitation training, particularly, in the acute phase after the incident or in patients suffering from attention deficits. However, it is important to keep in mind that visual distracters might interfere with the neural response in M1.     

Investigating saccade programming in the praying mantis Tenodera aridifolia using distracter interference paradigms

To investigate the saccadic system in the mantis, I applied distracter interference paradigms. These involved presenting the mantis with a fixation target and one or several distracters supposed to affect saccades towards the target. When a single target was presented, a medium-sized target located in its lower visual field elicited higher rates of saccade response. This preference for target size and position was also observed when a target and a distracter were presented simultaneously. That is, the mantis chose and fixated the target rather than a distracter that was much smaller or larger than the target, or was located above the target. Furthermore, the mantis' preference was not affected by increasing the number of distracters. However, the presence of the distracter decreased the occurrence rate of saccade and increased the response time to saccade. I conclude that distracter interference paradigms are an effective way of investigating the visual processing underlying saccade generation in the mantis. Possible mechanisms of saccade generation in the mantis are discussed.     

Frequent Words Do Not Break Continuous Flash Suppression Differently from Infrequent or Nonexistent Words: Implications for Semantic Processing of Words in the Absence of Awareness

Continuous flash suppression (CFS) has been used as a paradigm to probe the extent to which word stimuli are processed in the absence of awareness. In the two experiments reported here, no evidence is obtained that word stimuli are processed up to the semantic level when suppressed through CFS. In Experiment 1, word stimuli did not break suppression faster than their pseudo-word variants nor was suppression time modulated by word frequency. Experiment 2 replicated these findings, but more critically showed that differential effects can be obtained with this paradigm using a simpler stimulus. In addition, pixel density of the stimuli did prove to be related to suppression time in both experiments, indicating that the paradigm is sensitive to differences in detectability. A third and final experiment replicated the well-known face inversion effect using the same set-up as Experiments 1 and 2, thereby demonstrating that the employed methodology can capture more high-level effects as well. These results are discussed in the context of previous evidence on unconscious semantic processing and two potential explanations are advanced. Specifically, it is argued that CFS might act at a level too low in the visual system for high-level effects to be observed or that the widely used breaking CFS paradigm is merely ill-suited to capture effects in the context of words.     

Subitization and attentional engagement by transient stimuli

A series of experiments investigated the visual selection of moving and static items during enumeration. Small numbers of visual targets can be enumerated with little increase in reaction time and error with set size, a process referred to as 'subitization'. The number of items that can be subitized' is typically between one and four and known as the subitization range. This study looked for evidence of subitizing of subsets of items presented on a computer display. Fast and accurate enumeration was found for random configurations of moving targets even when presented among static distracters. This was not the case for static targets presented among moving or transient distracters. RTs to these targets were longer and showed a steady increase in RT with target number, even in the subitization range. However, when static targets and moving distracters were presented foveally, fast enumeration/subitization of the static targets was again possible. This was not due to reduced inter-item spacing, since linear effects of the number of targets still emerged when stimuli were presented peripherally but the size-spacing ratio was matched to the foveal presentations. There was indication that instead performance reflected perceived differences in movement speed for stimuli presented in parafoveal and more peripheral retinal regions. In support of this, subitization of static items improved as the movement speed of the distracters increased. The data suggest that the processes supporting subitization are highly sensitive to dynamic stimuli and depend on the ease of segmentation between static and moving arrays.