Changes in face expression recognition caused by additional visual-spatial load

Changes in face expression recognition and EEG synchronization arising from additional load on working memory were studied in healthy adults. Two types of additional task--semantic and visuospatial--were used to load working memory in an experiment with a visual set, formed to facial stimuli. During perception of new facial stimuli, both these types of additional task caused an increase of erroneous face expression recognitions in the form of assimilative illusions. Alpha-band (8-10 Hz) EEG synchronization analysis revealed that additional memory load causes a decrease of frontal attention system input in set-forming and set-shifting. As for theta-band (4-7 Hz) synchronization, it changed ambiguously at additional memory load--in right fronto-temporal region coherence function decreased; other coherence connections, especially intra-hemispheric and in the left hemisphere, increased. At issue is the crucial role of fronto-thalamic and cortico-hippocampal systems in plasticity of visual sets formed to facial expressions.     

Induced cortical electrical activity during different time-spans between warning and target stimuli

A certain alpha-band EEG dynamics was revealed in healthy adults (n = 16) at the interval between a warning and a target stimulus in a simple visuospatial task (subjects were instructed to locate a specific letter in the table of letters). Two series of experiment--either with a 2-sec or a 9-sec inter-stimulus interval were conducted, each consisting of 60 trials. In both series, we observed an induced desynchronization of low alpha (8-10 Hz) at the first second after the warning stimulus and its desynchronization just before the target stimulus. In series with a 9-sec inter-stimulus interval at the 4-6 s of it we observed an alpha-band synchronization, especially distinct in high alpha (10.5-13 Hz). This synchronization gradually reduced towards the end of the inter-stimulus interval. We consider the above changes in alpha-band spectral power during the inter-stimulus interval to be induced by "inner impulsations" caused by an internal representation (set) of the stimuli time-sequence. Changes in the level of cognitive control during the inter-stimulus interval cause increases and decreases in fronto-thalamic system activity, which are manifested in changes of alpha-band spectral power. Analysis of theta-band dynamics suggests that cortico-hippocampal system doesn't participate in this process.     

Induced synchronization/desynchronization of the theta and alpha cortical electrical activity to a face stimuli with increasing visual working memory load

Using a cognitive set to emotional facial expression as a model, induced synchronization/desynchronization of the cortical theta- and alpha-activities were studied in adult healthy people under conditions of increased load on the working memory (additional task of the verbal stimuli recognition). A correlation was found between behavioral (increase in the set rigidity) and electrophysiological (decrease of the induced theta-rhythm synchronization) data. A hypothesis is suggested that the earlier revealed increase in the tonic prestimulus theta-activity and suppression of the poststimulus phasic activation of the cortico-hippocampal system are one of the mechanisms of the decrease in plasticity of the cognitive function of the emotional facial expression recognition under conditions of the increased load on the working memory. Reciprocal relations between two functional systems of the brain activity integration (cortico-hippocampal and fronto-thalamic) in the process of recognition of emotional facial expression are discussed.     

Dependence of plasticity of set to emotional facial expression on working memory load

The effect of increasing working memory load (by introduction of an additional cognitive task into the experimental context) on the recognition of emotional facial expression in a visual set paradigm was studied in healthy adult subjects. The link between plasticity of the cognitive set to emotional facial expression and the working memory was revealed. It was found that an increase in the working memory load was associated with a delay of set shifting in a modified situation. The set became more rigid which appeared as increasing number of trials with erroneous assessments of facial expression in the form of contrast or assimilative illusions. The significance of inner states and priming for the insight into psychophysiological mechanisms of erroneous assessments under conditions of the working memory loading is discussed in terms of the concept of the integration of bottom-up and top-down streams.     

Additional memory load causes changes in induced EEG beta-rhythm in experiments with a visual set formed to facial expression

Subjects were divided into two equal groups 35 healthy subjects each. Formation of the visual set to facial emotion recognition was supplemented with two types of additional task: either visuospatial (to find a target stimulus among others) or verbal (to tell a word from a pseudoword). The results of the experiments were compared to those obtained in similar experiments without the memory load. Changes in the EEG beta rhythm during visual set forming and testing were studied. The EEG was analyzed by wavelet transformation. Changes in the mean level, maximum and latency of the maximum of wavelet coefficient were rated at different stages of the experiment. All these characteristics for the beta rhythm were higher in experiments with both types of additional memory load as compared to those without the memory load.     

Context effects recognition of the emotional facial expression

Influence of additional working memory load on emotional face recognition was studied in healthy adults. Visual set to emotional face expression was experimentally formed, and two types of additional task--visual-spatial or semantic--were embedded in the experiment. Additional task caused less plastic set, i.e., a slower set-shifting. This effect displayed itself in an increase of erroneous facial expression perceptions. The character of these erroneous perceptions (assimilative or contrast or visual illusions) depended on the type of the additional task. Pre-stimulus EEG coherence across experimental trials in theta (4-7), low alpha (8-10 Hz) and beta (14--20) bands was analysed. Data of low-alpha and beta-coherence supported the hypothesis that increased memory load caused less involvement of frontal lobes in selective attention mechanisms that are associated with set-forming. This results in a slower set-shifting. Increased memory load also led to a growth of theta-band coherence in the left hemisphere and its decrease in the right hemisphere. The account of theta-coherence decrease in the right hemisphere between prefrontal and temporal areas for a slower set-shifting is discussed.     

Changes in emotional face expression recognition caused by an additional visuospatial task

Changes in the recognition of facial expression and spatial synchronization of the cortical electrical activity of the θ- and α-potentials caused by load on working memory were studied in healthy adults by introducing an additional semantic or visuospatial task into the context of experiment with a visual set. An increase in the number of erroneous recognitions of facial stimuli in the form of assimilative illusions was revealed in both types of the additional task. The analysis of the function of coherence of the low-frequency α-potentials indicates (8–10 Hz) a decrease in this situation in the number of connections in the frontal cortical divisions with other cortical zones, which is regarded as a lesser involvement of the frontal system of selective attention in set-forming and set-shifting for an emotionally negative facial expression. Spatial synchronization of the θ-activity (4–7 Hz) with an increase in the load on working memory changes ambiguously in different cortical structures: it decreases in the system of the fronto-temporal connections of the right hemisphere; in the other cortical areas, especially in the left hemisphere, and in the system of interhemispheric connections it substantially increases. The facts confirming the hypothesis that the fronto-thalamic and cortico-hippocampal systems are the two key formations involved in changes in the plasticity of cognitive sets for facial expression are discussed.     

Dynamics of the gamma-responses in an 8-second interval between facial and trigger stimuli as dependent the success of task performance

A cognitive set to facial expression was used as a model with the loading on working memory being increased by increasing the interval between the facial and triggering stimuli to 8 seconds. The aim was to determine whether the intensity of brain potentials evoked in a range of 41–60 Hz (the range 15–60 Hz was used) by facial stimuli is associated with the “success” of task performance (mistake rate). An index of average amplitudes of EEG oscillations was used to measure the response to facial stimuli, and γ responses proved to be associated with the number of mistakes in performing the task. The results make it possible to consider the γ responses to facial stimuli as an EEG correlate of the internal states that correspond to adequate actions of the subject in the test with a 8-s interval between the facial and trigger stimuli.     

Changes in the low- and high-frequency oscillations of the EEG α-band in the intervals between meaningful visual stimuli

Differences in induced synchronization of the high-frequency and low-frequency α-rhythm between the group of subjects in whom switching and updating of the cognitive set are not accompanied by errors in recognition of facial expression (the high-plastic set) and the subjects making errors were revealed in healthy adults (n = 35) using the model set to the perception of an angry face. In the former, the well-marked synchronization of the high-frequency α-rhythm occurs in pauses between the trials. This phenomenon is not observed in the latter. Synchronization of both the high-frequency and low-frequency α-rhythm subbands, obviously more pronounced in the subjects who do not make errors in recognizing facial expression, occurs in the middle of an 8-s pause between the set (target) and trigger stimuli. The role of top-down cognitive control, in particular, the top-down inhibitory influences suppressing the action of irrelevant factors in the preparation for processing the target stimulus, and its significance in providing the plastic forms of a cognitive set are discussed.     

Shifting attention within memory representations involves early visual areas

Prior studies have shown that spatial attention modulates early visual cortex retinotopically, resulting in enhanced processing of external perceptual representations. However, it is not clear whether the same visual areas are modulated when attention is focused on, and shifted within a working memory representation. In the current fMRI study participants were asked to memorize an array containing four stimuli. After a delay, participants were presented with a verbal cue instructing them to actively maintain the location of one of the stimuli in working memory. Additionally, on a number of trials a second verbal cue instructed participants to switch attention to the location of another stimulus within the memorized representation. Results of the study showed that changes in the BOLD pattern closely followed the locus of attention within the working memory representation. A decrease in BOLD-activity (V1-V3) was observed at ROIs coding a memory location when participants switched away from this location, whereas an increase was observed when participants switched towards this location. Continuous increased activity was obtained at the memorized location when participants did not switch. This study shows that shifting attention within memory representations activates the earliest parts of visual cortex (including V1) in a retinotopic fashion. We conclude that even in the absence of visual stimulation, early visual areas support shifting of attention within memorized representations, similar to when attention is shifted in the outside world. The relationship between visual working memory and visual mental imagery is discussed in light of the current findings.     

Long-term Retention of Many Visual Patterns by Pigeons

Using a simultaneous discrimination procedure it was shown that pigeons were capable of learning to discriminate 100 different black and white visual patterns from a further 625 similar stimuli, where responses to the former were rewarded and responses to the latter were not rewarded. Tests in which novel stimuli replaced either the rewarded or nonrewarded stimuli showed that the pigeons had not only learned about the 100 positive stimuli but also about the 625 negative stimuli. The fact that novel stimuli enhanced discrimination performance when they replaced the many negative stimuli indicated that the pigeons had categorized the stimuli into two classes, familiar and less familiar. Long-term retention was examined after a 6-month interval. To begin with it seemed poor but a recognition test performed after the subjects had been retrained with a subset of the stimuli after an interval of 7 months confirmed that pigeons are capable of retaining in memory several 100 visual items over an extended period. It is proposed that the initial retrieval weakness was due to a forgetting of the contingencies between stimulus categories and response outcomes. Further tests involving variously modified stimuli indicated that while stimulus size variations had a negative effect on performance, orientation changes did not interfere with recognition, supporting the view that small visual stimuli are memorized by pigeons largely free of orientation labels. The experiment generally confirms that pigeons have the capacity of storing information about a large number of visual stimuli over long periods of time.     

Alpha-rhythm of the EEG in directed attention

Degree of manifestation and synchronization of rhythmic alpha-range components in various cortical areas was studied in healthy adult subjects in conditions of controlled computer experiment by spectral-correlation method of EEG analysis during directed attention previous to discrimination of tactile, auditory and visual stimuli. The obtained results show that in preparation for discrimination of heteromodal stimuli, specific reconstructions take place of alpha-range electrical activity determined by signal modality. These changes are expressed in a local increase of the degree of synchronization of alpha oscillations in those projection and associative brain areas which are connected with the analysis of the given stimuli. Functional significance of these reconstructions is seen in the fact that they are significantly more expressed at correct recognition. A hypothesis is suggested about participation of the alpha-range in neurophysiological mechanisms of directed attention.     

Dynamics of the gamma-band power of induced EEG responses to facial stimuli with increased visual working memory load

The dynamics of power of short-term (0.8 s) induced responses to facial stimuli (wavelet transform in the 15-60 Hz band) were assessed in the study of the visual cognitive set under conditions of different loads on working memory in two groups of subjects. Subjects of the first group had to react only to facial stimuli (n = 29), whereas the second group solved an additional task loading the working memory (they had to find a target stimulus in a matrix of letters, n = 35). We estimated wavelet spectra in the occipital, temporal, central and frontal areas of both hemispheres. In both groups of subjects with the plastic form of set, the power level in the gamma2 band (41-60 Hz) was significantly higher than in subject with the rigid form. In group A at the set-testing stage, the largest increase in the gamma2 band was related to the central areas of the left hemisphere. In more complex situation (group ), the increase in power in the gamma2 and gamma1 (21-40 Hz) bands was observed in the occipital and temporal areas of both hemispheres. At the same time, the EEG power of the central areas in these gamma bands was significantly lower. In the frontal areas there were no significant differences in the dynamics of power between the subjects of both groups.     

Weakening of the Top-Down Inhibitory Influences from the Prefrontal Cortex under the Loading of the Working Memory in Students with Learning Difficulties

Two groups of students (with and without learning difficulties) were studied. In the experimental technique used in experiments, an increased load on the working memory by lengthening the interstimulus interval between the conditioning positive (Go) or inhibitory (NoGo) signals and the triggering stimulus was combined with EEG recording. In students with learning difficulties, the evoked synchronization/desynchronization of low-frequency α-oscillations (8–10 Hz) in individual intersimulation intervals was significantly weakened. The hypothesis of the disturbed coordinating mechanism of selective implicit attention with the involvement of the prefrontal cortex in the organization of cognitive activity in students with learning difficulties is discussed and substantiated.     

Alpha ERD and human visual selective attention

We compared the alpha band EEG depression (event-related desynchnization, ERD) level in two tasks, involving activation of different attentional processes: visual search for a deviant relevant stimulus among many similar ones and visual oddball. Control data for the visual search task consisted of simple viewing of several stimuli being of the same shape as the relevant stimulus in the search trials. Gaze position was verified by eye tracking method. We interpreted alpha band ERD as a correlate of activation of attentional processes. Fixating the target in visual search task caused a significantly larger ERD than fixating the same stimuli in control trials over all leads. We suppose this to be related with task and visual environment complexities. The frontal ERD domination may indicate attentional control over voluntary movements execution (top-down attention). The caudal ERD may be related with updating of visual information as a result of search process (bottom-up attention). Both relevant and irrelevant stimuli in the oddball task also induced alpha band ERD, but it was larger in response to relevant one and reached maximum level over occipital leads. Domination of caudal ERD in oddball task is supposed to indicate bottom-up attention processes.     

Theta and alpha-band EEG spatial synchronization under the conditions of visual set, formed to an angry face expression. A study of 5-11-year-old children

EEG coherence in theta and alpha bands during set-forming and set-shifting was studied in 5-6-year-old (n=18) and 10-11-year-old (n=25) children. Set was formed to visual stimuli (facial photos with emotionally negative expression). Younger children displayed smaller coherence values, especially in the right hemisphere, than older ones. We also revealed differences in theta and alpha band coherence in cases of a rigid and a plastic set. For example, EEG-coherence values were smaller when cognitive processes were relatively rigid (i.e., in a case of a slower set-shifting). A strong correlation between electrophysiological and behavioral data supports the hypothesis that cortico-hippocampal and fronto-thalamic brain integration systems participate in facial expression recognition and provide cognition flexibility.     

Cognitive set depends on the involvement of the ventral and dorsal visual systems

Healthy adults were examined in three series of experiments with formation of an unconscious visual set: 1) the set was formed by repeated presentation of pairs of unequal circles (control); 2) an additional task of recognition of words/pseudowords was introduced into the context of the set-forming trials; 3) in the task additionally introduced, a subject had to spatially localize a certain target letter in a letter matrix. Scores of stability of the visual set to circles were compared. Coherence of the cortical electric activity in the alpha band was analyzed. We revealed a significant decrease in stability (rigidity) of the nonverbal visual set in the series with the additional task of spatial localization of the target stimulus. On the contrary, the set rigidity increased in the series with additional recognition of the verbal stimulus. EEG coherence patterns and behavioral data suggest that successful cognitive performance that demands dynamic situation-dependent shifts of unconscious sets takes place under conditions of alternation of tasks involving, predominantly, either the ventral ("what?") or dorsal ("where?") visual streams and, respectively, anterior or posterior systems of selective attention.     

Reversing Stimulus Timing in Visual Conditioning Leads to Memories with Opposite Valence in Drosophila

Animals need to associate different environmental stimuli with each other regardless of whether they temporally overlap or not. Drosophila melanogaster displays olfactory trace conditioning, where an odor is followed by electric shock reinforcement after a temporal gap, leading to conditioned odor avoidance. Reversing the stimulus timing in olfactory conditioning results in the reversal of memory valence such that an odor that follows shock is later on approached (i.e. relief conditioning). Here, we explored the effects of stimulus timing on memory in another sensory modality, using a visual conditioning paradigm. We found that flies form visual memories of opposite valence depending on stimulus timing and can associate a visual stimulus with reinforcement despite being presented with a temporal gap. These results suggest that associative memories with non-overlapping stimuli and the effect of stimulus timing on memory valence are shared across sensory modalities.     

Selective visual attention in a neurocomputational model of phase oscillators

In order to understand the dynamic property of covert selective visual attention, which is different from the proposed mechanism of the spotlight metaphor, a two-layered network of phase oscillators was developed. The first layer is related to the hippocampus and controls attention focus formation. The second layer is related to the visual cortex, and each cortical oscillator in it simulates an assembly of cells coding for a particular stimulus in the sense of feature binding. Selective visual attention is interpreted as the result of the emergent synchronization of hippocampus oscillators and a part of cortical oscillators. Numerical experiments are presented to illustrate attention focus formation and attention shifting from one set of stimuli to another. From a neurocomputational point of view, our results demonstrate that attention is an emergent property of the dynamical cell assemblies responding to the whole visual field. Received: 2 January 1998 / Accepted in revised form: 10 November 1998     

Multi-Variate EEG Analysis as a Novel Tool to Examine Brain Responses to Naturalistic Music Stimuli

Note onsets in music are acoustic landmarks providing auditory cues that underlie the perception of more complex phenomena such as beat, rhythm, and meter. For naturalistic ongoing sounds a detailed view on the neural representation of onset structure is hard to obtain, since, typically, stimulus-related EEG signatures are derived by averaging a high number of identical stimulus presentations. Here, we propose a novel multivariate regression-based method extracting onset-related brain responses from the ongoing EEG. We analyse EEG recordings of nine subjects who passively listened to stimuli from various sound categories encompassing simple tone sequences, full-length romantic piano pieces and natural (non-music) soundscapes. The regression approach reduces the 61-channel EEG to one time course optimally reflecting note onsets. The neural signatures derived by this procedure indeed resemble canonical onset-related ERPs, such as the N1-P2 complex. This EEG projection was then utilized to determine the Cortico-Acoustic Correlation (CACor), a measure of synchronization between EEG signal and stimulus. We demonstrate that a significant CACor (i) can be detected in an individual listener''s EEG of a single presentation of a full-length complex naturalistic music stimulus, and (ii) it co-varies with the stimuli’s average magnitudes of sharpness, spectral centroid, and rhythmic complexity. In particular, the subset of stimuli eliciting a strong CACor also produces strongly coordinated tension ratings obtained from an independent listener group in a separate behavioral experiment. Thus musical features that lead to a marked physiological reflection of tone onsets also contribute to perceived tension in music.