Audiovisual integration of speech falters under high attention demands

One of the most commonly cited examples of human multisensory integration occurs during exposure to natural speech, when the vocal and the visual aspects of the signal are integrated in a unitary percept. Audiovisual association of facial gestures and vocal sounds has been demonstrated in nonhuman primates and in prelinguistic children, arguing for a general basis for this capacity. One critical question, however, concerns the role of attention in such multisensory integration. Although both behavioral and neurophysiological studies have converged on a preattentive conceptualization of audiovisual speech integration, this mechanism has rarely been measured under conditions of high attentional load, when the observers' attention resources are depleted. We tested the extent to which audiovisual integration was modulated by the amount of available attentional resources by measuring the observers' susceptibility to the classic McGurk illusion in a dual-task paradigm. The proportion of visually influenced responses was severely, and selectively, reduced if participants were concurrently performing an unrelated visual or auditory task. In contrast with the assumption that crossmodal speech integration is automatic, our results suggest that these multisensory binding processes are subject to attentional demands.     

Decreased Coherent Motion Discrimination in Autism Spectrum Disorder: The Role of Attentional Zoom-Out Deficit

Autism spectrum disorder (ASD) has been associated with decreased coherent dot motion (CDM) performance, a task that measures magnocellular sensitivity as well as fronto-parietal attentional integration processing. In order to clarify the role of spatial attention in CDM tasks, we measured the perception of coherently moving dots displayed in the central or peripheral visual field in ASD and typically developing children. A dorsal-stream deficit in children with ASD should predict a generally poorer performance in both conditions. In our study, however, we show that in children with ASD, CDM perception was selectively impaired in the central condition. In addition, in the ASD group, CDM efficiency was correlated to the ability to zoom out the attentional focus. Importantly, autism symptoms severity was related to both the CDM and attentional zooming-out impairment. These findings suggest that a dysfunction in the attentional network might help to explain decreased CDM discrimination as well as the “core” social cognition deficits of ASD.     

Effect of psychostimulants on distinct attentional parameters in attentional deficit/hyperactivity disorder

Although there is extensive literature about the effects of stimulants on sustained attention tasks in attentional deficit/hyperactivity disorder (ADHD), little is known about the effect of these drugs on other attentional tasks involving different neural systems. In this study we measured the effect of stimulants on ADHD children, both in the electroencephalographic (EEG) activity during sustained attentional tasks and in psychometric performance during selective attentional tasks. These tasks are known to rely on different cortical networks. Our results in children medicated with 10 mg of d-amphetamine administered 60 min before the study indicate (i) a significant increase in amplitude but not latency of the P300 component of the event-related potential (ERP) during the sustained attentional task and (ii) a significant improvement in the reaction times and correct responses in the selective attentional task. In addition to supporting the use of stimulants in children with attentional deficit/hyperactivity disorder, these results show a multifocal activity improvement of cortical structures linked to dopamine, and interestingly, to attention. All these analyses are framed in a wider study of diverse attentional functions in this syndrome.     

Recalibration of the Multisensory Temporal Window of Integration Results from Changing Task Demands

The notion of the temporal window of integration, when applied in a multisensory context, refers to the breadth of the interval across which the brain perceives two stimuli from different sensory modalities as synchronous. It maintains a unitary perception of multisensory events despite physical and biophysical timing differences between the senses. The boundaries of the window can be influenced by attention and past sensory experience. Here we examined whether task demands could also influence the multisensory temporal window of integration. We varied the stimulus onset asynchrony between simple, short-lasting auditory and visual stimuli while participants performed two tasks in separate blocks: a temporal order judgment task that required the discrimination of subtle auditory-visual asynchronies, and a reaction time task to the first incoming stimulus irrespective of its sensory modality. We defined the temporal window of integration as the range of stimulus onset asynchronies where performance was below 75% in the temporal order judgment task, as well as the range of stimulus onset asynchronies where responses showed multisensory facilitation (race model violation) in the reaction time task. In 5 of 11 participants, we observed audio-visual stimulus onset asynchronies where reaction time was significantly accelerated (indicating successful integration in this task) while performance was accurate in the temporal order judgment task (indicating successful segregation in that task). This dissociation suggests that in some participants, the boundaries of the temporal window of integration can adaptively recalibrate in order to optimize performance according to specific task demands.     

Multisensory constraints on awareness

Given that multiple senses are often stimulated at the same time, perceptual awareness is most likely to take place in multisensory situations. However, theories of awareness are based on studies and models established for a single sense (mostly vision). Here, we consider the methodological and theoretical challenges raised by taking a multisensory perspective on perceptual awareness. First, we consider how well tasks designed to study unisensory awareness perform when used in multisensory settings, stressing that studies using binocular rivalry, bistable figure perception, continuous flash suppression, the attentional blink, repetition blindness and backward masking can demonstrate multisensory influences on unisensory awareness, but fall short of tackling multisensory awareness directly. Studies interested in the latter phenomenon rely on a method of subjective contrast and can, at best, delineate conditions under which individuals report experiencing a multisensory object or two unisensory objects. As there is not a perfect match between these conditions and those in which multisensory integration and binding occur, the link between awareness and binding advocated for visual information processing needs to be revised for multisensory cases. These challenges point at the need to question the very idea of multisensory awareness.     

The Competitive Influences of Perceptual Load and Working Memory Guidance on Selective Attention

The perceptual load theory in selective attention literature proposes that the interference from task-irrelevant distractor is eliminated when perceptual capacity is fully consumed by task-relevant information. However, the biased competition model suggests that the contents of working memory (WM) can guide attentional selection automatically, even when this guidance is detrimental to visual search. An intriguing but unsolved question is what will happen when selective attention is influenced by both perceptual load and WM guidance. To study this issue, behavioral performances and event-related potentials (ERPs) were recorded when participants were presented with a cue to either identify or hold in memory and had to perform a visual search task subsequently, under conditions of low or high perceptual load. Behavioural data showed that high perceptual load eliminated the attentional capture by WM. The ERP results revealed an obvious WM guidance effect in P1 component with invalid trials eliciting larger P1 than neutral trials, regardless of the level of perceptual load. The interaction between perceptual load and WM guidance was significant for the posterior N1 component. The memory guidance effect on N1 was eliminated by high perceptual load. Standardized Low Resolution Electrical Tomography Analysis (sLORETA) showed that the WM guidance effect and the perceptual load effect on attention can be localized into the occipital area and parietal lobe, respectively. Merely identifying the cue produced no effect on the P1 or N1 component. These results suggest that in selective attention, the information held in WM could capture attention at the early stage of visual processing in the occipital cortex. Interestingly, this initial capture of attention by WM could be modulated by the level of perceptual load and the parietal lobe mediates target selection at the discrimination stage.     

Reduced Inhibition of Return to Food Images in Obese Individuals

Previous research has shown that obese individuals may be biased towards attending to food over non-food information, and this bias may contribute to the development and/or maintenance of obesity. The present study sought to extend our understanding of maladaptive attentional processing in this population by investigating whether obese individuals have difficulty in disengaging attention from food compared with non-food images, relative to normal-weight controls. To address this question, we measured inhibition of return (IOR) in an attentional cueing task. The participants were 29 obese and 35 normal-weight satiated females without eating disorders. The obese group displayed less IOR to food images than the normal-weight group, while there was no difference in IOR between the groups for non-food images. This suggests that obese females have greater difficulty disengaging attention from food than normal-weight females. Our findings provide a new focus for studies investigating maintenance factors in obesity and are discussed in relation to a theory of incentive-sensitisation.     

The Pattern and Loci of Training-Induced Brain Changes in Healthy Older Adults Are Predicted by the Nature of the Intervention

There is enormous interest in designing training methods for reducing cognitive decline in healthy older adults. Because it is impaired with aging, multitasking has often been targeted and has been shown to be malleable with appropriate training. Investigating the effects of cognitive training on functional brain activation might provide critical indication regarding the mechanisms that underlie those positive effects, as well as provide models for selecting appropriate training methods. The few studies that have looked at brain correlates of cognitive training indicate a variable pattern and location of brain changes - a result that might relate to differences in training formats. The goal of this study was to measure the neural substrates as a function of whether divided attentional training programs induced the use of alternative processes or whether it relied on repeated practice. Forty-eight older adults were randomly allocated to one of three training programs. In the SINGLE REPEATED training, participants practiced an alphanumeric equation and a visual detection task, each under focused attention. In the DIVIDED FIXED training, participants practiced combining verification and detection by divided attention, with equal attention allocated to both tasks. In the DIVIDED VARIABLE training, participants completed the task by divided attention, but were taught to vary the attentional priority allocated to each task. Brain activation was measured with fMRI pre- and post-training while completing each task individually and the two tasks combined. The three training programs resulted in markedly different brain changes. Practice on individual tasks in the SINGLE REPEATED training resulted in reduced brain activation whereas DIVIDED VARIABLE training resulted in a larger recruitment of the right superior and middle frontal gyrus, a region that has been involved in multitasking. The type of training is a critical factor in determining the pattern of brain activation.     

Attentional Demands Influence Vocal Compensations to Pitch Errors Heard in Auditory Feedback

Auditory feedback is required to maintain fluent speech. At present, it is unclear how attention modulates auditory feedback processing during ongoing speech. In this event-related potential (ERP) study, participants vocalized/a/, while they heard their vocal pitch suddenly shifted downward a ½ semitone in both single and dual-task conditions. During the single-task condition participants passively viewed a visual stream for cues to start and stop vocalizing. In the dual-task condition, participants vocalized while they identified target stimuli in a visual stream of letters. The presentation rate of the visual stimuli was manipulated in the dual-task condition in order to produce a low, intermediate, and high attentional load. Visual target identification accuracy was lowest in the high attentional load condition, indicating that attentional load was successfully manipulated. Results further showed that participants who were exposed to the single-task condition, prior to the dual-task condition, produced larger vocal compensations during the single-task condition. Thus, when participants’ attention was divided, less attention was available for the monitoring of their auditory feedback, resulting in smaller compensatory vocal responses. However, P1-N1-P2 ERP responses were not affected by divided attention, suggesting that the effect of attentional load was not on the auditory processing of pitch altered feedback, but instead it interfered with the integration of auditory and motor information, or motor control itself.     

Impaired Prefrontal Hemodynamic Maturation in Autism and Unaffected Siblings

Background

Dysfunctions of the prefrontal cortex have been previously reported in individuals with autism spectrum disorders (ASD). Previous studies reported that first-degree relatives of individuals with ASD show atypical brain activity during tasks associated with social function. However, developmental changes in prefrontal dysfunction in ASD and genetic influences on the phenomena remain unclear. In the present study, we investigated the change in hemoglobin concentration in the prefrontal cortex as measured with near-infrared spectroscopy, in children and adults with ASD during the letter fluency test. Moreover, to clarify the genetic influences on developmental changes in the prefrontal dysfunction in ASD, unaffected siblings of the ASD participants were also assessed.

Methodology/Principal Findings

Study participants included 27 individuals with high-functioning ASD, age- and IQ-matched 24 healthy non-affected siblings, and 27 unrelated healthy controls aged 5 to 39 years. The relative concentration of hemoglobin ([Hb]) in the prefrontal cortex was measured during the letter fluency task. For children, neither the [oxy-Hb] change during the task nor task performances differed significantly among three groups. For adults, the [oxy-Hb] increases during the task were significantly smaller in the bilateral prefrontal cortex in ASD than those in control subjects, although task performances were similar. In the adult siblings the [oxy-Hb] change was intermediate between those in controls and ASDs.

Conclusion/Significance

Although indirectly due to a cross-sectional design, the results of this study indicate altered age-related change of prefrontal activity during executive processing in ASD. This is a first near-infrared spectroscopy study that implies alteration in the age-related changes of prefrontal activity in ASD and genetic influences on the phenomena.     

Experience with adults shapes multisensory representation of social familiarity in the brain of a songbird

Social animals learn to perceive their social environment, and their social skills and preferences are thought to emerge from greater exposure to and hence familiarity with some social signals rather than others. Familiarity appears to be tightly linked to multisensory integration. The ability to differentiate and categorize familiar and unfamiliar individuals and to build a multisensory representation of known individuals emerges from successive social interactions, in particular with adult, experienced models. In different species, adults have been shown to shape the social behavior of young by promoting selective attention to multisensory cues. The question of what representation of known conspecifics adult-deprived animals may build therefore arises. Here we show that starlings raised with no experience with adults fail to develop a multisensory representation of familiar and unfamiliar starlings. Electrophysiological recordings of neuronal activity throughout the primary auditory area of these birds, while they were exposed to audio-only or audiovisual familiar and unfamiliar cues, showed that visual stimuli did, as in wild-caught starlings, modulate auditory responses but that, unlike what was observed in wild-caught birds, this modulation was not influenced by familiarity. Thus, adult-deprived starlings seem to fail to discriminate between familiar and unfamiliar individuals. This suggests that adults may shape multisensory representation of known individuals in the brain, possibly by focusing the young's attention on relevant, multisensory cues. Multisensory stimulation by experienced, adult models may thus be ubiquitously important for the development of social skills (and of the neural properties underlying such skills) in a variety of species.     

Covert tracking: a combined ERP and fixational eye movement study

Attention can be directed to particular spatial locations, or to objects that appear at anticipated points in time. While most work has focused on spatial or temporal attention in isolation, we investigated covert tracking of smoothly moving objects, which requires continuous coordination of both. We tested two propositions about the neural and cognitive basis of this operation: first that covert tracking is a right hemisphere function, and second that pre-motor components of the oculomotor system are responsible for driving covert spatial attention during tracking. We simultaneously recorded event related potentials (ERPs) and eye position while participants covertly tracked dots that moved leftward or rightward at 12 or 20°/s. ERPs were sensitive to the direction of target motion. Topographic development in the leftward motion was a mirror image of the rightward motion, suggesting that both hemispheres contribute equally to covert tracking. Small shifts in eye position were also lateralized according to the direction of target motion, implying covert activation of the oculomotor system. The data addresses two outstanding questions about the nature of visuospatial tracking. First, covert tracking is reliant upon a symmetrical frontoparietal attentional system, rather than being right lateralized. Second, this same system controls both pursuit eye movements and covert tracking.     

Decreased Left Caudate Volume Is Associated with Increased Severity of Autistic-Like Symptoms in a Cohort of ADHD Patients and Their Unaffected Siblings

Autism spectrum disorder (ASD) symptoms frequently occur in individuals with attention-deficit/hyperactivity disorder (ADHD). While there is evidence that both ADHD and ASD have differential structural brain correlates, knowledge of the structural brain profile of individuals with ADHD with raised ASD symptoms is limited. The presence of ASD-like symptoms was measured by the Children''s Social Behavior Questionnaire (CSBQ) in a sample of typically developing controls (n = 154), participants with ADHD (n = 239), and their unaffected siblings (n = 144) between the ages of 8 and 29. Structural magnetic resonance imaging (MRI) correlates of ASD ratings were analysed by studying the relationship between ASD ratings and grey matter volumes using mixed effects models which controlled for ADHD symptom count and total brain volume. ASD ratings were significantly elevated in participants with ADHD relative to controls and unaffected siblings. For the entire group (participants with ADHD, unaffected siblings and TD controls), mixed effect models revealed that the left caudate nucleus volume was negatively correlated with ASD ratings (t = 2.83; P = 0.005). The current findings are consistent with the role of the caudate nucleus in executive function, including the selection of goals based on the evaluation of action outcomes and the use of social reward to update reward representations. There is a specific volumetric profile associated with subclinical ASD-like symptoms in participants with ADHD, unaffected siblings and controls with the caudate nucleus and globus pallidus being of critical importance in predicting the level of ASD-like symptoms in all three groups.     

Testing Sensory and Multisensory Function in Children with Autism Spectrum Disorder

In addition to impairments in social communication and the presence of restricted interests and repetitive behaviors, deficits in sensory processing are now recognized as a core symptom in autism spectrum disorder (ASD). Our ability to perceive and interact with the external world is rooted in sensory processing. For example, listening to a conversation entails processing the auditory cues coming from the speaker (speech content, prosody, syntax) as well as the associated visual information (facial expressions, gestures). Collectively, the “integration” of these multisensory (i.e., combined audiovisual) pieces of information results in better comprehension. Such multisensory integration has been shown to be strongly dependent upon the temporal relationship of the paired stimuli. Thus, stimuli that occur in close temporal proximity are highly likely to result in behavioral and perceptual benefits – gains believed to be reflective of the perceptual system''s judgment of the likelihood that these two stimuli came from the same source. Changes in this temporal integration are expected to strongly alter perceptual processes, and are likely to diminish the ability to accurately perceive and interact with our world. Here, a battery of tasks designed to characterize various aspects of sensory and multisensory temporal processing in children with ASD is described. In addition to its utility in autism, this battery has great potential for characterizing changes in sensory function in other clinical populations, as well as being used to examine changes in these processes across the lifespan.     

Auditory Processing in High-Functioning Adolescents with Autism Spectrum Disorder

Autism Spectrum Disorder (ASD) is a pervasive developmental disorder including abnormalities in perceptual processing. We measure perception in a battery of tests across speech (filtering, phoneme categorization, multisensory integration) and music (pitch memory, meter categorization, harmonic priming). We found that compared to controls, the ASD group showed poorer filtering, less audio-visual integration, less specialization for native phonemic and metrical categories, and a higher instance of absolute pitch. No group differences were found in harmonic priming. Our results are discussed in a developmental framework where culture-specific knowledge acquired early compared to late in development is most impaired, perhaps because of early-accelerated brain growth in ASD. These results suggest that early auditory remediation is needed for good communication and social functioning.     

Stronger Neural Modulation by Visual Motion Intensity in Autism Spectrum Disorders

Theories of autism spectrum disorders (ASD) have focused on altered perceptual integration of sensory features as a possible core deficit. Yet, there is little understanding of the neuronal processing of elementary sensory features in ASD. For typically developed individuals, we previously established a direct link between frequency-specific neural activity and the intensity of a specific sensory feature: Gamma-band activity in the visual cortex increased approximately linearly with the strength of visual motion. Using magnetoencephalography (MEG), we investigated whether in individuals with ASD neural activity reflect the coherence, and thus intensity, of visual motion in a similar fashion. Thirteen adult participants with ASD and 14 control participants performed a motion direction discrimination task with increasing levels of motion coherence. A polynomial regression analysis revealed that gamma-band power increased significantly stronger with motion coherence in ASD compared to controls, suggesting excessive visual activation with increasing stimulus intensity originating from motion-responsive visual areas V3, V6 and hMT/V5. Enhanced neural responses with increasing stimulus intensity suggest an enhanced response gain in ASD. Response gain is controlled by excitatory-inhibitory interactions, which also drive high-frequency oscillations in the gamma-band. Thus, our data suggest that a disturbed excitatory-inhibitory balance underlies enhanced neural responses to coherent motion in ASD.     

Effects of Multimodal Load on Spatial Monitoring as Revealed by ERPs

While the role of selective attention in filtering out irrelevant information has been extensively studied, its characteristics and neural underpinnings when multiple environmental stimuli have to be processed in parallel are much less known. Building upon a dual-task paradigm that induced spatial awareness deficits for contralesional hemispace in right hemisphere-damaged patients, we investigated the electrophysiological correlates of multimodal load during spatial monitoring in healthy participants. The position of appearance of briefly presented, lateralized targets had to be reported either in isolation (single task) or together with a concurrent task, visual or auditory, which recruited additional attentional resources (dual-task). This top-down manipulation of attentional load, without any change of the sensory stimulation, modulated the amplitude of the first positive ERP response (P1) and shifted its neural generators, with a suppression of the signal in the early visual areas during both visual and auditory dual tasks. Furthermore, later N2 contralateral components elicited by left targets were particularly influenced by the concurrent visual task and were related to increased activation of the supramarginal gyrus. These results suggest that the right hemisphere is particularly affected by load manipulations, and confirm its crucial role in subtending automatic orienting of spatial attention and in monitoring both hemispaces.     

Event-related potential measures of attention in moderately depressed subjects

Prior research has inferred attentional changes related to depression from evidence concerning other cognitive processes. The present experiment investigated attentional changes related to depression in a more direct manner. Subjects were 32 young adults attending college. Depression was measured by self-report measures. In an auditory selective attention task similar to that of Hansen and Hillyard (1980), auditory event-related potentials(ERPs) were recorded from central (Cz) and frontal (Fz) scalp locations. Evidence for selective attention was manifest as the difference wave (Nd), which showed larger mean and peak amplitudes for the less difficult of two attention conditions. Nd was also shown to have an earlier peak latency at the Cz scalp location. However, there was no significant difference between the Depressed and Control groups as measured by the Nd wave. Significant differences were found between groups for the amplitude of the N1 peak of the ERP at the Fz scalp location. This suggests that the Depressed group differed in arousal level or sensory sensitivity from the Control group.     

The attentional window modulates capture by audiovisual events

Visual search is markedly improved when a target color change is synchronized with a spatially non-informative auditory signal. This "pip and pop" effect is an automatic process as even a distractor captures attention when accompanied by a tone. Previous studies investigating visual attention have indicated that automatic capture is susceptible to the size of the attentional window. The present study investigated whether the pip and pop effect is modulated by the extent to which participants divide their attention across the visual field We show that participants were better in detecting a synchronized audiovisual event when they divided their attention across the visual field relative to a condition in which they focused their attention. We argue that audiovisual capture is reduced under focused conditions relative to distributed settings.     

人际关系影响竞争情境下结果评价的ERP证据

为了研究竞争情境下人际关系影响结果评价的认知神经机制,记录16名健康成人进行扑克大小游戏时的脑电波,分析被试与朋友以及与陌生人竞争时输赢反馈的ERP.在竞争情境下,人际关系对与结果评价有关的反馈负波（FRN）和P300两个成分都有影响.和朋友竞争的FRN波幅比和陌生人竞争的FRN波幅大,表明FRN除了反映对结果输赢的快速评价,还可能反映了由竞争所引起的认知冲突,和朋友竞争引发更大的认知冲突.和陌生人竞争的P300波幅比和朋友竞争的P300波幅大,表明P300可能反映了一种晚期和社会性注意分配以及动机/情感有关的自上而下的结果评价过程.