Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.     

Atypical Balance between Occipital and Fronto-Parietal Activation for Visual Shape Extraction in Dyslexia

Reading requires the extraction of letter shapes from a complex background of text, and an impairment in visual shape extraction would cause difficulty in reading. To investigate the neural mechanisms of visual shape extraction in dyslexia, we used functional magnetic resonance imaging (fMRI) to examine brain activation while adults with or without dyslexia responded to the change of an arrow’s direction in a complex, relative to a simple, visual background. In comparison to adults with typical reading ability, adults with dyslexia exhibited opposite patterns of atypical activation: decreased activation in occipital visual areas associated with visual perception, and increased activation in frontal and parietal regions associated with visual attention. These findings indicate that dyslexia involves atypical brain organization for fundamental processes of visual shape extraction even when reading is not involved. Overengagement in higher-order association cortices, required to compensate for underengagment in lower-order visual cortices, may result in competition for top-down attentional resources helpful for fluent reading.     

Neural systems for preparatory control of imitation

Humans have an automatic tendency to imitate others. Previous studies on how we control these tendencies have focused on reactive mechanisms, where inhibition of imitation is implemented after seeing an action. This work suggests that reactive control of imitation draws on at least partially specialized mechanisms. Here, we examine preparatory imitation control, where advance information allows control processes to be employed before an action is observed. Drawing on dual route models from the spatial compatibility literature, we compare control processes using biological and non-biological stimuli to determine whether preparatory imitation control recruits specialized neural systems that are similar to those observed in reactive imitation control. Results indicate that preparatory control involves anterior prefrontal, dorsolateral prefrontal, posterior parietal and early visual cortices regardless of whether automatic responses are evoked by biological (imitative) or non-biological stimuli. These results indicate both that preparatory control of imitation uses general mechanisms, and that preparatory control of imitation draws on different neural systems from reactive imitation control. Based on the regions involved, we hypothesize that preparatory control is implemented through top-down attentional biasing of visual processing.     

Temporal-Spatial Neural Activation Patterns Linked to Perceptual Encoding of Emotional Salience

It is well known that we continuously filter incoming sensory information, selectively allocating attention to what is important while suppressing distracting or irrelevant information. Yet questions remain about spatiotemporal patterns of neural processes underlying attentional biases toward emotionally significant aspects of the world. One index of affectively biased attention is an emotional variant of an attentional blink (AB) paradigm, which reveals enhanced perceptual encoding for emotionally salient over neutral stimuli under conditions of limited executive attention. The present study took advantage of the high spatial and temporal resolution of magnetoencephalography (MEG) to investigate neural activation related to emotional and neutral targets in an AB task. MEG data were collected while participants performed a rapid stimulus visual presentation task in which two target stimuli were embedded in a stream of distractor words. The first target (T1) was a number and the second (T2) either an emotionally salient or neutral word. Behavioural results replicated previous findings of greater accuracy for emotionally salient than neutral T2 words. MEG source analyses showed that activation in orbitofrontal cortex, characterized by greater power in the theta and alpha bands, and dorsolateral prefrontal activation were associated with successful perceptual encoding of emotionally salient relative to neutral words. These effects were observed between 250 and 550 ms, latencies associated with discrimination of perceived from unperceived stimuli. These data suggest that important nodes of both emotional salience and frontoparietal executive systems are associated with the emotional modulation of the attentional blink.     

Neural correlates of individual performance differences in resolving perceptual conflict

Attentional mechanisms are a crucial prerequisite to organize behavior. Most situations may be characterized by a 'competition' between salient, but irrelevant stimuli and less salient, relevant stimuli. In such situations top-down and bottom-up mechanisms interact with each other. In the present fMRI study, we examined how interindividual differences in resolving situations of perceptual conflict are reflected in brain networks mediating attentional selection. Doing so, we employed a change detection task in which subjects had to detect luminance changes in the presence and absence of competing distractors. The results show that good performers presented increased activation in the orbitofrontal cortex (BA 11), anterior cingulate (BA 25), inferior parietal lobule (BA 40) and visual areas V2 and V3 but decreased activation in BA 39. This suggests that areas mediating top-down attentional control are stronger activated in this group. Increased activity in visual areas reflects distinct neuronal enhancement relating to selective attentional mechanisms in order to solve the perceptual conflict. Opposed to good performers, brain areas activated by poor performers comprised the left inferior parietal lobule (BA 39) and fronto-parietal and visual regions were continuously deactivated, suggesting that poor performers perceive stronger conflict than good performers. Moreover, the suppression of neural activation in visual areas might indicate a strategy of poor performers to inhibit the processing of the irrelevant non-target feature. These results indicate that high sensitivity in perceptual areas and increased attentional control led to less conflict in stimulus processing and consequently to higher performance in competitive attentional selection.     

Interactions between the neural correlates of dispositional internally directed thought and visual imagery

Cognition is not always directed to the events in the here and now and we often self-generate thoughts and images in imagination. Important aspects of these self-generated experiences are associated with various dispositional traits. In this study, we explored whether these psychological associations relate to a common underlying neurocognitive mechanism. We acquired resting state functional magnetic resonance imaging data from a large cohort of participants and asked them to retrospectively report their experience during the scan. Participants also completed questionnaires reflecting a range of dispositional traits. We found thoughts emphasizing visual imagery at rest were associated with dispositional tendency towards internally directed attention (self-consciousness and attentional problems) and linked to a stronger correlation between a posterior parietal network and a lateral fronto-temporal network. Furthermore, decoupling between the brainstem and a lateral visual network was associated with dispositional internally directed attention. Critically, these brain–cognition associations were related: the correlation between parietal–frontal regions and reports of visual imagery was stronger for individuals with increased connectivity between brainstem and visual cortex. Our results highlight neural mechanisms linked to the dispositional basis for patterns of self-generated thought, and suggest that accounting for dispositional traits is important when exploring the neural substrates of self-generated experience (and vice versa).This article is part of the theme issue ‘Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation’.     

Applying an Attentional Set to Perceived and Remembered Features

Previous research has examined our ability to attend selectively to particular features of perceptual objects, as well as our ability to switch from attending to one type of feature to another. This is usually done in the context of anticipatory attentional-set control, comparing the neural mechanisms involved as participants prepare to attend to the same stimulus feature as on the previous trial (“task-stay” trials) with those required as participants prepare to attend to a different stimulus feature to that previously attended (“task-switch” trials). We wanted to establish how participants maintain or switch attentional set retrospectively, as they attend to features of objects held in visual short-term memory (VSTM). We found that switching, relative to maintaining attentional set retrospectively, was associated with a performance cost, which can be reduced over time. This control process was mirrored by a large parietal and frontal amplitude difference in the event-related brain potentials (ERPs) and significant differences in global field power (GFP) between switch and stay trials. However, when taking into account the switch/stay GFP differences, thereby controlling for this difference in amplitude, we could not distinguish these trial types topographically. By contrast, we found clear topographic differences between preparing an anticipatory feature-based attentional set versus applying it retrospectively within VSTM. These complementary topographical and amplitude analyses suggested that anticipatory and retrospective set control recruited a qualitatively different configuration of underlying neural generators. In contrast, switch/stay differences were largely quantitative, with them differing primarily in terms of amplitude rather than topography.     

Subjective visual perception: from local processing to emergent phenomena of brain activity

The combination of electrophysiological recordings with ambiguous visual stimulation made possible the detection of neurons that represent the content of subjective visual perception and perceptual suppression in multiple cortical and subcortical brain regions. These neuronal populations, commonly referred to as the neural correlates of consciousness, are more likely to be found in the temporal and prefrontal cortices as well as the pulvinar, indicating that the content of perceptual awareness is represented with higher fidelity in higher-order association areas of the cortical and thalamic hierarchy, reflecting the outcome of competitive interactions between conflicting sensory information resolved in earlier stages. However, despite the significant insights into conscious perception gained through monitoring the activities of single neurons and small, local populations, the immense functional complexity of the brain arising from correlations in the activity of its constituent parts suggests that local, microscopic activity could only partially reveal the mechanisms involved in perceptual awareness. Rather, the dynamics of functional connectivity patterns on a mesoscopic and macroscopic level could be critical for conscious perception. Understanding these emergent spatio-temporal patterns could be informative not only for the stability of subjective perception but also for spontaneous perceptual transitions suggested to depend either on the dynamics of antagonistic ensembles or on global intrinsic activity fluctuations that may act upon explicit neural representations of sensory stimuli and induce perceptual reorganization. Here, we review the most recent results from local activity recordings and discuss the potential role of effective, correlated interactions during perceptual awareness.     

Breakdown of functional connectivity in frontoparietal networks underlies behavioral deficits in spatial neglect

Spatial neglect is a syndrome following stroke manifesting attentional deficits in perceiving and responding to stimuli in the contralesional field. We examined brain network integrity in patients with neglect by measuring coherent fluctuations of fMRI signals (functional connectivity). Connectivity in two largely separate attention networks located in dorsal and ventral frontoparietal areas was assessed at both acute and chronic stages of recovery. Connectivity in the ventral network, part of which directly lesioned, was diffusely disrupted and showed no recovery. In the structurally intact dorsal network, interhemispheric connectivity in posterior parietal cortex was acutely disrupted but fully recovered. This acute disruption, and disrupted connectivity in specific pathways in the ventral network, strongly correlated with impaired attentional processing across subjects. Lastly, disconnection of the white matter tracts connecting frontal and parietal cortices was associated with more severe neglect and more disrupted functional connectivity. These findings support a network view in understanding neglect.     

Attention driven phantom vision: measuring the sensory strength of attentional templates and their relation to visual mental imagery and aphantasia

When we search for an object in an array or anticipate attending to a future object, we create an ‘attentional template'' of the object. The definitions of attentional templates and visual imagery share many similarities as well as many of the same neural characteristics. However, the phenomenology of these attentional templates and their neural similarities to visual imagery and perception are rarely, if ever discussed. Here, we investigate the relationship between these two forms of non-retinal phantom vision through the use of the binocular rivalry technique, which allows us to measure the sensory strength of attentional templates in the absence of concurrent perceptual stimuli. We find that attentional templates correlate with both feature-based attention and visual imagery. Attentional templates, like imagery, were significantly disrupted by the presence of irrelevant visual stimuli, while feature-based attention was not. We also found that a special population who lack the ability to visualize (aphantasia), showed evidence of feature-based attention when measured using the binocular rivalry paradigm, but not attentional templates. Taken together, these data suggest functional similarities between attentional templates and visual imagery, advancing the theory of visual imagery as a general simulation tool used across cognition.This article is part of the theme issue ‘Offline perception: voluntary and spontaneous perceptual experiences without matching external stimulation’.     

Decoding the different states of visual attention using functional and effective connectivity features in fMRI data

The present paper concentrates on the impact of visual attention task on structure of the brain functional and effective connectivity networks using coherence and Granger causality methods. Since most studies used correlation method and resting-state functional connectivity, the task-based approach was selected for this experiment to boost our knowledge of spatial and feature-based attention. In the present study, the whole brain was divided into 82 sub-regions based on Brodmann areas. The coherence and Granger causality were applied to construct functional and effective connectivity matrices. These matrices were converted into graphs using a threshold, and the graph theory measures were calculated from it including degree and characteristic path length. Visual attention was found to reveal more information during the spatial-based task. The degree was higher while performing a spatial-based task, whereas characteristic path length was lower in the spatial-based task in both functional and effective connectivity. Primary and secondary visual cortex (17 and 18 Brodmann areas) were highly connected to parietal and prefrontal cortex while doing visual attention task. Whole brain connectivity was also calculated in both functional and effective connectivity. Our results reveal that Brodmann areas of 17, 18, 19, 46, 3 and 4 had a significant role proving that somatosensory, parietal and prefrontal regions along with visual cortex were highly connected to other parts of the cortex during the visual attention task. Characteristic path length results indicated an increase in functional connectivity and more functional integration in spatial-based attention compared with feature-based attention. The results of this work can provide useful information about the mechanism of visual attention at the network level.     

Is Payoff Necessarily Weighted by Probability When Making a Risky Choice? Evidence from Functional Connectivity Analysis

How people make decisions under risk remains an as-yet-unresolved but fundamental question. Mainstream theories about risky decision making assume that the core processes involved in reaching a risky decision include weighting each payoff or reward magnitude by its probability and then summing the outcomes. However, recently developed theories question whether payoffs are necessarily weighted by probability when making a risky choice. Using functional connectivity analysis, we aimed to provide neural evidence to answer whether this key assumption of computing expectations holds when making a risky choice. We contrasted a trade-off instruction choice that required participants to integrate probability and payoff information with a preferential choice that did not. Based on the functional connectivity patterns between regions in which activity was detected during both of the decision-making tasks, we classified the regions into two networks. One network includes primarily the left and right lateral prefrontal cortices and posterior parietal cortices, which were found to be related to probability in previous reports, and the other network is composed of the bilateral basal ganglia, which have been implicated in payoff. We also found that connectivity between the payoff network and some regions in the probability network (including the left lateral prefrontal cortices and bilateral inferior parietal lobes) were stronger during the trade-off instruction choice task than during the preferential choice task. This indicates that the functional integration between the probability and payoff networks during preferential choice was not as strong as the integration during trade-off instruction choice. Our results provide neural evidence that the weighting process uniformly predicted by the mainstream theory is unnecessary during preferential choice. Thus, our functional integration findings can provide a new direction for the investigation of the principles of risky decision making.     

Self-Orientation Modulates the Neural Correlates of Global and Local Processing

Differences in self-orientation (or “self-construal”) may affect how the visual environment is attended, but the neural and cultural mechanisms that drive this remain unclear. Behavioral studies have demonstrated that people from Western backgrounds with predominant individualistic values are perceptually biased towards local-level information; whereas people from non-Western backgrounds that support collectivist values are preferentially focused on contextual and global-level information. In this study, we compared two groups differing in predominant individualistic (N = 15) vs collectivistic (N = 15) self-orientation. Participants completed a global/local perceptual conflict task whilst undergoing functional Magnetic Resonance Imaging (fMRI) scanning. When participants high in individualistic values attended to the global level (ignoring the local level), greater activity was observed in the frontoparietal and cingulo-opercular networks that underpin attentional control, compared to the match (congruent) baseline. Participants high in collectivistic values activated similar attentional control networks o only when directly compared with global processing. This suggests that global interference was stronger than local interference in the conflict task in the collectivistic group. Both groups showed increased activity in dorsolateral prefrontal regions involved in resolving perceptual conflict during heightened distractor interference. The findings suggest that self-orientation may play an important role in driving attention networks to facilitate interaction with the visual environment.     

Neocortical connectivity during episodic memory formation

During the formation of new episodic memories, a rich array of perceptual information is bound together for long-term storage. However, the brain mechanisms by which sensory representations (such as colors, objects, or individuals) are selected for episodic encoding are currently unknown. We describe a functional magnetic resonance imaging experiment in which participants encoded the association between two classes of visual stimuli that elicit selective responses in the extrastriate visual cortex (faces and houses). Using connectivity analyses, we show that correlation in the hemodynamic signal between face- and place-sensitive voxels and the left dorsolateral prefrontal cortex is a reliable predictor of successful face-house binding. These data support the view that during episodic encoding, "top-down" control signals originating in the prefrontal cortex help determine which perceptual information is fated to be bound into the new episodic memory trace.     

Converging levels of analysis in the cognitive neuroscience of visual attention.

Experiments using behavioural, lesion, functional imaging and single neuron methods are considered in the context of a neuropsychological model of visual attention. According to this model, inputs compete for representation in multiple visually responsive brain systems, sensory and motor, cortical and subcortical. Competition is biased by advance priming of neurons responsive to current behavioural targets. Across systems competition is integrated such that the same, selected object tends to become dominant throughout. The behavioural studies reviewed concern divided attention within and between modalities. They implicate within-modality competition as one main restriction on concurrent stimulus identification. In contrast to the conventional association of lateral attentional focus with parietal lobe function, the lesion studies show attentional bias to be a widespread consequence of unilateral cortical damage. Although the clinical syndrome of unilateral neglect may indeed be associated with parietal lesions, this probably reflects an assortment of further deficits accompanying a simple attentional imbalance. The functional imaging studies show joint involvement of lateral prefrontal and occipital cortex in lateral attentional focus and competition. The single unit studies suggest how competition in several regions of extrastriate cortex is biased by advance priming of neurons responsive to current behavioural targets. Together, the concepts of competition, priming and integration allow a unified theoretical approach to findings from behavioural to single neuron levels.     

Electrical Neuroimaging Reveals Timing of Attentional Control Activity in Human Brain

Voluntarily shifting attention to a location of the visual field improves the perception of events that occur there. Regions of frontal cortex are thought to provide the top-down control signal that initiates a shift of attention, but because of the temporal limitations of functional brain imaging, the timing and sequence of attentional-control operations remain unknown. We used a new analytical technique (beamformer spatial filtering) to reconstruct the anatomical sources of low-frequency brain waves in humans associated with attentional control across time. Following a signal to shift attention, control activity was seen in parietal cortex 100–200 ms before activity was seen in frontal cortex. Parietal cortex was then reactivated prior to anticipatory biasing of activity in occipital cortex. The magnitudes of early parietal activations were strongly predictive of the degree of attentional improvement in perceptual performance. These results show that parietal cortex, not frontal cortex, provides the initial signals to shift attention and indicate that top-down attentional control is not purely top down.     

Task-Related Changes in Functional Properties of the Human Brain Network Underlying Attentional Control

Previous studies have demonstrated task-related changes in brain activation and inter-regional connectivity but the temporal dynamics of functional properties of the brain during task execution is still unclear. In the present study, we investigated task-related changes in functional properties of the human brain network by applying graph-theoretical analysis to magnetoencephalography (MEG). Subjects performed a cue-target attention task in which a visual cue informed them of the direction of focus for incoming auditory or tactile target stimuli, but not the sensory modality. We analyzed the MEG signal in the cue-target interval to examine network properties during attentional control. Cluster-based non-parametric permutation tests with the Monte-Carlo method showed that in the cue-target interval, beta activity was desynchronized in the sensori-motor region including premotor and posterior parietal regions in the hemisphere contralateral to the attended side. Graph-theoretical analysis revealed that, in beta frequency, global hubs were found around the sensori-motor and prefrontal regions, and functional segregation over the entire network was decreased during attentional control compared to the baseline. Thus, network measures revealed task-related temporal changes in functional properties of the human brain network, leading to the understanding of how the brain dynamically responds to task execution as a network.     

How do we select perceptions and actions? Human brain imaging studies.

The selective nature of human perception and action implies a modulatory interaction between sensorimotor processes and attentional processes. This paper explores the use of functional imaging in humans to explore the mechanisms of perceptual selection and the fate of irrelevant stimuli that are not selected. Experiments with positron emission tomography show that two qualitatively different patterns of modulation of cerebral blood flow can be observed in experiments where non-spatial visual attention and auditory attention are manipulated. These patterns of modulation of cerebral blood flow modulation can be described as gain control and bias signal mechanisms. In visual and auditory cortex, the dominant change in cerebral blood flow associated with attention to either modality is related to a bias signal. The relation of these patterns of modulation to attentional effects that have been observed in single neurons is discussed. The existence of mechanisms for selective perception raises the more general question of whether irrelevant ignored stimuli are nevertheless perceived. Lavie''s theory of attention proposes that the degree to which ignored stimuli are processed varies depending on the perceptual load of the current task. Evidence from behavioural and functional magnetic resonance imaging studies of ignored visual motion processing is presented in support of this proposal.     

Bottlenecks of Motion Processing during a Visual Glance: The Leaky Flask Model

Where do the bottlenecks for information and attention lie when our visual system processes incoming stimuli? The human visual system encodes the incoming stimulus and transfers its contents into three major memory systems with increasing time scales, viz., sensory (or iconic) memory, visual short-term memory (VSTM), and long-term memory (LTM). It is commonly believed that the major bottleneck of information processing resides in VSTM. In contrast to this view, we show major bottlenecks for motion processing prior to VSTM. In the first experiment, we examined bottlenecks at the stimulus encoding stage through a partial-report technique by delivering the cue immediately at the end of the stimulus presentation. In the second experiment, we varied the cue delay to investigate sensory memory and VSTM. Performance decayed exponentially as a function of cue delay and we used the time-constant of the exponential-decay to demarcate sensory memory from VSTM. We then decomposed performance in terms of quality and quantity measures to analyze bottlenecks along these dimensions. In terms of the quality of information, two thirds to three quarters of the motion-processing bottleneck occurs in stimulus encoding rather than memory stages. In terms of the quantity of information, the motion-processing bottleneck is distributed, with the stimulus-encoding stage accounting for one third of the bottleneck. The bottleneck for the stimulus-encoding stage is dominated by the selection compared to the filtering function of attention. We also found that the filtering function of attention is operating mainly at the sensory memory stage in a specific manner, i.e., influencing only quantity and sparing quality. These results provide a novel and more complete understanding of information processing and storage bottlenecks for motion processing.     

The Frontal Eye Fields Limit the Capacity of Visual Short-Term Memory in Rhesus Monkeys

The frontal eye fields (FEF) in rhesus monkeys have been implicated in visual short-term memory (VSTM) as well as control of visual attention. Here we examined the importance of the area in the VSTM capacity and the relationship between VSTM and attention, using the chemical inactivation technique and multi-target saccade tasks with or without the need of target-location memory. During FEF inactivation, serial saccades to targets defined by color contrast were unaffected, but saccades relying on short-term memory were impaired when the target count was at the capacity limit of VSTM. The memory impairment was specific to the FEF-coded retinotopic locations, and subject to competition among targets distributed across visual fields. These results together suggest that the FEF plays a crucial role during the entry of information into VSTM, by enabling attention deployment on targets to be remembered. In this view, the memory capacity results from the limited availability of attentional resources provided by FEF: The FEF can concurrently maintain only a limited number of activations to register the targets into memory. When lesions render part of the area unavailable for activation, the number would decrease, further reducing the capacity of VSTM.