Interactive coding of visual spatial frequency and auditory amplitude-modulation rate

Spatial frequency is a fundamental visual feature coded in primary visual cortex, relevant for perceiving textures, objects, hierarchical structures, and scenes, as well as for directing attention and eye movements. Temporal amplitude-modulation (AM) rate is a fundamental auditory feature coded in primary auditory cortex, relevant for perceiving auditory objects, scenes, and speech. Spatial frequency and temporal AM rate are thus fundamental building blocks of visual and auditory perception. Recent results suggest that crossmodal interactions are commonplace across the primary sensory cortices and that some of the underlying neural associations develop through consistent multisensory experience such as audio-visually perceiving speech, gender, and objects. We demonstrate that people consistently and absolutely (rather than relatively) match specific auditory AM rates to specific visual spatial frequencies. We further demonstrate that this crossmodal mapping allows amplitude-modulated sounds to guide attention to and modulate awareness of specific visual spatial frequencies. Additional results show that the crossmodal association is approximately linear, based on physical spatial frequency, and generalizes to tactile pulses, suggesting that the association develops through multisensory experience during manual exploration of surfaces.     

Stimulus coding rules for perceptual learning

Perceptual learning of visual features occurs when multiple stimuli are presented in a fixed sequence (temporal patterning), but not when they are presented in random order (roving). This points to the need for proper stimulus coding in order for learning of multiple stimuli to occur. We examined the stimulus coding rules for learning with multiple stimuli. Our results demonstrate that: (1) stimulus rhythm is necessary for temporal patterning to take effect during practice; (2) learning consolidation is subject to disruption by roving up to 4 h after each practice session; (3) importantly, after completion of temporal-patterned learning, performance is undisrupted by extended roving training; (4) roving is ineffective if each stimulus is presented for five or more consecutive trials; and (5) roving is also ineffective if each stimulus has a distinct identity. We propose that for multi-stimulus learning to occur, the brain needs to conceptually “tag” each stimulus, in order to switch attention to the appropriate perceptual template. Stimulus temporal patterning assists in tagging stimuli and switching attention through its rhythmic stimulus sequence.     

Attentional modulation of firing rate and synchrony in a model cortical network

The response of a neuron in the visual cortex to stimuli of different contrast placed in its receptive field is commonly characterized using the contrast response curve. When attention is directed into the receptive field of a V4 neuron, its contrast response curve is shifted to lower contrast values (Reynolds et al., 2000). The neuron will thus be able to respond to weaker stimuli than it responded to without attention. Attention also increases the coherence between neurons responding to the same stimulus (Fries et al., 2001). We studied how the firing rate and synchrony of a densely interconnected cortical network varied with contrast and how they were modulated by attention. The changes in contrast and attention were modeled as changes in driving current to the network neurons. We found that an increased driving current to the excitatory neurons increased the overall firing rate of the network, whereas variation of the driving current to inhibitory neurons modulated the synchrony of the network. We explain the synchrony modulation in terms of a locking phenomenon during which the ratio of excitatory to inhibitory firing rates is approximately constant for a range of driving current values. We explored the hypothesis that contrast is represented primarily as a drive to the excitatory neurons, whereas attention corresponds to a reduction in driving current to the inhibitory neurons. Using this hypothesis, the model reproduces the following experimental observations: (1) the firing rate of the excitatory neurons increases with contrast; (2) for high contrast stimuli, the firing rate saturates and the network synchronizes; (3) attention shifts the contrast response curve to lower contrast values; (4) attention leads to stronger synchronization that starts at a lower value of the contrast compared with the attend-away condition. In addition, it predicts that attention increases the delay between the inhibitory and excitatory synchronous volleys produced by the network, allowing the stimulus to recruit more downstream neurons. Action Editor: David Golomb     

The Effect of Speed of Processing Training on Microsaccade Amplitude

Older adults experience cognitive deficits that can lead to driving errors and a loss of mobility. Fortunately, some of these deficits can be ameliorated with targeted interventions which improve the speed and accuracy of simultaneous attention to a central and a peripheral stimulus called Speed of Processing training. To date, the mechanisms behind this effective training are unknown. We hypothesized that one potential mechanism underlying this training is a change in distribution of eye movements of different amplitudes. Microsaccades are small amplitude eye movements made when fixating on a stimulus, and are thought to counteract the “visual fading” that occurs when static stimuli are presented. Due to retinal anatomy, larger microsaccadic eye movements are needed to move a peripheral stimulus between receptive fields and counteract visual fading. Alternatively, larger microsaccades may decrease performance due to neural suppression. Because larger microsaccades could aid or hinder peripheral vision, we examine the distribution of microsaccades during stimulus presentation. Our results indicate that there is no statistically significant change in the proportion of large amplitude microsaccades during a Useful Field of View-like task after training in a small sample of older adults. Speed of Processing training does not appear to result in changes in microsaccade amplitude, suggesting that the mechanism underlying Speed of Processing training is unlikely to rely on microsaccades.     

Implicit attentional selection of bound visual features

Traditionally, research on visual attention has been focused on the processes involved in conscious, explicit selection of task-relevant sensory input. Recently, however, it has been shown that attending to a specific feature of an object automatically increases neural sensitivity to this feature throughout the visual field. Here we show that directing attention to a specific color of an object results in attentional modulation of the processing of task-irrelevant and not consciously perceived motion signals that are spatiotemporally associated with this color throughout the visual field. Such implicit cross-feature spreading of attention takes place according to the veridical physical associations between the color and motion signals, even under special circumstances when they are perceptually misbound. These results imply that the units of implicit attentional selection are spatiotemporally colocalized feature clusters that are automatically bound throughout the visual field.     

Transient attention enhances perceptual performance and FMRI response in human visual cortex

When a visual stimulus suddenly appears, it captures attention, producing a transient improvement of performance on basic visual tasks. We investigate the effect of transient attention on stimulus representations in early visual areas using rapid event-related fMRI. Participants discriminated the orientation of one of two gratings preceded or followed by a nonpredictive peripheral cue. Compared to control conditions, precueing the target location improved performance and produced a larger fMRI response in corresponding retinotopic areas. This enhancement progressively increased from striate to extrastriate areas. Control conditions indicated that the enhanced fMRI response was not due to sensory summation of cue and target signals. Thus, an uninformative precue increases both perceptual performance and the concomitant stimulus-evoked activity in early visual areas. These results provide evidence regarding the retinotopically specific neural correlate for the effects of transient attention on early vision.     

Top-down anticipatory control in prefrontal cortex

Summary The prefrontal cortex has been implicated in a wide variety of executive functions, many involving some form of anticipatory attention. Anticipatory attention involves the pre-selection of specific sensory circuits to allow fast and efficient stimulus processing and a subsequently fast and accurate response. It is generally agreed that the prefrontal cortex plays a critical role in anticipatory attention by exerting a facilitatory “top-down” bias on sensory pathways. In this paper we review recent results indicating that synchronized activity in prefrontal cortex, during anticipation of visual stimulus, can predict features of early visual stimulus processing and behavioral response. Although the mechanisms involved in anticipatory attention are still largely unknown, we argue that the synchronized oscillation in prefrontal cortex is a plausible candidate during sustained visual anticipation. We further propose a learning hypothesis that explains how this top-down anticipatory control in prefrontal cortex is learned based on accumulated prior experience by adopting a Temporal Difference learning algorithm.     

Speeding up reaction time with invisible stimuli

Normal subjects react more quickly to a pair of visual stimuli than to a stimulus alone. This phenomenon is known as the redundant signal effect (RSE) and represents an example of divided visual attention in which signal processing is carried out in parallel to the advantage of response speed. A most interesting aspect of this phenomenon is that it can occur when one stimulus in a pair cannot be consciously detected because of hemianopia or unilateral extinction resulting from brain damage. Here, we report that a similar dissociation between visual awareness and visually guided behavior is present in normal subjects who show an RSE even when the luminance of one of a pair of stimuli is below detection threshold. The observed RSE cannot be attributed to probability summation because it violates Miller's race inequality and is likely to be related to neural summation between supra- and subthreshold stimuli. Given that a similar implicit RSE is present in hemispherectomy patients, we hypothesize that the site of this summation might be the superior colliculus (SC).     

Attention increases sensitivity of V4 neurons

When attention is directed to a location in the visual field, sensitivity to stimuli at that location is increased. At the neuronal level, this could arise either through a multiplicative increase in firing rate or through an increase in the effective strength of the stimulus. To test conflicting predictions of these alternative models, we recorded responses of V4 neurons to stimuli across a range of luminance contrasts and measured the change in response when monkeys attended to them in order to discriminate a target stimulus from nontargets. Attention caused greater increases in response at low contrast than at high contrast, consistent with an increase in effective stimulus strength. On average, attention increased the effective contrast of the attended stimulus by a factor of 1.51, an increase of 51% of its physical contrast.     

The role of attention in visual processing

Attention to a visual stimulus typically increases the responses of cortical neurons to that stimulus. Because many studies have shown a close relationship between the performance of individual neurons and behavioural performance of animal subjects, it is important to consider how attention affects this relationship. Measurements of behavioural and neuronal performance taken from rhesus monkeys while they performed a motion detection task with two attentional states show that attention alters the relationship between behaviour and neuronal response. Notably, attention affects the relationship differently in different cortical visual areas. This indicates that a close relationship between neuronal and behavioural performance on a given task persists over changes in attentional state only within limited regions of visual cortex.     

Attentional Tuning Resets after Failures of Perceptual Awareness

Key to successfully negotiating our environment is our ability to adapt to current settings based on recent experiences and behaviour. Response conflict paradigms (e.g., the Stroop task) have provided evidence for increases in executive control after errors, leading to slowed responses that are more likely to be correct, and less susceptible to response congruency effects. Here we investigate whether failures of perceptual awareness, rather than failures at decisional or response stages of information processing, lead to similar adjustments in visual attention. We employed an attentional blink task in which subjects often fail to consciously register the second of two targets embedded in a rapid serial visual presentation stream of distractors, and examined how target errors influence performance on subsequent trials. Performance was inferior after Target 2 errors and these inter-trial effects were independent of the temporal lag between the targets and were not due to more global changes in attention across runs of trials. These results shed light on the nature of attentional calibration in response to failures of perceptual consciousness.     

The temporal dynamics of reading: a PET study.

The temporal dynamics of evoked brain responses are normally characterized using electrophysiological techniques but the positron emission tomography study presented here revealed a temporal aspect of reading by correlating the duration a word remained in the visual field with evoked haemodynamic response. Three distinct types of effects were observed: in visual processing areas, there were linear increases in activity with duration suggesting that visual processing endures throughout the time the stimulus remains in the visual field. In right hemisphere areas, there were monotonic decreases in activity with increased duration which we relate to decreased attention for longer stimulus durations. In left hemisphere word processing areas there were inverted U-shaped dependencies between activity and word duration indicating that, after 400-600 ms, activity in word processing areas is progressively reduced if the word remains in the visual field. We conclude that these inverted U effects in left hemisphere language areas reflect the temporal dynamics of visual word processing and we highlight the implication of these effects for the design of activation studies involving reading.     

How Long Depends on How Fast—Perceived Flicker Dilates Subjective Duration

How do humans perceive the passage of time and the duration of events without a dedicated sensory system for timing? Previous studies have demonstrated that when a stimulus changes over time, its duration is subjectively dilated, indicating that duration judgments are based on the number of changes within an interval. In this study, we tested predictions derived from three different accounts describing the relation between a changing stimulus and its subjective duration as either based on (1) the objective rate of changes of the stimulus, (2) the perceived saliency of the changes, or (3) the neural energy expended in processing the stimulus. We used visual stimuli flickering at different frequencies (4–166 Hz) to study how the number of changes affects subjective duration. To this end, we assessed the subjective duration of these stimuli and measured participants'' behavioral flicker fusion threshold (the highest frequency perceived as flicker), as well as their threshold for a frequency-specific neural response to the flicker using EEG. We found that only consciously perceived flicker dilated perceived duration, such that a 2 s long stimulus flickering at 4 Hz was perceived as lasting as long as a 2.7 s steady stimulus. This effect was most pronounced at the slowest flicker frequencies, at which participants reported the most consistent flicker perception. Flicker frequencies higher than the flicker fusion threshold did not affect perceived duration at all, even if they evoked a significant frequency-specific neural response. In sum, our findings indicate that time perception in the peri-second range is driven by the subjective saliency of the stimulus'' temporal features rather than the objective rate of stimulus changes or the neural response to the changes.     

Lip-Reading Aids Word Recognition Most in Moderate Noise: A Bayesian Explanation Using High-Dimensional Feature Space

Watching a speaker''s facial movements can dramatically enhance our ability to comprehend words, especially in noisy environments. From a general doctrine of combining information from different sensory modalities (the principle of inverse effectiveness), one would expect that the visual signals would be most effective at the highest levels of auditory noise. In contrast, we find, in accord with a recent paper, that visual information improves performance more at intermediate levels of auditory noise than at the highest levels, and we show that a novel visual stimulus containing only temporal information does the same. We present a Bayesian model of optimal cue integration that can explain these conflicts. In this model, words are regarded as points in a multidimensional space and word recognition is a probabilistic inference process. When the dimensionality of the feature space is low, the Bayesian model predicts inverse effectiveness; when the dimensionality is high, the enhancement is maximal at intermediate auditory noise levels. When the auditory and visual stimuli differ slightly in high noise, the model makes a counterintuitive prediction: as sound quality increases, the proportion of reported words corresponding to the visual stimulus should first increase and then decrease. We confirm this prediction in a behavioral experiment. We conclude that auditory-visual speech perception obeys the same notion of optimality previously observed only for simple multisensory stimuli.     

Modelling Visual Change Detection and Identification under Free Viewing Conditions

We examined whether the abilities of observers to perform an analogue of a real-world monitoring task involving detection and identification of changes to items in a visual display could be explained better by models based on signal detection theory (SDT) or high threshold theory (HTT). Our study differed from most previous studies in that observers were allowed to inspect the initial display for 3s, simulating the long inspection times typical of natural viewing, and their eye movements were not constrained. For the majority of observers, combined change detection and identification performance was best modelled by a SDT-based process that assumed that memory resources were distributed across all eight items in our displays. Some observers required a parameter to allow for sometimes making random guesses at the identities of changes they had missed. However, the performance of a small proportion of observers was best explained by a HTT-based model that allowed for lapses of attention.     

The persistences of vision

Human observers continue to experience a visual stimulus for some time after the offset that stimulus. The neural activity evoked by a visual stimulus continues for some time after its offset. The information extracted from a visual stimulus continues to be registered in a visual form of memory ('iconic memory') for some time after its offset. We may thus distinguish three distinct senses in which a visual stimulus may be said to persist after its physical offset: there is phenomenological persistence, neural persistence and informational persistence. Various assumptions have been made about the relation between these forms of visual persistence. The most frequent assumption is that they correspond simply to three different methods for studying a single entity. Detailed consideration of what is known about the properties of these three forms of persistence suggests, however, that this assumption is not correct. It can reasonably be proposed that visible persistence is the phenomenological correlate of neural persistence occurring at various stages of the visual system: photoreceptors, ganglion cells and the stereopsis system. Iconic memory on the other hand, does not correspond to visible persistence, nor to neural persistence in any stage of the visual system. Recent work, in fact, suggests that iconic memory is a property of some relatively late stage in the visual information-processing system, rather than being a peripheral sensory buffer store. This suggestion raises some fundamental theoretical issues concerning the psychology of visual perception, issues with which cognitive psychology has yet to come to grips.     

Generalized flash suppression of salient visual targets

A pattern of light striking the retina of an alert observer is normally readily perceived. While a handful of conditions exist in which even salient visual stimuli can be rendered invisible, the mechanisms underlying such suppression remain poorly understood. Here, we describe experiments using a novel stimulation sequence that gives rise to the sudden and reliable subjective disappearance of a wide range of visual patterns. We found that a parafoveal target immediately vanished from perception following the abrupt onset of a surrounding texture. The probability of disappearance was influenced by the ocular configuration of the target and surround, as well as their spatial separation. In addition, suppression was critically dependent upon several hundred milliseconds of stimulus-specific adaptation. These findings demonstrate that the all-or-none disappearance of a salient visual target, which is reminiscent of a high-level selection process, is inextricably linked to topographic stimulus representations, presumably in the early visual cortex.     

A model of subjective report and objective discrimination as categorical decisions in a vast representational space

Subliminal perception studies have shown that one can objectively discriminate a stimulus without subjectively perceiving it. We show how a minimalist framework based on Signal Detection Theory and Bayesian inference can account for this dissociation, by describing subjective and objective tasks with similar decision-theoretic mechanisms. Each of these tasks relies on distinct response classes, and therefore distinct priors and decision boundaries. As a result, they may reach different conclusions. By formalizing, within the same framework, forced-choice discrimination responses, subjective visibility reports and confidence ratings, we show that this decision model suffices to account for several classical characteristics of conscious and unconscious perception. Furthermore, the model provides a set of original predictions on the nonlinear profiles of discrimination performance obtained at various levels of visibility. We successfully test one such prediction in a novel experiment: when varying continuously the degree of perceptual ambiguity between two visual symbols presented at perceptual threshold, identification performance varies quasi-linearly when the stimulus is unseen and in an ‘all-or-none’ manner when it is seen. The present model highlights how conscious and non-conscious decisions may correspond to distinct categorizations of the same stimulus encoded by a high-dimensional neuronal population vector.     

EEG θ rhythm in preterm and full-term infants at the age of five months in endogenous attention

EEG indicators of endogenous attention (EnA) were studied in healthy infants carried to term and extremely preterm infants at a corrected age of five months. The cortical topography of the spectral amplitudes of the EEG θ rhythm was studied during long-term attention of the children to a new visual stimulus (exogenous attention, ExA) and during the retention of anticipatory attention under the conditions of constant appearance and disappearance of a stimulus in the paradigm of visual expectation (EnA). The relationship between reactive changes in the EEG θ rhythm during the retention of EnA and the behavioral parameters of the infant’s ability to retain this type of attention was also assessed. In five-month-old infants, the retention of EnA, in contrast to simpler types of attention to an exogenous stimulus, was accompanied by the appearance of a highly synchronized EEG θ rhythm (3.6–5.2 Hz) with a topical amplitude maximum in the lower temporal associative areas of the cortex. The ability to maintain EnA in children of this age is directly related to the reactive increase in the θ rhythm in the lower temporal areas of the cortex during the retention of EnA as compared to ExA. The deficit of EnA control in healthy extremely preterm (HEP) infants was associated with a relative deficit of the functional synchronization of the θ rhythm in the lower temporal areas of the cortex during the retention of EnA compared to full-term infants. In HEP infants, a decreased synchronization of the θ rhythm was equally typical of EnA and ExA. However, its cortical location depended on the type of attention. The functional nature of the θ rhythm inhibition in HEP infants suggests that this abnormality was related to alterations in the neurotransmitter interactions between the limbic and cortical structures, rather than to structural defects. These alterations could be one of the causes of the partial deficit of EnA in HEP infants.     

The neural fate of consciously perceived and missed events in the attentional blink

Cognitive models of attention propose that visual perception is a product of two stages of visual processing: early operations permit rapid initial categorization of the visual world, while later attention-demanding capacity-limited stages are necessary for the conscious report of the stimuli. Here we used the attentional blink paradigm and fMRI to neurally distinguish these two stages of vision. Subjects detected a face target and a scene target presented rapidly among distractors at fixation. Although the second, scene target frequently went undetected by the subjects, it nonetheless activated regions of the medial temporal cortex involved in high-level scene representations, the parahippocampal place area (PPA). This PPA activation was amplified when the stimulus was consciously perceived. By contrast, the frontal cortex was activated only when scenes were successfully reported. These results suggest that medial temporal cortex permits rapid categorization of the visual input, while the frontal cortex is part of a capacity-limited attentional bottleneck to conscious report.