Mental fatigue modulates dynamic adaptation to perceptual demand in speeded detection

When stimulus intensity in simple reaction-time tasks randomly varies across trials, detection speed usually improves after a low-intensity trial. With auditory stimuli, this improvement was often found to be asymmetric, being greater on current low-intensity trials. Our study investigated (1) whether asymmetric sequential intensity adaptation also occurs with visual stimuli; (2) whether these adjustments reflect decision-criterion shifts or, rather, a modulation of perceptual sensitivity; and (3) how sequential intensity adaptation and its underlying mechanisms are affected by mental fatigue induced through prolonged performance. In a continuous speeded detection task with randomly alternating high- and low-intensity visual stimuli, the reaction-time benefit after low-intensity trials was greater on subsequent low- than high-intensity trials. This asymmetry, however, only developed with time on task (TOT). Signal-detection analyses showed that the decision criterion transiently became more liberal after a low-intensity trial, whereas observer sensitivity increased when the preceding and current stimulus were of equal intensity. TOT-induced mental fatigue only affected sensitivity, which dropped more on low- than on high-intensity trials. This differential fatigue-related sensitivity decrease selectively enhanced the impact of criterion down-shifts on low-intensity trials, revealing how the interplay of two perceptual mechanisms and their modulation by fatigue combine to produce the observed overall pattern of asymmetric performance adjustments to varying visual intensity in continuous speeded detection. Our results have implications for similar patterns of sequential demand adaptation in other cognitive domains as well as for real-world prolonged detection performance.     

Emotional Picture and Word Processing: An fMRI Study on Effects of Stimulus Complexity

Neuroscientific investigations regarding aspects of emotional experiences usually focus on one stimulus modality (e.g., pictorial or verbal). Similarities and differences in the processing between the different modalities have rarely been studied directly. The comparison of verbal and pictorial emotional stimuli often reveals a processing advantage of emotional pictures in terms of larger or more pronounced emotion effects evoked by pictorial stimuli. In this study, we examined whether this picture advantage refers to general processing differences or whether it might partly be attributed to differences in visual complexity between pictures and words. We first developed a new stimulus database comprising valence and arousal ratings for more than 200 concrete objects representable in different modalities including different levels of complexity: words, phrases, pictograms, and photographs. Using fMRI we then studied the neural correlates of the processing of these emotional stimuli in a valence judgment task, in which the stimulus material was controlled for differences in emotional arousal. No superiority for the pictorial stimuli was found in terms of emotional information processing with differences between modalities being revealed mainly in perceptual processing regions. While visual complexity might partly account for previously found differences in emotional stimulus processing, the main existing processing differences are probably due to enhanced processing in modality specific perceptual regions. We would suggest that both pictures and words elicit emotional responses with no general superiority for either stimulus modality, while emotional responses to pictures are modulated by perceptual stimulus features, such as picture complexity.     

Opposite Distortions in Interval Timing Perception for Visual and Auditory Stimuli with Temporal Modulations

When an object is presented visually and moves or flickers, the perception of its duration tends to be overestimated. Such an overestimation is called time dilation. Perceived time can also be distorted when a stimulus is presented aurally as an auditory flutter, but the mechanisms and their relationship to visual processing remains unclear. In the present study, we measured interval timing perception while modulating the temporal characteristics of visual and auditory stimuli, and investigated whether the interval times of visually and aurally presented objects shared a common mechanism. In these experiments, participants compared the durations of flickering or fluttering stimuli to standard stimuli, which were presented continuously. Perceived durations for auditory flutters were underestimated, while perceived durations of visual flickers were overestimated. When auditory flutters and visual flickers were presented simultaneously, these distortion effects were cancelled out. When auditory flutters were presented with a constantly presented visual stimulus, the interval timing perception of the visual stimulus was affected by the auditory flutters. These results indicate that interval timing perception is governed by independent mechanisms for visual and auditory processing, and that there are some interactions between the two processing systems.     

Perceptual grouping of biological motion promotes binocular rivalry

Investigation of perceptual rivalry between conflicting stimuli presented one to each eye can further understanding of the neural underpinnings of conscious visual perception. During rivalry, visual awareness fluctuates between perceptions of the two stimuli. Here, we demonstrate that high-level perceptual grouping can promote rivalry between stimulus pairs that would otherwise be perceived as nonrivalrous. Perceptual grouping was generated with point-light walker stimuli that simulate human motion, visible only as lights placed on the joints. Although such walking figures are unrecognizable when stationary, recognition judgments as complex as gender and identity can accurately be made from animated displays, demonstrating the efficiency with which our visual system can group dynamic local signals into a globally coherent walking figure. We find that point-light walker stimuli presented one to each eye and in different colors and configurations results in strong rivalry. However, rivalry is minimal when the two walkers are split between the eyes or both presented to one eye. This pattern of results suggests that processing animated walker figures promotes rivalry between signals from the two eyes rather than between higher-level representations of the walkers. This leads us to hypothesize that awareness during binocular rivalry involves the integrated activity of high-level perceptual mechanisms in conjunction with lower-level ocular suppression modulated via cortical feedback.     

Single units and conscious vision.

Figures that can be seen in more than one way are invaluable tools for the study of the neural basis of visual awareness, because such stimuli permit the dissociation of the neural responses that underlie what we perceive at any given time from those forming the sensory representation of a visual pattern. To study the former type of responses, monkeys were subjected to binocular rivalry, and the response of neurons in a number of different visual areas was studied while the animals reported their alternating percepts by pulling levers. Perception-related modulations of neural activity were found to occur to different extents in different cortical visual areas. The cells that were affected by suppression were almost exclusively binocular, and their proportion was found to increase in the higher processing stages of the visual system. The strongest correlations between neural activity and perception were observed in the visual areas of the temporal lobe. A strikingly large number of neurons in the early visual areas remained active during the perceptual suppression of the stimulus, a finding suggesting that conscious visual perception might be mediated by only a subset of the cells exhibiting stimulus selective responses. These physiological findings, together with a number of recent psychophysical studies, offer a new explanation of the phenomenon of binocular rivalry. Indeed, rivalry has long been considered to be closely linked with binocular fusion and stereopsis, and the sequences of dominance and suppression have been viewed as the result of competition between the two monocular channels. The physiological data presented here are incompatible with this interpretation. Rather than reflecting interocular competition, the rivalry is most probably between the two different central neural representations generated by the dichoptically presented stimuli. The mechanisms of rivalry are probably the same as, or very similar to, those underlying multistable perception in general, and further physiological studies might reveal much about the neural mechanisms of our perceptual organization.     

Perceptual Grouping Enhances Visual Plasticity

Visual perceptual learning, a manifestation of neural plasticity, refers to improvements in performance on a visual task achieved by training. Attention is known to play an important role in perceptual learning, given that the observer''s discriminative ability improves only for those stimulus feature that are attended. However, the distribution of attention can be severely constrained by perceptual grouping, a process whereby the visual system organizes the initial retinal input into candidate objects. Taken together, these two pieces of evidence suggest the interesting possibility that perceptual grouping might also affect perceptual learning, either directly or via attentional mechanisms. To address this issue, we conducted two experiments. During the training phase, participants attended to the contrast of the task-relevant stimulus (oriented grating), while two similar task-irrelevant stimuli were presented in the adjacent positions. One of the two flanking stimuli was perceptually grouped with the attended stimulus as a consequence of its similar orientation (Experiment 1) or because it was part of the same perceptual object (Experiment 2). A test phase followed the training phase at each location. Compared to the task-irrelevant no-grouping stimulus, orientation discrimination improved at the attended location. Critically, a perceptual learning effect equivalent to the one observed for the attended location also emerged for the task-irrelevant grouping stimulus, indicating that perceptual grouping induced a transfer of learning to the stimulus (or feature) being perceptually grouped with the task-relevant one. Our findings indicate that no voluntary effort to direct attention to the grouping stimulus or feature is necessary to enhance visual plasticity.     

The dynamics of visual adaptation to faces

Several recent demonstrations using visual adaptation have revealed high-level aftereffects for complex patterns including faces. While traditional aftereffects involve perceptual distortion of simple attributes such as orientation or colour that are processed early in the visual cortical hierarchy, face adaptation affects perceived identity and expression, which are thought to be products of higher-order processing. And, unlike most simple aftereffects, those involving faces are robust to changes in scale, position and orientation between the adapting and test stimuli. These differences raise the question of how closely related face aftereffects are to traditional ones. Little is known about the build-up and decay of the face aftereffect, and the similarity of these dynamic processes to traditional aftereffects might provide insight into this relationship. We examined the effect of varying the duration of both the adapting and test stimuli on the magnitude of perceived distortions in face identity. We found that, just as with traditional aftereffects, the identity aftereffect grew logarithmically stronger as a function of adaptation time and exponentially weaker as a function of test duration. Even the subtle aspects of these dynamics, such as the power-law relationship between the adapting and test durations, closely resembled that of other aftereffects. These results were obtained with two different sets of face stimuli that differed greatly in their low-level properties. We postulate that the mechanisms governing these shared dynamics may be dissociable from the responses of feature-selective neurons in the early visual cortex.     

Influences of attention on auditory aftereffects following purely visual adaptation

Recently, Kitagawa and Ichihara (2002) demonstrated that visual adaptation to an expanding or contracting disk produces a cross-modal visually-induced auditory loudness aftereffect (VALAE), which they attributed to cross-correlations of motion in three-dimensional space. Our experiments extend their results by providing evidence that attending selectively to one of two competing visual stimuli of the same saliency produces a cross-modal VALAE that favors the attended stimulus. These cross-modal attentional effects suggest the existence of integrative spatial mechanisms between vision and audition that are affected by attention.     

Unconscious orientation processing

Recent findings have shown that certain attributes of visual stimuli, like orientation, are registered in cortical areas when the stimulus is unresolvable or perceptually invisible; however, there is no evidence to show that complex forms of orientation processing (e.g., modulatory effects of orientation on the processing of other features) could occur in the absence of awareness. To address these questions, different psychophysical paradigms were designed in six experiments to probe unconscious orientation processing. First we demonstrated orientation-selective adaptation and color-contingent orientation adaptation for peripheral unresolvable Gabor patches. The next experiments showed the modulatory effects of perceptually indiscriminable orientations on apparent motion processing and attentional mechanisms. Finally we investigated disappearance patterns of unresolvable Gabor stimuli during motion-induced blindness (MIB). Abrupt changes in local unresolvable orientations truncated MIB; however, orientation-based grouping failed to affect the MIB pattern when the orientations were unresolvable. Overall results revealed that unresolvable orientations substantially influence perception at multiple levels.     

Transfer of tilt after-effects between second-order cues

Second-order cues are visual stimuli that are detectable by human observers, without eliciting a peak in Fourier energy that corresponds to their perceptual properties. The most commonly studied exemplars of second-order cues are those defined by modulation of local contrast (CM). It is widely accepted that such cues are initially detected separately from first-order, luminance modulated (LM), cues. However, after-effects have been shown to transfer between first- and second-order cues (LM and CM, respectively). This suggests the existence of a late link in the mechanisms that subserve their processing. To extend the investigation of the mechanisms for processing second-order cues we consider cues defined by modulations in local orientation (OM). Using a tilt-after-effect (TAE) paradigm, we found partial transfer of adaptation between LM and OM cues, confirming the presence of a link between first and second-order cues. Furthermore, we found a partial transfer of TAE between OM and CM cues. These results suggest that, at or before the site of adaptation, information from all visual cues is combined. However, as transfer of adaptation is below 100% in all cases, this is only a partial integration of information.     

The effect of stimulus duration and motor response in hemispatial neglect during a visual search task

Patients with hemispatial neglect exhibit a myriad of profound deficits. A hallmark of this syndrome is the patients' absence of awareness of items located in their contralesional space. Many studies, however, have demonstrated that neglect patients exhibit some level of processing of these neglected items. It has been suggested that unconscious processing of neglected information may manifest as a fast denial. This theory of fast denial proposes that neglected stimuli are detected in the same way as non-neglected stimuli, but without overt awareness. We evaluated the fast denial theory by conducting two separate visual search task experiments, each differing by the duration of stimulus presentation. Specifically, in Experiment 1 each stimulus remained in the participants' visual field until a response was made. In Experiment 2 each stimulus was presented for only a brief duration. We further evaluated the fast denial theory by comparing verbal to motor task responses in each experiment. Overall, our results from both experiments and tasks showed no evidence for the presence of implicit knowledge of neglected stimuli. Instead, patients with neglect responded the same when they neglected stimuli as when they correctly reported stimulus absence. These findings thus cast doubt on the concept of the fast denial theory and its consequent implications for non-conscious processing. Importantly, our study demonstrated that the only behavior affected was during conscious detection of ipsilesional stimuli. Specifically, patients were slower to detect stimuli in Experiment 1 compared to Experiment 2, suggesting a duration effect occurred during conscious processing of information. Additionally, reaction time and accuracy were similar when reporting verbally versus motorically. These results provide new insights into the perceptual deficits associated with neglect and further support other work that falsifies the fast denial account of non-conscious processing in hemispatial visual neglect.     

Neural Representation of Ambiguous Visual Objects in the Inferior Temporal Cortex

Inferior temporal (IT) cortex as the final stage of the ventral visual pathway is involved in visual object recognition. In our everyday life we need to recognize visual objects that are degraded by noise. Psychophysical studies have shown that the accuracy and speed of the object recognition decreases as the amount of visual noise increases. However, the neural representation of ambiguous visual objects and the underlying neural mechanisms of such changes in the behavior are not known. Here, by recording the neuronal spiking activity of macaque monkeys’ IT we explored the relationship between stimulus ambiguity and the IT neural activity. We found smaller amplitude, later onset, earlier offset and shorter duration of the response as visual ambiguity increased. All of these modulations were gradual and correlated with the level of stimulus ambiguity. We found that while category selectivity of IT neurons decreased with noise, it was preserved for a large extent of visual ambiguity. This noise tolerance for category selectivity in IT was lost at 60% noise level. Interestingly, while the response of the IT neurons to visual stimuli at 60% noise level was significantly larger than their baseline activity and full (100%) noise, it was not category selective anymore. The latter finding shows a neural representation that signals the presence of visual stimulus without signaling what it is. In general these findings, in the context of a drift diffusion model, explain the neural mechanisms of perceptual accuracy and speed changes in the process of recognizing ambiguous objects.     

Satiation in cutaneous saltation.

The extent of cutaneous saltation (the illusory displacement of a tap presented to one skin locus by another tap occurring close in time at another locus) was modified by a "preconditioning" stimulus presented prior to and at a site distant from the saltatory test pattern. The 10-sec vibratory preconditioning (PC) stimulus appears to be analogous to inspection figures that "satiate" the perceptual field in experiments on figural aftereffects, producing changes in the perceived size, position, or shape of subsequent stimuli. The direction of displacement of the saltatory phantom was always away from the locus of the prior PC stimulus, consistent with results observed in studies of visual and kinesthetic aftereffects. Th- amount of repulsion and the rate at which the saltatory phantom returned to its initial position depend on the intensity, locus, and number of PC stimuli. As with figural aftereffects, these results resist explanation by peripheral mechanisms such as adaptation.     

Opponent and nonopponent contributions to the zebrafish electroretinogram using heterochromatic flicker photometry

While some lower vertebrates, such as zebrafish, do not appear to possess anatomically separate pathways of processing visual information (such as M-pathways and P-pathways), it is believed that separate processing of the visual stimulus (such as luminance and chromatic processing) is a basic requirement of vertebrate vision. In this study, spectral sensitivity functions were obtained from electroretinogram responses to heterochromatic flicker photometry stimuli at several flicker rates, including a low flicker rate (2 Hz), in an attempt to predominantly stimulate chromatic processes and a high flicker rate (16 Hz), in an attempt to predominantly stimulate luminance processes. In addition, chromatic adaptation was used to isolate and examine the temporal properties of the different cone-type contributions to the electroretinogram response. Spectral sensitivity functions based on responses to heterochromatic stimuli of a low flicker rate appeared to receive both opponent and nonopponent contributions; however, when the stimulus flicker rate was high, spectral sensitivity appeared to be a function of only nonopponent mechanisms. Also, the differences in cone contributions to the spectral sensitivity functions across the different flicker rates appear to be related to the temporal properties of the cone contributions to the electroretinogram response.     

Stimulus dependency of object-evoked responses in human visual cortex: an inverse problem for category specificity

Many studies have linked the processing of different object categories to specific event-related potentials (ERPs) such as the face-specific N170. Despite reports showing that object-related ERPs are influenced by visual stimulus features, there is consensus that these components primarily reflect categorical aspects of the stimuli. Here, we re-investigated this idea by systematically measuring the effects of visual feature manipulations on ERP responses elicited by both structure-from-motion (SFM)-defined and luminance-defined object stimuli. SFM objects elicited a novel component at 200-250 ms (N250) over parietal and posterior temporal sites. We found, however, that the N250 amplitude was unaffected by restructuring SFM stimuli into meaningless objects based on identical visual cues. This suggests that this N250 peak was not uniquely linked to categorical aspects of the objects, but is strongly determined by visual stimulus features. We provide strong support for this hypothesis by parametrically manipulating the depth range of both SFM- and luminance-defined object stimuli and showing that the N250 evoked by SFM stimuli as well as the well-known N170 to static faces were sensitive to this manipulation. Importantly, this effect could not be attributed to compromised object categorization in low depth stimuli, confirming a strong impact of visual stimulus features on object-related ERP signals. As ERP components linked with visual categorical object perception are likely determined by multiple stimulus features, this creates an interesting inverse problem when deriving specific perceptual processes from variations in ERP components.     

How do we select perceptions and actions? Human brain imaging studies.

The selective nature of human perception and action implies a modulatory interaction between sensorimotor processes and attentional processes. This paper explores the use of functional imaging in humans to explore the mechanisms of perceptual selection and the fate of irrelevant stimuli that are not selected. Experiments with positron emission tomography show that two qualitatively different patterns of modulation of cerebral blood flow can be observed in experiments where non-spatial visual attention and auditory attention are manipulated. These patterns of modulation of cerebral blood flow modulation can be described as gain control and bias signal mechanisms. In visual and auditory cortex, the dominant change in cerebral blood flow associated with attention to either modality is related to a bias signal. The relation of these patterns of modulation to attentional effects that have been observed in single neurons is discussed. The existence of mechanisms for selective perception raises the more general question of whether irrelevant ignored stimuli are nevertheless perceived. Lavie''s theory of attention proposes that the degree to which ignored stimuli are processed varies depending on the perceptual load of the current task. Evidence from behavioural and functional magnetic resonance imaging studies of ignored visual motion processing is presented in support of this proposal.     

Attending to the possibilities of action

Actions taking place in the environment are critical for our survival. We review evidence on attention to action, drawing on sets of converging evidence from neuropsychological patients through to studies of the time course and neural locus of action-based cueing of attention in normal observers. We show that the presence of action relations between stimuli helps reduce visual extinction in patients with limited attention to the contralesional side of space, while the first saccades made by normal observers and early perceptual and attentional responses measured using electroencephalography/event-related potentials are modulated by preparation of action and by seeing objects being grasped correctly or incorrectly for action. With both normal observers and patients, there is evidence for two components to these effects based on both visual perceptual and motor-based responses. While the perceptual responses reflect factors such as the visual familiarity of the action-related information, the motor response component is determined by factors such as the alignment of the objects with the observer''s effectors and not by the visual familiarity of the stimuli. In addition to this, we suggest that action relations between stimuli can be coded pre-attentively, in the absence of attention to the stimulus, and action relations cue perceptual and motor responses rapidly and automatically. At present, formal theories of visual attention are not set up to account for these action-related effects; we suggest ways that theories could be expected to enable action effects to be incorporated.     

Neural correlates of individual performance differences in resolving perceptual conflict

Attentional mechanisms are a crucial prerequisite to organize behavior. Most situations may be characterized by a 'competition' between salient, but irrelevant stimuli and less salient, relevant stimuli. In such situations top-down and bottom-up mechanisms interact with each other. In the present fMRI study, we examined how interindividual differences in resolving situations of perceptual conflict are reflected in brain networks mediating attentional selection. Doing so, we employed a change detection task in which subjects had to detect luminance changes in the presence and absence of competing distractors. The results show that good performers presented increased activation in the orbitofrontal cortex (BA 11), anterior cingulate (BA 25), inferior parietal lobule (BA 40) and visual areas V2 and V3 but decreased activation in BA 39. This suggests that areas mediating top-down attentional control are stronger activated in this group. Increased activity in visual areas reflects distinct neuronal enhancement relating to selective attentional mechanisms in order to solve the perceptual conflict. Opposed to good performers, brain areas activated by poor performers comprised the left inferior parietal lobule (BA 39) and fronto-parietal and visual regions were continuously deactivated, suggesting that poor performers perceive stronger conflict than good performers. Moreover, the suppression of neural activation in visual areas might indicate a strategy of poor performers to inhibit the processing of the irrelevant non-target feature. These results indicate that high sensitivity in perceptual areas and increased attentional control led to less conflict in stimulus processing and consequently to higher performance in competitive attentional selection.     

General validity of Levelt's propositions reveals common computational mechanisms for visual rivalry

The mechanisms underlying conscious visual perception are often studied with either binocular rivalry or perceptual rivalry stimuli. Despite existing research into both types of rivalry, it remains unclear to what extent their underlying mechanisms involve common computational rules. Computational models of binocular rivalry mechanisms are generally tested against Levelt's four propositions, describing the psychophysical relation between stimulus strength and alternation dynamics in binocular rivalry. Here we use a bistable rotating structure-from-motion sphere, a generally studied form of perceptual rivalry, to demonstrate that Levelt's propositions also apply to the alternation dynamics of perceptual rivalry. Importantly, these findings suggest that bistability in structure-from-motion results from active cross-inhibition between neural populations with computational principles similar to those present in binocular rivalry. Thus, although the neural input to the computational mechanism of rivalry may stem from different cortical neurons and different cognitive levels the computational principles just prior to the production of visual awareness appear to be common to the two types of rivalry.