The effects of task and saliency on latencies for colour and motion processing

In human visual perception, there is evidence that different visual attributes, such as colour, form and motion, have different neural-processing latencies. Specifically, recent studies have suggested that colour changes are processed faster than motion changes. We propose that the processing latencies should not be considered as fixed quantities for different attributes, but instead depend upon attribute salience and the observer's task. We asked observers to respond to high- and low-salience colour and motion changes in three different tasks. The tasks varied from having a strong motor component to having a strong perceptual component. Increasing salience led to shorter processing times in all three tasks. We also found an interaction between task and attribute: motion was processed more quickly in reaction-time tasks, whereas colour was processed more quickly in more perceptual tasks. Our results caution against making direct comparisons between latencies for processing different visual attributes without equating salience or considering task effects. More-salient attributes are processed faster than less-salient ones, and attributes that are critical for the task are also processed more quickly.     

A massively asynchronous,parallel brain

Whether the visual brain uses a parallel or a serial, hierarchical, strategy to process visual signals, the end result appears to be that different attributes of the visual scene are perceived asynchronously—with colour leading form (orientation) by 40 ms and direction of motion by about 80 ms. Whatever the neural root of this asynchrony, it creates a problem that has not been properly addressed, namely how visual attributes that are perceived asynchronously over brief time windows after stimulus onset are bound together in the longer term to give us a unified experience of the visual world, in which all attributes are apparently seen in perfect registration. In this review, I suggest that there is no central neural clock in the (visual) brain that synchronizes the activity of different processing systems. More likely, activity in each of the parallel processing-perceptual systems of the visual brain is reset independently, making of the brain a massively asynchronous organ, just like the new generation of more efficient computers promise to be. Given the asynchronous operations of the brain, it is likely that the results of activities in the different processing-perceptual systems are not bound by physiological interactions between cells in the specialized visual areas, but post-perceptually, outside the visual brain.     

Temporal hierarchy of the visual perceptive systems in the Mondrian world.

Our earlier psychophysical work has shown that colour and motion are not perceived at the same time, with colour leading motion by about 50-100 ms. In pursuing this work, we thought it would be interesting to use a more complex colour stimulus, one in which the wavelength composition of the light reflected or emitted from surfaces changes continually, without entailing a change in the perceived colour (colour constancy). We therefore used a Mondrian figure--an abstract multi-coloured scene with no recognizable objects--against which squares (either red or green) moved up and down, changing colour from red to green in various phase differences with the change in direction of motion. The red and green squares changed continually in their spectral characteristics, as did every other patch on the Mondrian. The results showed that colour is still perceived before motion, by about 80 ms.     

视觉空间注意事件相关电位的协同学分析

在用事件相关电位(event-related potentials,ERP)研究视觉空间注意问题时,直接观察ERP数据就可得出,空间注意的主要作用是对视觉信息处理的调制,它出现在刺激开始后大约80～250ms,主要表现为枕叶的P1、N1和P2波有明显的增强但它们的潜伏期没有变化。采用基于协同学的时空模式分解方法,把视觉空间注意ERP分解为3个模式成分。结果表明,注意不仅使模式1的第一个正波成分(P11)、第一个负波成分(N11)以及第二个正波成分(P12)增强,还使模式3的第一个正波成分(P31)的潜伏期缩短。用探照灯模型对这些现象作了初步解释,说明该方法是研究注意ERP的一种有潜力的新方法。     

Auditory event-related potentials to the apparent auditory image motion

Auditory event-related potentials (ERP) were registered to the dichotically presented white noise stimuli (duration 1500 ms, band 150-1200 Hz). Abrupt or gradual change ofinteraural time difference in the middle of stimuli (750 ms after sound offset) was perceived as an apparent auditory image (AI) instant relocation or motion from the midline to one of the ears. In responses these stimuli two ERPs were observed: one to the sound onset, and second--to the onset of motion or AI relocation. ERPs to AI relocation differed from those to sound onset in longer components latencies (123 ms versus 105 ms for N 1,227 ms versus 190 ms for P2). In responses to AI motion component latencies were even longer (N1: 137 ms, P2: 240 ms); N1 amplitude was greater at sites contralateral to the AI motion direction.     

Functional segregation and temporal hierarchy of the visual perceptive systems.

In extending our previous work, we addressed the question of whether different visual attributes are perceived separately when they belong to different objects, rather than the same one. Using our earlier psychophysical method, but separating the attributes to be paired in two different halves of the screen, we found that human subjects misbind the colour and the direction of motion, or the colour and the orientation of lines, because colour, form, and motion are perceived separately and at different times. The results therefore show that there is a perceptual temporal hierarchy in vision.     

The asynchrony of consciousness.

We present below a simple hypothesis on what we believe is a characteristic of visual consciousness. It is derived from facts about the visual brain revealed in the past quarter of a century, but it relies most especially on psychophysical evidence which shows that different attributes of the visual scene are consciously perceived at different times. This temporal asynchrony in visual perception reveals, we believe, a plurality of visual consciousnesses that are asynchronous with respect to each other, reflecting the modular organization of the visual brain. We further hypothesize that when two attributes (e.g. colour and motion) are presented simultaneously, the activity of cells in a given processing system is sufficient to create a conscious experience of the corresponding attribute (e.g. colour), without the necessity for interaction with the activities of cells in other processing systems (e.g. motion). Thus, any binding of the activity of cells in different systems should be more properly thought of as a binding of the conscious experiences generated in each system.     

Emotion Separation Is Completed Early and It Depends on Visual Field Presentation

It is now apparent that the visual system reacts to stimuli very fast, with many brain areas activated within 100 ms. It is, however, unclear how much detail is extracted about stimulus properties in the early stages of visual processing. Here, using magnetoencephalography we show that the visual system separates different facial expressions of emotion well within 100 ms after image onset, and that this separation is processed differently depending on where in the visual field the stimulus is presented. Seven right-handed males participated in a face affect recognition experiment in which they viewed happy, fearful and neutral faces. Blocks of images were shown either at the center or in one of the four quadrants of the visual field. For centrally presented faces, the emotions were separated fast, first in the right superior temporal sulcus (STS; 35–48 ms), followed by the right amygdala (57–64 ms) and medial pre-frontal cortex (83–96 ms). For faces presented in the periphery, the emotions were separated first in the ipsilateral amygdala and contralateral STS. We conclude that amygdala and STS likely play a different role in early visual processing, recruiting distinct neural networks for action: the amygdala alerts sub-cortical centers for appropriate autonomic system response for fight or flight decisions, while the STS facilitates more cognitive appraisal of situations and links appropriate cortical sites together. It is then likely that different problems may arise when either network fails to initiate or function properly.     

Sequences of predictive eye movements form a fractional Brownian series – implications for self-organized criticality in the oculomotor system

When human subjects are presented with a pair of visual targets that alternate periodically, they track the targets with rapid eye movements known as saccades. In previous work we demonstrated that at low pacing rates (<0.5 Hz), saccades have a latency of about 180 ms, and the latencies are uncorrelated from trial to trial. At high pacing rates (>0.6 Hz), latencies are much shorter: subjects make predictive saccades that anticipate target motion. The predictive latencies are correlated and appear to form a fractional Brownian motion. Here we confirm this finding by examining the rate of decay of nonlinear forecasting of predictive latencies. We further characterize the nature of predictive saccade latencies through the use of detrended fluctuation analysis and surrogate data. These results lead us to conclude that predictive saccades may exhibit a form of self-organized criticality, which enables rapid response to changes in stimulus timing. We provide an experimental demonstration of this.     

Changes in early cortical visual processing predict enhanced reactivity in deaf individuals

Individuals with profound deafness rely critically on vision to interact with their environment. Improvement of visual performance as a consequence of auditory deprivation is assumed to result from cross-modal changes occurring in late stages of visual processing. Here we measured reaction times and event-related potentials (ERPs) in profoundly deaf adults and hearing controls during a speeded visual detection task, to assess to what extent the enhanced reactivity of deaf individuals could reflect plastic changes in the early cortical processing of the stimulus. We found that deaf subjects were faster than hearing controls at detecting the visual targets, regardless of their location in the visual field (peripheral or peri-foveal). This behavioural facilitation was associated with ERP changes starting from the first detectable response in the striate cortex (C1 component) at about 80 ms after stimulus onset, and in the P1 complex (100-150 ms). In addition, we found that P1 peak amplitudes predicted the response times in deaf subjects, whereas in hearing individuals visual reactivity and ERP amplitudes correlated only at later stages of processing. These findings show that long-term auditory deprivation can profoundly alter visual processing from the earliest cortical stages. Furthermore, our results provide the first evidence of a co-variation between modified brain activity (cortical plasticity) and behavioural enhancement in this sensory-deprived population.     

Spatio-temporal brain mapping of motion-onset VEPs combined with fMRI and retinotopic maps

Neuroimaging studies have identified several motion-sensitive visual areas in the human brain, but the time course of their activation cannot be measured with these techniques. In the present study, we combined electrophysiological and neuroimaging methods (including retinotopic brain mapping) to determine the spatio-temporal profile of motion-onset visual evoked potentials for slow and fast motion stimuli and to localize its neural generators. We found that cortical activity initiates in the primary visual area (V1) for slow stimuli, peaking 100 ms after the onset of motion. Subsequently, activity in the mid-temporal motion-sensitive areas, MT+, peaked at 120 ms, followed by peaks in activity in the more dorsal area, V3A, at 160 ms and the lateral occipital complex at 180 ms. Approximately 250 ms after stimulus onset, activity fast motion stimuli was predominant in area V6 along the parieto-occipital sulcus. Finally, at 350 ms (100 ms after the motion offset) brain activity was visible again in area V1. For fast motion stimuli, the spatio-temporal brain pattern was similar, except that the first activity was detected at 70 ms in area MT+. Comparing functional magnetic resonance data for slow vs. fast motion, we found signs of slow-fast motion stimulus topography along the posterior brain in at least three cortical regions (MT+, V3A and LOR).     

Representation of motion onset and offset in an augmented Barlow-Levick model of motion detection

Kinetic occlusion produces discontinuities in the optic flow field, whose perception requires the detection of an unexpected onset or offset of otherwise predictably moving or stationary contrast patches. Many cells in primate visual cortex are directionally selective for moving contrasts, and recent reports suggest that this selectivity arises through the inhibition of contrast signals moving in the cells’ null direction, as in the rabbit retina. This nulling inhibition circuit (Barlow-Levick) is here extended to also detect motion onsets and offsets. The selectivity of extended circuit units, measured as a peak evidence accumulation response to motion onset/offset compared to the peak response to constant motion, is analyzed as a function of stimulus speed. Model onset cells are quiet during constant motion, but model offset cells activate during constant motion at slow speeds. Consequently, model offset cell speed tuning is biased towards higher speeds than onset cell tuning, similarly to the speed tuning of cells in the middle temporal area when exposed to speed ramps. Given a population of neurons with different preferred speeds, this asymmetry addresses a behavioral paradox—why human subjects in a simple reaction time task respond more slowly to motion offsets than onsets for low speeds, even though monkey neuron firing rates react more quickly to the offset of a preferred stimulus than to its onset.     

Work of the human visual system. I. Adequate visual stimulus]

It is shown that the adequate stimulus permitting to detect the presence of colour differentiation in the visual field is the change of relative space-time differences of light actions in different retinal points. Differences only in space or only in time are not sufficient for perception.     

Motion-reversal visual evoked responses.

Motion-reversal visual evoked responses (VERs) have remarkable waveform variability. In our opinion this is caused by the alternative predominance of either motion or pattern-onset/offset related components. The motion dependent component of motion-reversal VER closely resembles motion-onset VER (main negative peak with the latency of about 170 ms), the first positive peak (with the latency of about 100 ms) corresponds to the pattern-onset component and the second non-constant positive peak (with the latency of about 130 ms) seems to be identical with the pattern-offset positivity. The differences in expression of these components are dependent on some stimulus characteristics (mainly on the contrast of a structure, velocity of motion, retinal localization of the stimulus) and on substantial differences in the sensitivity of subjects to motion stimulation.     

Assessment of novel binocular colour, motion and contrast tests in glaucoma

The effects of glaucoma on binocular visual sensitivity for the detection of various stimulus attributes are investigated at the fovea and in four paracentral retinal regions. The study employed a number of visual stimuli designed to isolate the processing of various stimulus attributes. We measured absolute contrast detection thresholds and functional contrast sensitivity by using Landolt ring stimuli. This psychophysical Landolt C-based contrast test of detection and gap discrimination allowed us to test parafoveally at 6 ° from fixation and foveally by employing interleaved testing locations. First-order motion perception was examined by using moving stimuli embedded in static luminance contrast noise. Red/green (RG) and yellow/blue (YB) colour thresholds were measured with the Colour Assessment and Diagnosis (CAD) test, which utilises random dynamic luminance contrast noise (± 45 %) to ensure that only colour and not luminance signals are available for target detection. Subjects were normal controls (n?=?65) and glaucoma patients with binocular visual field defects (n?=?15) classified based on their Humphrey Field Analyzer mean deviation (MD) scores. The impairment of visual function varied depending on the stimulus attribute and location tested. Progression of loss was noted for all tests as the degree of glaucoma increased. For subjects with mild glaucoma (MD ?0.01 dB to ?6.00 dB) significantly more data points fell outside the normal age-representative range for RG colour thresholds than for any other visual test, followed by motion thresholds. This was particularly the case for the parafoveal data compared with the foveal data. Thus, a multifaceted measure of binocular visual performance, incorporating RG colour and motion test at multiple locations, might provide a better index for comparison with quality of life measures in glaucoma.     

Gustatory neural response latencies in the frog

The major rate limiting steps in bullfrog peripheral nerve gustatoryresponse latencies were studied by measuring glossopharyngealnerve multi-unit activity, detecting response onset times, andcalculating rates of stimulus diffusion to receptor cells andsignal propagation along first order neurons. The stimulus deliverytechnique minimized physicochemical and mechanical artifacts,as well as neural responses to mechanical stimulation of thetongue. Neural activity was processed in 10 ms bins. Responseonsets were determined by a criterion that compared the statisticalprobability of the neural events during stimulus liquid presentationswith those during both Ringer's solution presentations afteradaptation to Ringer's and no-stimulus control conditions. Thiscriterion yielded response latencies of 70–110 ms for10 mM CaCl₂, 2 mM quinine hydrochloride, and 10^–5 M and10^–6 M cantharidin or Ringer's, and H₂O. No responsesoccurred during presentations of 10^–7 M cantharidin orRinger's after adaptation to Ringer's, or during the no-stimuluscontrol condition. From the measured latencies and calculatedrates of stimulus diffusion to receptor cells, and signal propagationalong first order neurons, we conclude that taste receptor cellevents and not perireceptor or signal propagatiog events arethe major rate limiting steps in gustatory response latencies.     

Characteristics of Human Luminance Discrimination and Modeling a Neural Network Based on the Response Properties of the Visual Cortex

Reaction time (RT) and error rate that depend on stimulus duration were measured in a luminance-discrimination reaction time task. Two patches of light with different luminance were presented to participants for ‘short’ (150 ms) or ‘long’ (1 s) period on each trial. When the stimulus duration was ‘short’, the participants responded more rapidly with poorer discrimination performance than they did in the longer duration. The results suggested that different sensory responses in the visual cortices were responsible for the dependence of response speed and accuracy on the stimulus duration during the luminance-discrimination reaction time task. It was shown that the simple winner-take-all-type neural network model receiving transient and sustained stimulus information from the primary visual cortex successfully reproduced RT distributions for correct responses and error rates. Moreover, temporal spike sequences obtained from the model network closely resembled to the neural activity in the monkey prefrontal or parietal area during other visual decision tasks such as motion discrimination and oddball detection tasks.     

Visual event-related potentials to moving stimuli: normative data

Visual cognitive responses (P300) to moving stimuli were tested in 36 subjects with the aim to find the normal range of P300 parameters. Concomitantly, the circadian intra-individual variability of the P300 was studied in a subgroup of 6 subjects. Visual stimuli consisted of either coherent (frequent stimulus) or non-coherent motion (random stimulus). The oddball paradigm was applied for recording cognitive responses. P300 to rare stimuli had an average latency of 447.3 +/- 46.6 ms and amplitude of 12.9 +/- 6.0 microV. The average reaction time was in the range from 322 to 611 ms and there was no correlation between the reaction time and P300 latency. We did not find any significant circadian changes of the P300 parameters in the 6 subjects tested four times during the same day. Cognitive (event-related) responses (P300) displayed distinctly greater inter-individual variability (S.D. of 50 ms) when compared with pattern-reversal and motion-onset VEPs (S.D. of 6.0 ms and 14 ms, respectively). For this reason, the clinical use of P300 elicited by this kind of visual stimuli seems to be rather restricted and the evaluation of its intra-individual changes is preferable.     

Fast detection of unexpected sound intensity decrements as revealed by human evoked potentials

The detection of deviant sounds is a crucial function of the auditory system and is reflected by the automatically elicited mismatch negativity (MMN), an auditory evoked potential at 100 to 250 ms from stimulus onset. It has recently been shown that rarely occurring frequency and location deviants in an oddball paradigm trigger a more negative response than standard sounds at very early latencies in the middle latency response of the human auditory evoked potential. This fast and early ability of the auditory system is corroborated by the finding of neurons in the animal auditory cortex and subcortical structures, which restore their adapted responsiveness to standard sounds, when a rare change in a sound feature occurs. In this study, we investigated whether the detection of intensity deviants is also reflected at shorter latencies than those of the MMN. Auditory evoked potentials in response to click sounds were analyzed regarding the auditory brain stem response, the middle latency response (MLR) and the MMN. Rare stimuli with a lower intensity level than standard stimuli elicited (in addition to an MMN) a more negative potential in the MLR at the transition from the Na to the Pa component at circa 24 ms from stimulus onset. This finding, together with the studies about frequency and location changes, suggests that the early automatic detection of deviant sounds in an oddball paradigm is a general property of the auditory system.     

Fast coding of orientation in primary visual cortex

Understanding how populations of neurons encode sensory information is a major goal of systems neuroscience. Attempts to answer this question have focused on responses measured over several hundred milliseconds, a duration much longer than that frequently used by animals to make decisions about the environment. How reliably sensory information is encoded on briefer time scales, and how best to extract this information, is unknown. Although it has been proposed that neuronal response latency provides a major cue for fast decisions in the visual system, this hypothesis has not been tested systematically and in a quantitative manner. Here we use a simple 'race to threshold' readout mechanism to quantify the information content of spike time latency of primary visual (V1) cortical cells to stimulus orientation. We find that many V1 cells show pronounced tuning of their spike latency to stimulus orientation and that almost as much information can be extracted from spike latencies as from firing rates measured over much longer durations. To extract this information, stimulus onset must be estimated accurately. We show that the responses of cells with weak tuning of spike latency can provide a reliable onset detector. We find that spike latency information can be pooled from a large neuronal population, provided that the decision threshold is scaled linearly with the population size, yielding a processing time of the order of a few tens of milliseconds. Our results provide a novel mechanism for extracting information from neuronal populations over the very brief time scales in which behavioral judgments must sometimes be made.