Saccadic motor planning by integrating visual information and pre-information on neural dynamic fields

A functional model of target selection in the saccadic system is presented, incorporating elements of visual processing, motor planning, and motor control. We address the integration of visual information with pre-information. which is provided by manipulating the probability that a target appears at a certain location. This integration is achieved within a dynamic representation of planned eye movement which is modeled through distributions of activation on a topographic field. Visual input evokes activation, which is also constrained by lateral interaction within the field and by preshaping input representing pre-information. The model describes target selection observable in paradigms in which visual goals are presented at more than one location. Specifically, we model the transition from averaging, where endpoints of first saccades fall between two visual target locations, to decision making, where endpoints of first saccades fall accurately onto one of two simultaneously presented visual targets. We make predictions about how metrical biases of first saccades are induced by pre-information about target locations acquired by learning. When coupled to a motor control stage, activation dynamics on the planning level contribute to stabilizing gaze under fixation conditions. The neurophysiological relevance of our functional model is discussed.     

A model of the saccade-generating system that accounts for trajectory variations produced by competing visual stimuli

Variable saccade trajectories are produced in visual search paradigms in which multiple potential target stimuli are present. These variable trajectories provide a rich source of information that may lead to a deeper understanding of the basic control mechanisms of the saccadic system. We have used published behavioral observations and neural recordings in the superior colliculus (SC), gathered in monkeys performing visual search paradigms, to guide the construction of a new distributed model of the saccadic system. The new model can account for many of the variations in saccade trajectory produced by the appearance of multiple visual stimuli in a search paradigm. The model uses distributed feedback about current eye motion from the brainstem to the SC to reduce activity there at physiologically realistic rates during saccades. The long-range lateral inhibitory connections between SC cells used in previous models have been eliminated to match recent physiological evidence. The model features interactions between visually activated multiple populations of cells in the SC and distributed and topologically organized inhibitory input to the SC from the SNr to produce some of the types of variable saccadic trajectories, including slightly curved and averaging saccades, observed in visual search tasks. The distributed perisaccadic disinhibition of SC from the substantia nigra (SNr) is assumed to have broad spatial tuning. In order to produce the strongly curved saccades occasionally recorded in visual search, the existence of a parallel input to the saccadic burst generators in addition to that provided by the distributed input from the SC is required. The spatiotemporal form of this additional parallel input is computed based on the assumption that the input from the model SC is realistic. In accordance with other recent models, it is assumed that the parallel input comes from the cerebellum, but our model predicts that the parallel input is delayed during highly curved saccadic trajectories.     

Dissociated Roles of the Inferior Frontal Gyrus and Superior Temporal Sulcus in Audiovisual Processing: Top-Down and Bottom-Up Mismatch Detection

Visual inputs can distort auditory perception, and accurate auditory processing requires the ability to detect and ignore visual input that is simultaneous and incongruent with auditory information. However, the neural basis of this auditory selection from audiovisual information is unknown, whereas integration process of audiovisual inputs is intensively researched. Here, we tested the hypothesis that the inferior frontal gyrus (IFG) and superior temporal sulcus (STS) are involved in top-down and bottom-up processing, respectively, of target auditory information from audiovisual inputs. We recorded high gamma activity (HGA), which is associated with neuronal firing in local brain regions, using electrocorticography while patients with epilepsy judged the syllable spoken by a voice while looking at a voice-congruent or -incongruent lip movement from the speaker. The STS exhibited stronger HGA if the patient was presented with information of large audiovisual incongruence than of small incongruence, especially if the auditory information was correctly identified. On the other hand, the IFG exhibited stronger HGA in trials with small audiovisual incongruence when patients correctly perceived the auditory information than when patients incorrectly perceived the auditory information due to the mismatched visual information. These results indicate that the IFG and STS have dissociated roles in selective auditory processing, and suggest that the neural basis of selective auditory processing changes dynamically in accordance with the degree of incongruity between auditory and visual information.     

A model of binocular brightness and binaural loudness perception in humans with general applications to nonlinear summation of sensory inputs

A single neural model is proposed to account for how responses of the two eyes and two ears combine to form the perception of binocular brightness and binaural loudness respectively. It involves nonlinear reciprocal feedback inhibition between left and right channels, followed by linear summation between the channels. Local circuit synaptic interactions are an important source of nonlinearity. The model combines inputs in a manner that approximates vector magnitude models in general. This suggests that the model can be applied to a variety of circumstances beyond the visual and auditory data discussed here.     

Visual responses on neck muscles reveal selective gating that prevents express saccades

Express saccades promote the acquisition of visual targets at extremely short reaction times. Because of the head's considerable inertia, it is unknown whether express saccades are accompanied by a parallel command to the head. Here, by recording electromyographic (EMG) activity from monkey neck muscles, we demonstrate that visual target presentation elicits time-locked, lateralized recruitment of neck muscles at extremely short latencies (55-95 ms). Remarkably, such recruitment not only accompanies express saccades, but also precedes nonexpress saccades, occasionally by up to 150 ms. These results demonstrate selective gating of components of descending commands from the superior colliculus to prevent express saccades yet permit recruitment of a head orienting synergy. We conclude that such selective gating aids eye-head coordination by permitting force development at neck muscles while a decision to commit to a gaze shift is being made, optimizing the contribution of the more inertial head to the ensuing gaze shift.     

Enhanced tactile performance at the destination of an upcoming saccade

Previous work has demonstrated that upcoming saccades influence visual and auditory performance even for stimuli presented before the saccade is executed. These studies suggest a close relationship between saccade generation and visual/auditory attention. Furthermore, they provide support for Rizzolatti et al.'s premotor model of attention, which suggests that the same circuits involved in motor programming are also responsible for shifts in covert orienting (shifting attention without moving the eyes or changing posture). In a series of experiments, we demonstrate that saccade programming also affects tactile perception. Participants made speeded saccades to the left and right side as well as tactile discriminations of up versus down. The first experiment demonstrates that participants were reliably faster at responding to tactile stimuli near the location of upcoming saccades. In our second experiment, we had the subjects cross their hands and demonstrated that the effect occurs in visual space (rather than the early representations of touch). In our third experiment, the tactile events usually occurred on the opposite side of upcoming eye movement. We found that the benefit at the saccade target location vanished, suggesting that this shift is not obligatory but that it may be vetoed on the basis of expectation.     

A neuro-synaptic model of the auditory masking and unmasking process

A model of bilateral information processing in the auditory system was presented on the basis of the interaction of postsynaptic potentials intra- and internuclei in order to analyze the mechanism of binaural unmasking as well as monaural masking. The system was composed of a bilateral pair of auditory nuclei, which were organized in two parallel afferent systems as well as an efferent system. In the model, bilateral inputs were processed in three stages, i.e., the detection of interaural differences by the first afferent system, the equalization of relative neural timing by the efferent system, and the cancellation of specific spectral components by the second afferent system. Assuming the masking process to be forward and backward inhibitions on the auditory memory of signal by the ones of noise, the unmasking process could be explained as a result of disinhibition by the cancellation of bilateral masker inputs.This study was supported in part by a Grant-in-Aid for Scientific Research (No. 59870082) from the Japanese Ministry of Education, Science, and Culture     

Cross-Modal Stimulus Conflict: The Behavioral Effects of Stimulus Input Timing in a Visual-Auditory Stroop Task

Cross-modal processing depends strongly on the compatibility between different sensory inputs, the relative timing of their arrival to brain processing components, and on how attention is allocated. In this behavioral study, we employed a cross-modal audio-visual Stroop task in which we manipulated the within-trial stimulus-onset-asynchronies (SOAs) of the stimulus-component inputs, the grouping of the SOAs (blocked vs. random), the attended modality (auditory or visual), and the congruency of the Stroop color-word stimuli (congruent, incongruent, neutral) to assess how these factors interact within a multisensory context. One main result was that visual distractors produced larger incongruency effects on auditory targets than vice versa. Moreover, as revealed by both overall shorter response times (RTs) and relative shifts in the psychometric incongruency-effect functions, visual-information processing was faster and produced stronger and longer-lasting incongruency effects than did auditory. When attending to either modality, stimulus incongruency from the other modality interacted with SOA, yielding larger effects when the irrelevant distractor occurred prior to the attended target, but no interaction with SOA grouping. Finally, relative to neutral-stimuli, and across the wide range of the SOAs employed, congruency led to substantially more behavioral facilitation than did incongruency to interference, in contrast to findings that within-modality stimulus-compatibility effects tend to be more evenly split between facilitation and interference. In sum, the present findings reveal several key characteristics of how we process the stimulus compatibility of cross-modal sensory inputs, reflecting stimulus processing patterns that are critical for successfully navigating our complex multisensory world.     

Summation of Spatiotemporal Input Patterns in Leaky Integrate-and-Fire Neurons: Application to Neurons in the Cochlear Nucleus Receiving Converging Auditory Nerve Fiber Input

The response of leaky integrate-and-fire neurons is analyzed for periodic inputs whose phases vary with their spatial location. The model gives the relationship between the spatial summation distance and the degree of phase locking of the output spikes (i.e., locking to the periodic stochastic inputs, measured by the synchronization index). The synaptic inputs are modeled as an inhomogeneous Poisson process, and the analysis is carried out in the Gaussian approximation. The model has been applied to globular bushy cells of the cochlear nucleus, which receive converging inputs from auditory nerve fibers that originate at neighboring sites in the cochlea. The model elucidates the roles played by spatial summation and coincidence detection, showing how synchronization decreases with an increase in both frequency and spatial spread of inputs. It also shows under what conditions an enhancement of synchronization of the output relative to the input takes place.     

Distortions of subjective time perception within and across senses

Background

The ability to estimate the passage of time is of fundamental importance for perceptual and cognitive processes. One experience of time is the perception of duration, which is not isomorphic to physical duration and can be distorted by a number of factors. Yet, the critical features generating these perceptual shifts in subjective duration are not understood.

Methodology/Findings

We used prospective duration judgments within and across sensory modalities to examine the effect of stimulus predictability and feature change on the perception of duration. First, we found robust distortions of perceived duration in auditory, visual and auditory-visual presentations despite the predictability of the feature changes in the stimuli. For example, a looming disc embedded in a series of steady discs led to time dilation, whereas a steady disc embedded in a series of looming discs led to time compression. Second, we addressed whether visual (auditory) inputs could alter the perception of duration of auditory (visual) inputs. When participants were presented with incongruent audio-visual stimuli, the perceived duration of auditory events could be shortened or lengthened by the presence of conflicting visual information; however, the perceived duration of visual events was seldom distorted by the presence of auditory information and was never perceived shorter than their actual durations.

Conclusions/Significance

These results support the existence of multisensory interactions in the perception of duration and, importantly, suggest that vision can modify auditory temporal perception in a pure timing task. Insofar as distortions in subjective duration can neither be accounted for by the unpredictability of an auditory, visual or auditory-visual event, we propose that it is the intrinsic features of the stimulus that critically affect subjective time distortions.     

A competitive integration model of exogenous and endogenous eye movements

We present a model of the eye movement system in which the programming of an eye movement is the result of the competitive integration of information in the superior colliculi (SC). This brain area receives input from occipital cortex, the frontal eye fields, and the dorsolateral prefrontal cortex, on the basis of which it computes the location of the next saccadic target. Two critical assumptions in the model are that cortical inputs are not only excitatory, but can also inhibit saccades to specific locations, and that the SC continue to influence the trajectory of a saccade while it is being executed. With these assumptions, we account for many neurophysiological and behavioral findings from eye movement research. Interactions within the saccade map are shown to account for effects of distractors on saccadic reaction time (SRT) and saccade trajectory, including the global effect and oculomotor capture. In addition, the model accounts for express saccades, the gap effect, saccadic reaction times for antisaccades, and recorded responses from neurons in the SC and frontal eye fields in these tasks.     

Sensory processing of motor inaccuracy depends on previously performed movement and on subsequent motor corrections: a study of the saccadic system

When goal-directed movements are inaccurate, two responses are generated by the brain: a fast motor correction toward the target and an adaptive motor recalibration developing progressively across subsequent trials. For the saccadic system, there is a clear dissociation between the fast motor correction (corrective saccade production) and the adaptive motor recalibration (primary saccade modification). Error signals used to trigger corrective saccades and to induce adaptation are based on post-saccadic visual feedback. The goal of this study was to determine if similar or different error signals are involved in saccadic adaptation and in corrective saccade generation. Saccadic accuracy was experimentally altered by systematically displacing the visual target during motor execution. Post-saccadic error signals were studied by manipulating visual information in two ways. First, the duration of the displaced target after primary saccade termination was set at 15, 50, 100 or 800 ms in different adaptation sessions. Second, in some sessions, the displaced target was followed by a visual mask that interfered with visual processing. Because they rely on different mechanisms, the adaptation of reactive saccades and the adaptation of voluntary saccades were both evaluated. We found that saccadic adaptation and corrective saccade production were both affected by the manipulations of post-saccadic visual information, but in different ways. This first finding suggests that different types of error signal processing are involved in the induction of these two motor corrections. Interestingly, voluntary saccades required a longer duration of post-saccadic target presentation to reach the same amount of adaptation as reactive saccades. Finally, the visual mask interfered with the production of corrective saccades only during the voluntary saccades adaptation task. These last observations suggest that post-saccadic perception depends on the previously performed action and that the differences between saccade categories of motor correction and adaptation occur at an early level of visual processing.     

A general rule for sensory cue summation: evidence from photographic, musical, phonetic and cross-modal stimuli

The Euclidean and MAX metrics have been widely used to model cue summation psychophysically and computationally. Both rules happen to be special cases of a more general Minkowski summation rule , where m = 2 and ∞, respectively. In vision research, Minkowski summation with power m = 3-4 has been shown to be a superior model of how subthreshold components sum to give an overall detection threshold. Recently, we have previously reported that Minkowski summation with power m = 2.84 accurately models summation of suprathreshold visual cues in photographs. In four suprathreshold discrimination experiments, we confirm the previous findings with new visual stimuli and extend the applicability of this rule to cue combination in auditory stimuli (musical sequences and phonetic utterances, where m = 2.95 and 2.54, respectively) and cross-modal stimuli (m = 2.56). In all cases, Minkowski summation with power m = 2.5-3 outperforms the Euclidean and MAX operator models. We propose that this reflects the summation of neuronal responses that are not entirely independent but which show some correlation in their magnitudes. Our findings are consistent with electrophysiological research that demonstrates signal correlations (r = 0.1-0.2) between sensory neurons when these are presented with natural stimuli.     

Nonlinear visuoauditory integration in the mouse superior colliculus

Sensory information from different modalities is processed in parallel, and then integrated in associative brain areas to improve object identification and the interpretation of sensory experiences. The Superior Colliculus (SC) is a midbrain structure that plays a critical role in integrating visual, auditory, and somatosensory input to assess saliency and promote action. Although the response properties of the individual SC neurons to visuoauditory stimuli have been characterized, little is known about the spatial and temporal dynamics of the integration at the population level. Here we recorded the response properties of SC neurons to spatially restricted visual and auditory stimuli using large-scale electrophysiology. We then created a general, population-level model that explains the spatial, temporal, and intensity requirements of stimuli needed for sensory integration. We found that the mouse SC contains topographically organized visual and auditory neurons that exhibit nonlinear multisensory integration. We show that nonlinear integration depends on properties of auditory but not visual stimuli. We also find that a heuristically derived nonlinear modulation function reveals conditions required for sensory integration that are consistent with previously proposed models of sensory integration such as spatial matching and the principle of inverse effectiveness.     

Control of the superior colliculus by the lateral prefrontal cortex

Several decades of patient, functional imaging and neurophysiological studies have supported a model in which the lateral prefrontal cortex (PFC) acts to suppress unwanted saccades by inhibiting activity in the oculomotor system. However, recent results from combined PFC deactivation and neural recordings of the superior colliculus in monkeys demonstrate that the primary influence of the PFC on the oculomotor system is excitatory, and stands in direct contradiction to the inhibitory model of PFC function. Although erroneous saccades towards a visual stimulus are commonly labelled reflexive in patients with PFC damage or dysfunction, the latencies of most of these saccades are outside of the range of express saccades, which are triggered directly by the visual stimulus. Deactivation and pharmacological manipulation studies in monkeys suggest that response errors following PFC damage or dysfunction are not the result of a failure in response suppression but can best be understood in the context of a failure to maintain and implement the proper task set.     

Bilateral collicular interaction: modulation of auditory signal processing in amplitude domain

In the ascending auditory pathway, the inferior colliculus (IC) receives and integrates excitatory and inhibitory inputs from many lower auditory nuclei, intrinsic projections within the IC, contralateral IC through the commissure of the IC and from the auditory cortex. All these connections make the IC a major center for subcortical temporal and spectral integration of auditory information. In this study, we examine bilateral collicular interaction in modulating amplitude-domain signal processing using electrophysiological recording, acoustic and focal electrical stimulation. Focal electrical stimulation of one (ipsilateral) IC produces widespread inhibition (61.6%) and focused facilitation (9.1%) of responses of neurons in the other (contralateral) IC, while 29.3% of the neurons were not affected. Bilateral collicular interaction produces a decrease in the response magnitude and an increase in the response latency of inhibited IC neurons but produces opposite effects on the response of facilitated IC neurons. These two groups of neurons are not separately located and are tonotopically organized within the IC. The modulation effect is most effective at low sound level and is dependent upon the interval between the acoustic and electric stimuli. The focal electrical stimulation of the ipsilateral IC compresses or expands the rate-level functions of contralateral IC neurons. The focal electrical stimulation also produces a shift in the minimum threshold and dynamic range of contralateral IC neurons for as long as 150 minutes. The degree of bilateral collicular interaction is dependent upon the difference in the best frequency between the electrically stimulated IC neurons and modulated IC neurons. These data suggest that bilateral collicular interaction mainly changes the ratio between excitation and inhibition during signal processing so as to sharpen the amplitude sensitivity of IC neurons. Bilateral interaction may be also involved in acoustic-experience-dependent plasticity in the IC. Three possible neural pathways underlying the bilateral collicular interaction are discussed.     

Latency of visually evoked saccadic eye movements

The paper deals with the initiation of visually guided saccades, in order to break down the saccadic reaction time into functionally different periods of time. It takes into account that spatial processing of information is so basic that modelling of saccadic control properties should include spatio-temporal arrangements. The output signal of the saccadic system was measured in response to visual stimuli in which the time between the appearance of a visual stimulus in the peripheral field and the disappearance of the central fixation point was varied. The variation of the mean saccadic latency time, measured with respect to the onset of the peripheral stimulus, as a function of stimulus asynchrony was highly significant. This variation may be represented by a so-called gap-overlap curve, which is characterized here by means of five parameters. A facilitation model is introduced to fit the results of the gap-overlap experiments. The facilitation model for the initiation of visually evoked saccades incorporates a mechanism which governs the efficiency of processing of signals that arise from a stimulus presented at a particular position in space. It explains how visual information may be affected by other sensory information before it is used to command further saccades. It allows determination of saccadic system parameters, such as the peripheral and the foveal afferent processing time, the central processing time for a saccade and the degree of facilitation. These quantities were found to be characteristic for the given test subjects, and where these data could be compared with neurophysiological data, the agreement was within the experimental error.     

Duration channels mediate human time perception

The task of deciding how long sensory events seem to last is one that the human nervous system appears to perform rapidly and, for sub-second intervals, seemingly without conscious effort. That these estimates can be performed within and between multiple sensory and motor domains suggest time perception forms one of the core, fundamental processes of our perception of the world around us. Given this significance, the current paucity in our understanding of how this process operates is surprising. One candidate mechanism for duration perception posits that duration may be mediated via a system of duration-selective 'channels', which are differentially activated depending on the match between afferent duration information and the channels' 'preferred' duration. However, this model awaits experimental validation. In the current study, we use the technique of sensory adaptation, and we present data that are well described by banks of duration channels that are limited in their bandwidth, sensory-specific, and appear to operate at a relatively early stage of visual and auditory sensory processing. Our results suggest that many of the computational principles the nervous system applies to coding visual spatial and auditory spectral information are common to its processing of temporal extent.     

Speeding up reaction time with invisible stimuli

Normal subjects react more quickly to a pair of visual stimuli than to a stimulus alone. This phenomenon is known as the redundant signal effect (RSE) and represents an example of divided visual attention in which signal processing is carried out in parallel to the advantage of response speed. A most interesting aspect of this phenomenon is that it can occur when one stimulus in a pair cannot be consciously detected because of hemianopia or unilateral extinction resulting from brain damage. Here, we report that a similar dissociation between visual awareness and visually guided behavior is present in normal subjects who show an RSE even when the luminance of one of a pair of stimuli is below detection threshold. The observed RSE cannot be attributed to probability summation because it violates Miller's race inequality and is likely to be related to neural summation between supra- and subthreshold stimuli. Given that a similar implicit RSE is present in hemispherectomy patients, we hypothesize that the site of this summation might be the superior colliculus (SC).     

The three-loop model: a neural network for the generation of saccadic reaction times

This paper presents a computer simulation of the three-loop model for the temporal aspects of the generation of visually guided saccadic eye movements. The intention is to reproduce complex experimental reaction time distributions by a simple neural network. The operating elements are artificial but realistic neurones. Four modules are constructed, each consisting of 16 neural elements. Within each module, the elements are connected in an all-to-all manner. The modules are working parallel and serial according to the anatomically and physiologically identified visuomotor pathways including the superior colliculus, the frontal eye fields, and the parietal cortex. Two transient-sustained input lines drive the network: one represents the visual activity produced by the onset of the saccade target, the other represents a central activity controlling the preparation of saccades, e.g. the end of active fixation. The model works completely deterministically; its stochastic output is a consequence of the stochastic properties of the input only. Simulations show how multimodal distributions of saccadic reaction times are produced as a natural consequence of the model structure. The gap effect on saccadic reaction times is correctly produced by the model: depending only on the gap duration (all model parameters unchanged) express, fast-regular, and slow-regular saccades are obtained in different numbers. In agreement with the experiments, bi- or trimodal distributions are produced only for medium gap durations (around 200 ms), while for shorter or longer gaps the express mode disappears and the distributions turn bi- or even unimodal. The effect of varying the strength of the transient-sustained components and the ongoing activity driving the hierarchically highest module are considered to account for the interindividual variability of the latency distributions obtained from different subjects, effects of different instructions to the same subject, and the observation of express makers (subjects who produce exclusively express saccades). How the model can be extended to describe the spatial aspects of the saccade system will be discussed as well as the effects of training and/or rapid adaptation to experimental conditions.