Correlation versus gradient type motion detectors: the pros and cons

Visual motion contains a wealth of information about self-motion as well as the three-dimensional structure of the environment. Therefore, it is of utmost importance for any organism with eyes. However, visual motion information is not explicitly represented at the photoreceptor level, but rather has to be computed by the nervous system from the changing retinal images as one of the first processing steps. Two prominent models have been proposed to account for this neural computation: the Reichardt detector and the gradient detector. While the Reichardt detector correlates the luminance levels derived from two adjacent image points, the gradient detector provides an estimate of the local retinal image velocity by dividing the spatial and the temporal luminance gradient. As a consequence of their different internal processing structure, both the models differ in a number of functional aspects such as their dependence on the spatial-pattern structure as well as their sensitivity to photon noise. These different properties lead to the proposal that an ideal motion detector should be of Reichardt type at low luminance levels, but of gradient type at high luminance levels. However, experiments on the fly visual systems provided unambiguous evidence in favour of the Reichardt detector under all luminance conditions. Does this mean that the fly nervous system uses suboptimal computations, or is there a functional aspect missing in the optimality criterion? In the following, I will argue in favour of the latter, showing that Reichardt detectors have an automatic gain control allowing them to dynamically adjust their input–output relationships to the statistical range of velocities presented, while gradient detectors do not have this property. As a consequence, Reichardt detectors, but not gradient detectors, always provide a maximum amount of information about stimulus velocity over a large range of velocities. This important property might explain why Reichardt type of computations have been demonstrated to underlie the extraction of motion information in the fly visual system under all luminance levels.     

Non-directional motion detectors can be used to mimic optic flow dependent behaviors

Insect navigational behaviors including obstacle avoidance, grazing landings, and visual odometry are dependent on the ability to estimate flight speed based only on visual cues. In honeybees, this visual estimate of speed is largely independent of both the direction of motion and the spatial frequency content of the image. Electrophysiological recordings from the motion-sensitive cells believed to underlie these behaviors have long supported spatio-temporally tuned correlation-type models of visual motion detection whose speed tuning changes as the spatial frequency of a stimulus is varied. The result is an apparent conflict between behavioral experiments and the electrophysiological and modeling data. In this article, we demonstrate that conventional correlation-type models are sufficient to reproduce some of the speed-dependent behaviors observed in honeybees when square wave gratings are used, contrary to the theoretical predictions. However, these models fail to match the behavioral observations for sinusoidal stimuli. Instead, we show that non-directional motion detectors, which underlie the correlation-based computation of directional motion, can be used to mimic these same behaviors even when narrowband gratings are used. The existence of such non-directional motion detectors is supported both anatomically and electrophysiologically, and they have been hypothesized to be critical in the Dipteran elementary motion detector (EMD) circuit.     

Properties of individual movement detectors as derived from behavioural experiments on the visual system of the fly

The performance of the fly's movement detection system is analysed using the visually induced yaw torque generated during tethered flight as a behavioural indicator. In earlier studies usually large parts of the visual field were exposed to the movement stimuli; the fly's response, therefore, represented the spatially pooled output signals of a large number of local movement detectors. Here we examined the responses of individual movement detectors. The stimulus pattern was presented to the fly via small vertical slits, thus, nearly avoiding spatial integration of local movement information along the horizontal axis of the eye. The stimulus consisted of a vertically oriented sine-wave grating which was moved with a constant velocity either clockwise or counterclockwise. In agreement with the theory of movement detectors of the correlation type, the time-course of the detector signal is modulated with the spatial phase of the stimulus pattern. It can even assume negative values for some time during the response cycle and thus signal the wrong direction of motion. By spatially integrating the response over sufficiently large arrays of movement detectors these response modulations disappear. Finally, one obtains a signal of the movement detection system which is constant while the pattern moves in one direction and only changes its sign when the pattern reverses its direction of motion. Spatial integration thus represents a simple means to obtain a meaningful representations of motion information.     

Integration of Motion Responses Underlying Directional Motion Anisotropy in Human Early Visual Cortical Areas

Recent imaging studies have reported directional motion biases in human visual cortex when perceiving moving random dot patterns. It has been hypothesized that these biases occur as a result of the integration of motion detector activation along the path of motion in visual cortex. In this study we investigate the nature of such motion integration with functional MRI (fMRI) using different motion stimuli. Three types of moving random dot stimuli were presented, showing either coherent motion, motion with spatial decorrelations or motion with temporal decorrelations. The results from the coherent motion stimulus reproduced the centripetal and centrifugal directional motion biases in V1, V2 and V3 as previously reported. The temporally decorrelated motion stimulus resulted in both centripetal and centrifugal biases similar to coherent motion. In contrast, the spatially decorrelated motion stimulus resulted in small directional motion biases that were only present in parts of visual cortex coding for higher eccentricities of the visual field. In combination with previous results, these findings indicate that biased motion responses in early visual cortical areas most likely depend on the spatial integration of a simultaneously activated motion detector chain.     

Primary visual cortex activity along the apparent-motion trace reflects illusory perception

The illusion of apparent motion can be induced when visual stimuli are successively presented at different locations. It has been shown in previous studies that motion-sensitive regions in extrastriate cortex are relevant for the processing of apparent motion, but it is unclear whether primary visual cortex (V1) is also involved in the representation of the illusory motion path. We investigated, in human subjects, apparent-motion-related activity in patches of V1 representing locations along the path of illusory stimulus motion using functional magnetic resonance imaging. Here we show that apparent motion caused a blood-oxygenation-level-dependent response along the V1 representations of the apparent-motion path, including regions that were not directly activated by the apparent-motion-inducing stimuli. This response was unaltered when participants had to perform an attention-demanding task that diverted their attention away from the stimulus. With a bistable motion quartet, we confirmed that the activity was related to the conscious perception of movement. Our data suggest that V1 is part of the network that represents the illusory path of apparent motion. The activation in V1 can be explained either by lateral interactions within V1 or by feedback mechanisms from higher visual areas, especially the motion-sensitive human MT/V5 complex.     

Shifts in coding properties and maintenance of information transmission during adaptation in barrel cortex

Neuronal responses to ongoing stimulation in many systems change over time, or “adapt.” Despite the ubiquity of adaptation, its effects on the stimulus information carried by neurons are often unknown. Here we examine how adaptation affects sensory coding in barrel cortex. We used spike-triggered covariance analysis of single-neuron responses to continuous, rapidly varying vibrissa motion stimuli, recorded in anesthetized rats. Changes in stimulus statistics induced spike rate adaptation over hundreds of milliseconds. Vibrissa motion encoding changed with adaptation as follows. In every neuron that showed rate adaptation, the input–output tuning function scaled with the changes in stimulus distribution, allowing the neurons to maintain the quantity of information conveyed about stimulus features. A single neuron that did not show rate adaptation also lacked input–output rescaling and did not maintain information across changes in stimulus statistics. Therefore, in barrel cortex, rate adaptation occurs on a slow timescale relative to the features driving spikes and is associated with gain rescaling matched to the stimulus distribution. Our results suggest that adaptation enhances tactile representations in primary somatosensory cortex, where they could directly influence perceptual decisions.     

Propagation of photon noise and information transfer in visual motion detection

The extraction of the direction of motion from the time varying retinal images is one of the most basic tasks any visual system is confronted with. However, retinal images are severely corrupted by photon noise, in particular at low light levels, thus limiting the performance of motion detection mechanisms of what sort so ever. Here, we study how photon noise propagates through an array of Reichardt-type motion detectors that are commonly believed to underlie fly motion vision. We provide closed-form analytical expressions of the signal and noise spectra at the output of such a motion detector array. We find that Reichardt detectors reveal favorable noise suppression in the frequency range where most of the signal power resides. Most notably, due to inherent adaptive properties, the transmitted information about stimulus velocity remains nearly constant over a large range of velocity entropies. Action editor: Matthew Wiener     

Adaptation accentuates responses of fly motion-sensitive visual neurons to sudden stimulus changes

Adaptation in sensory and neuronal systems usually leads to reduced responses to persistent or frequently presented stimuli. In contrast to simple fatigue, adapted neurons often retain their ability to encode changes in stimulus intensity and to respond when novel stimuli appear. We investigated how the level of adaptation of a fly visual motion-sensitive neuron affects its responses to discontinuities in the stimulus, i.e. sudden brief changes in one of the stimulus parameters (velocity, contrast, grating orientation and spatial frequency). Although the neuron''s overall response decreased gradually during ongoing motion stimulation, the response transients elicited by stimulus discontinuities were preserved or even enhanced with adaptation. Moreover, the enhanced sensitivity to velocity changes by adaptation was not restricted to a certain velocity range, but was present regardless of whether the neuron was adapted to a baseline velocity below or above its steady-state velocity optimum. Our results suggest that motion adaptation helps motion-sensitive neurons to preserve their sensitivity to novel stimuli even in the presence of strong tonic stimulation, for example during self-motion.     

The Effect of Visual Apparent Motion on Audiovisual Simultaneity

Visual motion information from dynamic environments is important in multisensory temporal perception. However, it is unclear how visual motion information influences the integration of multisensory temporal perceptions. We investigated whether visual apparent motion affects audiovisual temporal perception. Visual apparent motion is a phenomenon in which two flashes presented in sequence in different positions are perceived as continuous motion. Across three experiments, participants performed temporal order judgment (TOJ) tasks. Experiment 1 was a TOJ task conducted in order to assess audiovisual simultaneity during perception of apparent motion. The results showed that the point of subjective simultaneity (PSS) was shifted toward a sound-lead stimulus, and the just noticeable difference (JND) was reduced compared with a normal TOJ task with a single flash. This indicates that visual apparent motion affects audiovisual simultaneity and improves temporal discrimination in audiovisual processing. Experiment 2 was a TOJ task conducted in order to remove the influence of the amount of flash stimulation from Experiment 1. The PSS and JND during perception of apparent motion were almost identical to those in Experiment 1, but differed from those for successive perception when long temporal intervals were included between two flashes without motion. This showed that the result obtained under the apparent motion condition was unaffected by the amount of flash stimulation. Because apparent motion was produced by a constant interval between two flashes, the results may be accounted for by specific prediction. In Experiment 3, we eliminated the influence of prediction by randomizing the intervals between the two flashes. However, the PSS and JND did not differ from those in Experiment 1. It became clear that the results obtained for the perception of visual apparent motion were not attributable to prediction. Our findings suggest that visual apparent motion changes temporal simultaneity perception and improves temporal discrimination in audiovisual processing.     

Motion-sensitive neurones in V5/MT modulate perceived spatial position

Until recently, it was widely believed that object position and object motion were represented independently in the visual cortex. However, several studies have shown that adaptation to motion produces substantial shifts in the perceived position of subsequently viewed stationary objects. Two stages of motion adaptation have been proposed: an initial stage at the level of V1 and a secondary stage thought to be located in V5/MT. Indeed, selective adaptation can be demonstrated at each of these levels of motion analysis. What remains unknown is which of these cortical sites are involved in modulating the positional representation of subsequently viewed objects. To answer this question directly, we disrupted cortical activity by using transcranial magnetic stimulation (TMS) immediately after motion adaptation. When TMS was delivered to V5/MT after motion adaptation, the perceived offset of the test stimulus was greatly reduced. In marked contrast, TMS of V1 had no effect on the changes that normally occur in perceived position after motion adaptation. This result demonstrates that the anatomical locus at which motion and positional information interact is area V5/MT rather than V1/V2.     

Dynamic response properties of movement detectors: Theoretical analysis and electrophysiological investigation in the visual system of the fly

Dynamic aspects of the computation of visual motion information are analysed both theoretically and experimentally. The theoretical analysis is based on the type of movement detector which has been proposed to be realized in the visual system of insects (e.g. Hassenstein and Reichardt 1956; Reichardt 1957, 1961; Buchner 1984), but also of man (e.g. van Doorn and Koenderink 1982a, b; van Santen and Sperling 1984; Wilson 1985). The output of both a single movement detector and a one-dimensional array of detectors is formulated mathematically as a function of time. The resulting movement detector theory can be applied to a much wider range of moving stimuli than has been possible on the basis of previous formulations of the detector output. These stimuli comprise one-dimensional smooth detector input functions, i.e. functions which can be expanded into a time-dependent convergent Taylor series for any value of the spatial coordinate.The movement detector response can be represented by a power series. Each term of this series consists of one exclusively time-dependent component and of another component that depends, in addition, on the properties of the pattern. Even the exclusively time-dependent components of the movement detector output are not solely determined by the stimulus velocity. They rather depend in a non-linear way on the weighted sum of the instantaneous velocity and all its higher order time derivatives. The latter point represents another reason — not discussed so far in the literature — that movement detectors of the type analysed here do not represent pure velocity sensors.The significance of this movement detector theory is established for the visual system of the fly. This is done by comparing the spatially integrated movement detector response with the functional properties of the directionally-selective motion-sensitive. Horizontal Cells of the third visual ganglion of the fly's brain.These integrate local motion information over large parts of the visual field. The time course of the spatially integrated movement detector response is about proportional to the velocity of the stimulus pattern only as long as the pattern velocity and its time derivatives are sufficiently small. For large velocities and velocity changes of the stimulus pattern characteristic deviations of the response profiles from being proportional to pattern velocity are predicted on the basis of the detector theory developed here. These deviations are clearly reflected in the response of the wide-field Horizontal Cells, thus, providing very specific evidence that the movement detector theory developed here can be applied to motion detection in the fly. The characteristic dynamic features of the theoretically predicted and the experimentally determined cellular responses are exploited to estimate the time constant of the movement detector filter.     

Implication of STAT1 and STAT3 transcription factors in the response to superantigens

The activation of STAT1 and STAT3 in response to SEB was analyzed in spleen of Balb/c mice. The intraperitoneal injection of the superantigen SEB activated STAT1 and STAT3 in spleen. Activated STAT1 almost completely disappeared in 24 h even though activated STAT3 was present for more than 48 h after SEB injection. Cyclosporine A was able to block the initial STAT1 activation, but STAT3 activation was only partially affected. SEB also increased the mRNA levels for STAT1, STAT3 and SOCS1. When a second injection with SEB was given 72 h after the first stimulus, STAT1 activation was much lower than that observed after the first stimulation with SEB and no increase in the STAT1 mRNA level was observed. Nevertheless, after this second injection, STAT3 was again activated without any significant interference from the first stimulus and the STAT3 and SOCS1 mRNA levels again increased. These data indicate that a first stimulation with superantigen re-programs cells so that they respond to a second stimulation in a different way. Understanding the mechanisms implicated in this re-programming is basic for designing therapeutic strategies in processes such as septic shock.     

Auditory adaptation in voice perception

Perceptual aftereffects following adaptation to simple stimulus attributes (e.g., motion, color) have been studied for hundreds of years. A striking recent discovery was that adaptation also elicits contrastive aftereffects in visual perception of complex stimuli and faces [1-6]. Here, we show for the first time that adaptation to nonlinguistic information in voices elicits systematic auditory aftereffects. Prior adaptation to male voices causes a voice to be perceived as more female (and vice versa), and these auditory aftereffects were measurable even minutes after adaptation. By contrast, crossmodal adaptation effects were absent, both when male or female first names and when silently articulating male or female faces were used as adaptors. When sinusoidal tones (with frequencies matched to male and female voice fundamental frequencies) were used as adaptors, no aftereffects on voice perception were observed. This excludes explanations for the voice aftereffect in terms of both pitch adaptation and postperceptual adaptation to gender concepts and suggests that contrastive voice-coding mechanisms may routinely influence voice perception. The role of adaptation in calibrating properties of high-level voice representations indicates that adaptation is not confined to vision but is a ubiquitous mechanism in the perception of nonlinguistic social information from both faces and voices.     

Vector representation of associative learning

I. P. Pavlov [12] has shown that conditioned reflexes are selective both with respect to conditioned stimuli and to conditioned reflexes elicited by those conditioned stimuli. At the neuronal level selective aspects of conditioned stimuli are based on detectors selectively tuned to respective stimuli. The selective aspects of conditioned reflexes are due to command neurons representing specific unconditioned reflexes. It can be assumed that conditioned reflexes result from association between selective detectors and specific command neurons. The detectors activated by a conditioned stimulus constitute a combination of excitations--a detector excitation vector. The detector excitation vector acts on a command neuron via a set of plastic synapses--a synaptic weight vector. Plastic synapses are modified in the process of learning making command neuron selectively tuned to a specific conditioned stimulus. The selective tuning of a particular command neuron to a specific excitation vector referred to a conditioned stimulus is a basis of associative learning. The probabilities of conditioned reflexes elicited by conditioned and differential stimuli implicitly contain information concerning excitation vectors that encode respective stimuli. Contribution of the vector code to associative learning was explored combining differential color conditioning with intracellular recording from color-coding neurons. It was shown that colors in carps and monkeys are represented on a hypersphere in the four-dimensional space similar to human color space. The basis of the color space is constituted by red-green, blue-yellow, brightness and darkness neurons.     

The continuous Wagon wheel illusion and the 'when' pathway of the right parietal lobe: a repetitive transcranial magnetic stimulation study

A continuous periodic motion stimulus can sometimes be perceived moving in the wrong direction. These illusory reversals have been taken as evidence that part of the motion perception system samples its inputs as a series of discrete snapshots -although other explanations of the phenomenon have been proposed, that rely on the spurious activation of low-level motion detectors in early visual areas. We have hypothesized that the right inferior parietal lobe ('when' pathway) plays a critical role in timing perceptual events relative to one another, and thus we examined the role of the right parietal lobe in the generation of this "continuous Wagon Wheel Illusion" (c-WWI). Consistent with our hypothesis, we found that the illusion was effectively weakened following disruption of right, but not left, parietal regions by low frequency repetitive transcranial magnetic stimulation (1 Hz, 10 min). These results were independent of whether the motion stimulus was shown in the left or the right visual field. Thus, the c-WWI appears to depend on higher-order attentional mechanisms that are supported by the 'when' pathway of the right parietal lobe.     

Spatially localized distortions of event time

A fundamental question about the perception of time is whether the neural mechanisms underlying temporal judgements are universal and centralized in the brain or modality specific and distributed. Time perception has traditionally been thought to be entirely dissociated from spatial vision. Here we show that the apparent duration of a dynamic stimulus can be manipulated in a local region of visual space by adapting to oscillatory motion or flicker. This implicates spatially localized temporal mechanisms in duration perception. We do not see concomitant changes in the time of onset or offset of the test patterns, demonstrating a direct local effect on duration perception rather than an indirect effect on the time course of neural processing. The effects of adaptation on duration perception can also be dissociated from motion or flicker perception per se. Although 20 Hz adaptation reduces both the apparent temporal frequency and duration of a 10 Hz test stimulus, 5 Hz adaptation increases apparent temporal frequency but has little effect on duration perception. We conclude that there is a peripheral, spatially localized, essentially visual component involved in sensing the duration of visual events.     

Temporal modulation of luminance adapts time constant of fly movement detectors

The time constant of movement detectors in the fly visual system has been proposed to adapt in response to moving stimuli (de Ruyter van Steveninck et al. 1986). The objective of the present study is to analyse, whether this adaptation can be induced as well, if the luminance of a stationary uniform field is modulated in time. The experiments were done on motion-sensitive wide-field neurones of the lobula plate, the posterior part of the third visual ganglion of the blowfly, calliphora erythrocephala. These cells are assumed to receive input from large retinotopic arrays of movement detectors. In order to demonstrate that our results concern the properties of the movement detectors rather than those of a particular wide-field cell we recorded from two different types of them, the H1- and the HSE-cell. Both cell types respond to a brief movement stimulus in their preferred direction with a transient excitation. This response decays about exponentially. The time constant of this decay reflects, in a first approximation, the time constant of the presynaptic movement detectors. It was determined after prestimulation of the cell by the following stimuli: (a) periodic stationary grating; (b) uniform field, the intensity of which was modulated sinusoidally in time (flicker stimulation); (c) periodic grating moving front-to-back; (d) periodic grating moving back-to-front. The decay of the response is significantly faster not only after movement but also after flicker stimulation as compared with pre-stimulation with a stationary stimulus. This is interpreted as an adaptation of the movement detector's time constant. The finding that flicker stimulation also leads to an adaptation shows that movement is not necessary for this process. Instead the adaptation of the time constant appears to be governed mainly by the temporal modulation (i.e., contrast frequency) of the signal in each visual channel.     

A general model for visual motion detection

We propose a general model for detection of both first-order motion and second-order motion. In this model an input stimulus is divided into a number of partially overlapping spatiotemporal local regions. Spatiotemporal frequency analysis is done for every local region using Gabor filters, then the input stimulus (original spatiotemporal signal) is replaced by the outputs of Gabor filters. Local motion is detected by applying Gabor motion detectors to each local spatiotemporal pattern depicted by each local feature value. Outputs of all the detectors are integrated to give the final output for global motion of the input stimulus. The model was simulated on a computer and was confirmed to correctly detect second-order motion as well as first-order motion.     

Stabilized structure from motion without disparity induces disparity adaptation

3D structures can be perceived based on the patterns of 2D motion signals. With orthographic projection of a 3D stimulus onto a 2D plane, the kinetic information can give a vivid impression of depth, but the depth order is intrinsically ambiguous, resulting in bistable or even multistable interpretations. For example, an orthographic projection of dots on the surface of a rotating cylinder is perceived as a rotating cylinder with ambiguous direction of rotation. We show that the bistable rotation can be stabilized by adding information, not to the dots themselves, but to their spatial context. More interestingly, the stabilized bistable motion can generate consistent rotation aftereffects. The rotation aftereffect can only be observed when the adapting and test stimuli are presented at the same stereo depth and the same retinal location, and it is not due to attentional tracking. The observed rotation aftereffect is likely due to direction-contingent disparity adaptation, implying that stimuli with kinetic depth may have activated neurons sensitive to different disparities, even though the stimuli have zero relative disparity. Stereo depth and kinetic depth may be supported by a common neural mechanism at an early stage in the visual system.     

Adaptability of the receptive fields of neurons of the posterior temporal cortex and their sensitivity to parameters of photic stimulation in the cat

Study of receptive fields (RFs) of neurones in the postero-temporal cortex (field 21) of alert cat at three levels of visual adaptation: light photopic, light mesopic and practically dark or extremely low scotopic adaptations--revealed invariance of the most part of the studied RFs to the level of visual adaptation. Reorganization of RFs, connected with change of background luminosity were observed only in 12% of visually activated neurones. Significant reduction of responses to optic stimulation is shown at increase of the level of luminosity in 75% of neurones, revealing adaptive reorganizations. It is suggested that these reorganizations may take place in analogy with neurones of the field 17 on account of different involvement of intracortical inhibitory mechanisms (and, probably, not only in the postero-temporal cortex, but also in structures which precede it in visual hierarchy). Study of neurones sensitivity in the field 21 to parameters of optic stimulation revealed their considerable invariance to the length and orientation of the optic stimulus moving through the RF (60% of cases). Testing of RF by a rhombic optic stimulus did not change neuronal reactions, the form and dimensions of RF did not significantly change.