A model for the detection of moving targets in visual clutter inspired by insect physiology

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as 'small target motion detectors' (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1(st) order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter' to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.     

Insect detection of small targets moving in visual clutter

Detection of targets that move within visual clutter is a common task for animals searching for prey or conspecifics, a task made even more difficult when a moving pursuer needs to analyze targets against the motion of background texture (clutter). Despite the limited optical acuity of the compound eye of insects, this challenging task seems to have been solved by their tiny visual system. Here we describe neurons found in the male hoverfly,Eristalis tenax, that respond selectively to small moving targets. Although many of these target neurons are inhibited by the motion of a background pattern, others respond to target motion within the receptive field under a surprisingly large range of background motion stimuli. Some neurons respond whether or not there is a speed differential between target and background. Analysis of responses to very small targets (smaller than the size of the visual field of single photoreceptors) or those targets with reduced contrast shows that these neurons have extraordinarily high contrast sensitivity. Our data suggest that rejection of background motion may result from extreme selectivity for small targets contrasting against local patches of the background, combined with this high sensitivity, such that background patterns rarely contain features that satisfactorily drive the neuron.     

Retinotopic organization of small-field-target-detecting neurons in the insect visual system

BACKGROUND: Despite having tiny brains and relatively low-resolution compound eyes, many fly species frequently engage in precisely controlled aerobatic pursuits of conspecifics. Recent investigations into high-order processing in the fly visual system have revealed a class of neurons, coined small-target-motion detectors (STMDs), capable of responding robustly to target motion against the motion of background clutter. Despite limited spatial acuity in the insect eye, these neurons display exquisite sensitivity to small targets. RESULTS: We recorded intracellularly from morphologically identified columnar neurons in the lobula complex of the hoverfly Eristalis tenax. We show that these columnar neurons with exquisitely small receptive fields, like their large-field counterparts recently described from both male and female flies, have an extreme selectivity for the motion of small targets. In doing so, we provide the first physiological characterization of small-field neurons in female flies. These retinotopically organized columnar neurons include both direction-selective and nondirection-selective classes covering a large area of visual space. CONCLUSIONS: The retinotopic arrangement of lobula columnar neurons sensitive to the motion of small targets makes a strong case for these neurons as important precursors in the local processing of target motion. Furthermore, the continued response of STMDs with such small receptive fields to the motion of small targets in the presence of moving background clutter places further constraints on the potential mechanisms underlying their small-target tuning.     

Activity of descending contralateral movement detector neurons and collision avoidance behaviour in response to head-on visual stimuli in locusts

We recorded the activity of the right and left descending contralateral movement detectors responding to 10-cm (small) or 20-cm (large) computer-generated spheres approaching along different trajectories in the locust's frontal field of view. In separate experiments we examined the steering responses of tethered flying locusts to identical stimuli. The descending contralateral movement detectors were more sensitive to variations in target trajectory in the horizontal plane than in the vertical plane. Descending contralateral movement detector activity was related to target trajectory and to target size and was most sensitive to small objects converging on a direct collision course from above and to one side. Small objects failed to induce collision avoidance manoeuvres whereas large objects produced reliable collision avoidance responses. Large targets approaching along a converging trajectory produced steering responses that were either away from or toward the side of approach of the object, whereas targets approaching along trajectories that were offset from the locust's mid-longitudinal body axis primarily evoked responses away from the target. We detected no differences in the discharge properties of the descending contralateral movement detector pair that could account for the different collision avoidance behaviours evoked by varying the target size and trajectories. We suggest that descending contralateral movement detector properties are better suited to predator evasion than collision avoidance.     

Neural specializations for small target detection in insects

Despite being equipped with low-resolution eyes and tiny brains, many insects show exquisite abilities to detect and pursue targets even in highly textured surrounds. Target tracking behavior is subserved by neurons that are sharply tuned to the motion of small high-contrast targets. These neurons respond robustly to target motion, even against self-generated optic flow. A recent model, supported by neurophysiology, generates target selectivity by being sharply tuned to the unique spatiotemporal profile associated with target motion. Target neurons are likely connected in a complex network where some provide more direct output to behavior, whereas others serve an inter-regulatory role. These interactions may regulate attention and aid in the robust detection of targets in clutter observed in behavior.     

Echolocating bats use a nearly time-optimal strategy to intercept prey

Acquisition of food in many animal species depends on the pursuit and capture of moving prey. Among modern humans, the pursuit and interception of moving targets plays a central role in a variety of sports, such as tennis, football, Frisbee, and baseball. Studies of target pursuit in animals, ranging from dragonflies to fish and dogs to humans, have suggested that they all use a constant bearing (CB) strategy to pursue prey or other moving targets. CB is best known as the interception strategy employed by baseball outfielders to catch ballistic fly balls. CB is a time-optimal solution to catch targets moving along a straight line, or in a predictable fashion--such as a ballistic baseball, or a piece of food sinking in water. Many animals, however, have to capture prey that may make evasive and unpredictable maneuvers. Is CB an optimum solution to pursuing erratically moving targets? Do animals faced with such erratic prey also use CB? In this paper, we address these questions by studying prey capture in an insectivorous echolocating bat. Echolocating bats rely on sonar to pursue and capture flying insects. The bat's prey may emerge from foliage for a brief time, fly in erratic three-dimensional paths before returning to cover. Bats typically take less than one second to detect, localize and capture such insects. We used high speed stereo infra-red videography to study the three dimensional flight paths of the big brown bat, Eptesicus fuscus, as it chased erratically moving insects in a dark laboratory flight room. We quantified the bat's complex pursuit trajectories using a simple delay differential equation. Our analysis of the pursuit trajectories suggests that bats use a constant absolute target direction strategy during pursuit. We show mathematically that, unlike CB, this approach minimizes the time it takes for a pursuer to intercept an unpredictably moving target. Interestingly, the bat's behavior is similar to the interception strategy implemented in some guided missiles. We suggest that the time-optimal strategy adopted by the bat is in response to the evolutionary pressures of having to capture erratic and fast moving insects.     

Novel tracking function of moving target using chaotic dynamics in a recurrent neural network model

Chaotic dynamics introduced in a recurrent neural network model is applied to controlling an object to track a moving target in two-dimensional space, which is set as an ill-posed problem. The motion increments of the object are determined by a group of motion functions calculated in real time with firing states of the neurons in the network. Several cyclic memory attractors that correspond to several simple motions of the object in two-dimensional space are embedded. Chaotic dynamics introduced in the network causes corresponding complex motions of the object in two-dimensional space. Adaptively real-time switching of control parameter results in constrained chaos (chaotic itinerancy) in the state space of the network and enables the object to track a moving target along a certain trajectory successfully. The performance of tracking is evaluated by calculating the success rate over 100 trials with respect to nine kinds of trajectories along which the target moves respectively. Computer experiments show that chaotic dynamics is useful to track a moving target. To understand the relations between these cases and chaotic dynamics, dynamical structure of chaotic dynamics is investigated from dynamical viewpoint.     

Response properties of visual interneurons to motion stimuli in the praying mantis,Tenodera aridifolia

Intracellular responses of motion-sensitive visual interneurons were recorded from the lobula complex of the mantis, Tenodera aridifolia. The interneurons were divided into four classes according to the response polarity, spatial tuning, and directional selectivity. Neurons of the first class had small, medium, or large receptive fields and showed a strong excitation in response to a small-field motion such as a small square moving in any direction (SF neurons). The second class neurons showed non-directionally selective responses: an excitation to a large-field motion of gratings in any direction (ND neurons). Most ND neurons had small or medium-size receptive fields. Neurons of the third class had large receptive fields and exhibited directionally selective responses: an excitation to a large-field motion of gratings in preferred direction and an inhibition to a motion in opposite, null direction (DS neurons). The last class neurons had small receptive fields and showed inhibitory responses to a moving square and gratings (I neurons). The functional roles of these neurons in prey recognition and optomotor response were discussed.     

Prediction of a moving target's position in fast goal-directed action

 Subjects made fast goal-directed arm movements towards moving targets. In some cases, the perceived direction of target motion was manipulated by moving the background. By comparing the trajectories towards moving targets with those towards static targets, we determined the position towards which subjects were aiming at movement onset. We showed that this position was an extrapolation in the target’s perceived direction from its position at that moment using its perceived direction of motion. If subjects were to continue to extrapolate in the perceived direction of target motion from the position at which they perceive the target at each instant, the error would decrease during the movements. By analysing the differences between subjects’ arm movements towards targets moving in different (apparent) directions with a linear second-order model, we show that the reduction in the error that this predicts is not enough to explain how subjects compensate for their initial misjudgements. Received: 10 February 1995/Accepted in revised form: 30 May 1995     

Segregation of object and background motion in visual area MT: effects of microstimulation on eye movements

To track a moving object, its motion must first be distinguished from that of the background. The center-surround properties of neurons in the middle temporal visual area (MT) may be important for signaling the relative motion between object and background. To test this, we microstimulated within MT and measured the effects on monkeys' eye movements to moving targets. We found that stimulation at "local motion" sites, where receptive fields possessed antagonistic surrounds, shifted pursuit in the preferred direction of the neurons, whereas stimulation at "wide-field motion" sites shifted pursuit in the opposite, or null, direction. We propose that activating wide-field sites simulated background motion, thus inducing a target motion signal in the opposite direction. Our results support the hypothesis that neuronal center-surround mechanisms contribute to the behavioral segregation of objects from the background.     

How Moving Backgrounds Influence Interception

Reaching movements towards an object are continuously guided by visual information about the target and the arm. Such guidance increases precision and allows one to adjust the movement if the target unexpectedly moves. On-going arm movements are also influenced by motion in the surrounding. Fast responses to motion in the surrounding could help cope with moving obstacles and with the consequences of changes in one’s eye orientation and vantage point. To further evaluate how motion in the surrounding influences interceptive movements we asked subjects to tap a moving target when it reached a second, static target. We varied the direction and location of motion in the surrounding, as well as details of the stimuli that are known to influence eye movements. Subjects were most sensitive to motion in the background when such motion was near the targets. Whether or not the eyes were moving, and the direction of the background motion in relation to the direction in which the eyes were moving, had very little influence on the response to the background motion. We conclude that the responses to background motion are driven by motion near the target rather than by a global analysis of the optic flow and its relation with other information about self-motion.     

Distance in motion: response trajectories reveal the dynamics of number comparison

Cognitive and neuroscientific evidence has challenged the widespread view that perception, cognition and action constitute independent, discrete stages. For example, in continuous response trajectories toward a target response location, evidence suggests that a decision on which target to reach for (i.e., the cognition stage) is not reached before the movement starts (i.e., the action stage). As a result, instead of a straight trajectory to the correct target response, movement trajectories may curve toward competing responses or away from inhibited responses. In the present study, we examined response trajectories during a number comparison task. Participants had to decide whether a target number was smaller or larger than 5. They had to respond by moving to a left or a right response location. Replicating previous results, response trajectories were more curved toward the incorrect response location when distance to 5 was small (e.g., target number 4) than when distance to 5 was large (e.g., target number 1). Importantly, we manipulated the response mapping, which allowed us to demonstrate that this response trajectory effect results from the relative amount of evidence for the available responses across time. In this way, the present study stresses the tight coupling of number representations (i.e., cognition) and response related processes (i.e., action) and shows that these stages are not separable in time.     

Visual motion: homing in on small target detectors

Tracking moving targets is essential for animals that pursue prey or conspecifics. Recent studies in male and female hoverflies have described classes of neurons that detect the movements of small targets against a moving background but the mechanisms generating their responses remain unclear.     

Tuning Properties of MT and MSTd and Divisive Interactions for Eye-Movement Compensation

The primate brain intelligently processes visual information from the world as the eyes move constantly. The brain must take into account visual motion induced by eye movements, so that visual information about the outside world can be recovered. Certain neurons in the dorsal part of monkey medial superior temporal area (MSTd) play an important role in integrating information about eye movements and visual motion. When a monkey tracks a moving target with its eyes, these neurons respond to visual motion as well as to smooth pursuit eye movements. Furthermore, the responses of some MSTd neurons to the motion of objects in the world are very similar during pursuit and during fixation, even though the visual information on the retina is altered by the pursuit eye movement. We call these neurons compensatory pursuit neurons. In this study we develop a computational model of MSTd compensatory pursuit neurons based on physiological data from single unit studies. Our model MSTd neurons can simulate the velocity tuning of monkey MSTd neurons. The model MSTd neurons also show the pursuit compensation property. We find that pursuit compensation can be achieved by divisive interaction between signals coding eye movements and signals coding visual motion. The model generates two implications that can be tested in future experiments: (1) compensatory pursuit neurons in MSTd should have the same direction preference for pursuit and retinal visual motion; (2) there should be non-compensatory pursuit neurons that show opposite preferred directions of pursuit and retinal visual motion.     

Smooth Pursuit Eye Movements Improve Temporal Resolution for Color Perception

Human observers see a single mixed color (yellow) when different colors (red and green) rapidly alternate. Accumulating evidence suggests that the critical temporal frequency beyond which chromatic fusion occurs does not simply reflect the temporal limit of peripheral encoding. However, it remains poorly understood how the central processing controls the fusion frequency. Here we show that the fusion frequency can be elevated by extra-retinal signals during smooth pursuit. This eye movement can keep the image of a moving target in the fovea, but it also introduces a backward retinal sweep of the stationary background pattern. We found that the fusion frequency was higher when retinal color changes were generated by pursuit-induced background motions than when the same retinal color changes were generated by object motions during eye fixation. This temporal improvement cannot be ascribed to a general increase in contrast gain of specific neural mechanisms during pursuit, since the improvement was not observed with a pattern flickering without changing position on the retina or with a pattern moving in the direction opposite to the background motion during pursuit. Our findings indicate that chromatic fusion is controlled by a cortical mechanism that suppresses motion blur. A plausible mechanism is that eye-movement signals change spatiotemporal trajectories along which color signals are integrated so as to reduce chromatic integration at the same locations (i.e., along stationary trajectories) on the retina that normally causes retinal blur during fixation.     

Visual learning in multiple-object tracking

Background

Tracking moving objects in space is important for the maintenance of spatiotemporal continuity in everyday visual tasks. In the laboratory, this ability is tested using the Multiple Object Tracking (MOT) task, where participants track a subset of moving objects with attention over an extended period of time. The ability to track multiple objects with attention is severely limited. Recent research has shown that this ability may improve with extensive practice (e.g., from action videogame playing). However, whether tracking also improves in a short training session with repeated trajectories has rarely been investigated. In this study we examine the role of visual learning in multiple-object tracking and characterize how varieties of attention interact with visual learning.

Methodology/Principal Findings

Participants first conducted attentive tracking on trials with repeated motion trajectories for a short session. In a transfer phase we used the same motion trajectories but changed the role of tracking targets and nontargets. We found that compared with novel trials, tracking was enhanced only when the target subset was the same as that used during training. Learning did not transfer when the previously trained targets and nontargets switched roles or mixed up. However, learning was not specific to the trained temporal order as it transferred to trials where the motion was played backwards.

Conclusions/Significance

These findings suggest that a demanding task of tracking multiple objects can benefit from learning of repeated motion trajectories. Such learning potentially facilitates tracking in natural vision, although learning is largely confined to the trajectories of attended objects. Furthermore, we showed that learning in attentive tracking relies on relational coding of all target trajectories. Surprisingly, learning was not specific to the trained temporal context, probably because observers have learned motion paths of each trajectory independently of the exact temporal order.     

综述: 初级视皮层的亮度信息加工

亮度(luminance)是最基本的视觉信息.与其他视觉特征相比,由于视神经元对亮度刺激的反应较弱,并且许多神经元对均匀亮度无反应,对亮度信息编码的神经机制知之甚少.初级视皮层部分神经元对亮度的反应要慢于对比度反应,被认为是由边界对比度诱导的亮度知觉(brightness)的神经基础.我们的研究表明,初级视皮层许多神经元的亮度反应要快于对比度反应,并且这些神经元偏好低的空间频率、高的时间频率和高的运动速度,提示皮层下具有低空间频率和高运动速度通路的信息输入对产生初级视皮层神经元的亮度反应有贡献.已经知道初级视皮层神经元对空间频率反应的时间过程是从低空间频率到高空间频率,我们发现的早期亮度反应是对极低空间频率的反应,与这一时间过程是一致的,是这一从粗到细的视觉信息加工过程的第一步,揭示了处理最早的粗的视觉信息的神经基础.另外,初级视皮层含有偏好亮度下降和高运动速度的神经元,这群神经元的活动有助于在光照差的环境中检测高速运动的低亮度物体.     

Sensory and behavioral properties of neurons in posterior parietal cortex of the awake, trained monkey.

Neurons in posterior parietal cortex of the awake, trained monkey respond to passive visual and/or somatosensory stimuli. In general, the receptive fields of these cells are large and nonspecific. When these neurons are studied during visually guided hand movements and eye movements, most of their activity can be accounted for by passive sensory stimulation. However, for some visual cells, the response to a stimulus is enhanced when it is to be the target for a saccadic eye movement. This enhancement is selective for eye movements into the visual receptive field since it does not occur with eye movements to other parts of the visual field. Cells that discharge in association with a visual fixation task have foveal receptive fields and respond to the spots of light used as fixation targets. Cells discharging selectively in association with different directions of tracking eye movements have directionally selective responses to moving visual stimuli. Every cell in our sample discharging in association with movement could be driven by passive sensory stimuli. We conclude that the activity of neurons in posterior parietal cortex is dependent on and indicative of external stimuli but not predictive of movement.     

Bursts and isolated spikes code for opposite movement directions in midbrain electrosensory neurons

Directional selectivity, in which neurons respond strongly to an object moving in a given direction but weakly or not at all to the same object moving in the opposite direction, is a crucial computation that is thought to provide a neural correlate of motion perception. However, directional selectivity has been traditionally quantified by using the full spike train, which does not take into account particular action potential patterns. We investigated how different action potential patterns, namely bursts (i.e. packets of action potentials followed by quiescence) and isolated spikes, contribute to movement direction coding in a mathematical model of midbrain electrosensory neurons. We found that bursts and isolated spikes could be selectively elicited when the same object moved in opposite directions. In particular, it was possible to find parameter values for which our model neuron did not display directional selectivity when the full spike train was considered but displayed strong directional selectivity when bursts or isolated spikes were instead considered. Further analysis of our model revealed that an intrinsic burst mechanism based on subthreshold T-type calcium channels was not required to observe parameter regimes for which bursts and isolated spikes code for opposite movement directions. However, this burst mechanism enhanced the range of parameter values for which such regimes were observed. Experimental recordings from midbrain neurons confirmed our modeling prediction that bursts and isolated spikes can indeed code for opposite movement directions. Finally, we quantified the performance of a plausible neural circuit and found that it could respond more or less selectively to isolated spikes for a wide range of parameter values when compared with an interspike interval threshold. Our results thus show for the first time that different action potential patterns can differentially encode movement and that traditional measures of directional selectivity need to be revised in such cases.     

An elaborated model of fly small-target tracking

Flies have the capability to visually track small moving targets, even across cluttered backgrounds. Previous computational models, based on figure detection (FD) cells identified in the fly, have suggested how this may be accomplished at a neuronal level based on information about relative motion between the target and the background. We experimented with the use of this small-field system model for the tracking of small moving targets by a simulated fly in a cluttered environment and discovered some functional limitations. As a result of these experiments, we propose elaborations of the original small-field system model to support stronger effects of background motion on small-field responses, proper accounting for more complex optical flow fields, and more direct guidance toward the target. We show that the elaborated model achieves much better tracking performance than the original model in complex visual environments and discuss the biological implications of our elaborations. The elaborated model may help to explain recent electrophysiological data on FD cells that seem to contradict the original model.Acknowledgement This work was supported by the US Office of Naval Research under agreement number N68936-00-2-0002.