Harbor Seals (Phoca vitulina) Can Perceive Optic Flow under Water

Optic flow, the pattern of apparent motion elicited on the retina during movement, has been demonstrated to be widely used by animals living in the aerial habitat, whereas underwater optic flow has not been intensively studied so far. However optic flow would also provide aquatic animals with valuable information about their own movement relative to the environment; even under conditions in which vision is generally thought to be drastically impaired, e. g. in turbid waters. Here, we tested underwater optic flow perception for the first time in a semi-aquatic mammal, the harbor seal, by simulating a forward movement on a straight path through a cloud of dots on an underwater projection. The translatory motion pattern expanded radially out of a singular point along the direction of heading, the focus of expansion. We assessed the seal''s accuracy in determining the simulated heading in a task, in which the seal had to judge whether a cross superimposed on the flow field was deviating from or congruent with the actual focus of expansion. The seal perceived optic flow and determined deviations from the simulated heading with a threshold of 0.6 deg of visual angle. Optic flow is thus a source of information seals, fish and most likely aquatic species in general may rely on for e. g. controlling locomotion and orientation under water. This leads to the notion that optic flow seems to be a tool universally used by any moving organism possessing eyes.     

Competitive Dynamics in MSTd: A Mechanism for Robust Heading Perception Based on Optic Flow

Human heading perception based on optic flow is not only accurate, it is also remarkably robust and stable. These qualities are especially apparent when observers move through environments containing other moving objects, which introduce optic flow that is inconsistent with observer self-motion and therefore uninformative about heading direction. Moving objects may also occupy large portions of the visual field and occlude regions of the background optic flow that are most informative about heading perception. The fact that heading perception is biased by no more than a few degrees under such conditions attests to the robustness of the visual system and warrants further investigation. The aim of the present study was to investigate whether recurrent, competitive dynamics among MSTd neurons that serve to reduce uncertainty about heading over time offer a plausible mechanism for capturing the robustness of human heading perception. Simulations of existing heading models that do not contain competitive dynamics yield heading estimates that are far more erratic and unstable than human judgments. We present a dynamical model of primate visual areas V1, MT, and MSTd based on that of Layton, Mingolla, and Browning that is similar to the other models, except that the model includes recurrent interactions among model MSTd neurons. Competitive dynamics stabilize the model’s heading estimate over time, even when a moving object crosses the future path. Soft winner-take-all dynamics enhance units that code a heading direction consistent with the time history and suppress responses to transient changes to the optic flow field. Our findings support recurrent competitive temporal dynamics as a crucial mechanism underlying the robustness and stability of perception of heading.     

A Bio-inspired Collision Avoidance Model Based on Spatial Information Derived from Motion Detectors Leads to Common Routes

Avoiding collisions is one of the most basic needs of any mobile agent, both biological and technical, when searching around or aiming toward a goal. We propose a model of collision avoidance inspired by behavioral experiments on insects and by properties of optic flow on a spherical eye experienced during translation, and test the interaction of this model with goal-driven behavior. Insects, such as flies and bees, actively separate the rotational and translational optic flow components via behavior, i.e. by employing a saccadic strategy of flight and gaze control. Optic flow experienced during translation, i.e. during intersaccadic phases, contains information on the depth-structure of the environment, but this information is entangled with that on self-motion. Here, we propose a simple model to extract the depth structure from translational optic flow by using local properties of a spherical eye. On this basis, a motion direction of the agent is computed that ensures collision avoidance. Flying insects are thought to measure optic flow by correlation-type elementary motion detectors. Their responses depend, in addition to velocity, on the texture and contrast of objects and, thus, do not measure the velocity of objects veridically. Therefore, we initially used geometrically determined optic flow as input to a collision avoidance algorithm to show that depth information inferred from optic flow is sufficient to account for collision avoidance under closed-loop conditions. Then, the collision avoidance algorithm was tested with bio-inspired correlation-type elementary motion detectors in its input. Even then, the algorithm led successfully to collision avoidance and, in addition, replicated the characteristics of collision avoidance behavior of insects. Finally, the collision avoidance algorithm was combined with a goal direction and tested in cluttered environments. The simulated agent then showed goal-directed behavior reminiscent of components of the navigation behavior of insects.     

Retinal optic flow during natural locomotion

We examine the structure of the visual motion projected on the retina during natural locomotion in real world environments. Bipedal gait generates a complex, rhythmic pattern of head translation and rotation in space, so without gaze stabilization mechanisms such as the vestibular-ocular-reflex (VOR) a walker’s visually specified heading would vary dramatically throughout the gait cycle. The act of fixation on stable points in the environment nulls image motion at the fovea, resulting in stable patterns of outflow on the retinae centered on the point of fixation. These outflowing patterns retain a higher order structure that is informative about the stabilized trajectory of the eye through space. We measure this structure by applying the curl and divergence operations on the retinal flow velocity vector fields and found features that may be valuable for the control of locomotion. In particular, the sign and magnitude of foveal curl in retinal flow specifies the body’s trajectory relative to the gaze point, while the point of maximum divergence in the retinal flow field specifies the walker’s instantaneous overground velocity/momentum vector in retinotopic coordinates. Assuming that walkers can determine the body position relative to gaze direction, these time-varying retinotopic cues for the body’s momentum could provide a visual control signal for locomotion over complex terrain. In contrast, the temporal variation of the eye-movement-free, head-centered flow fields is large enough to be problematic for use in steering towards a goal. Consideration of optic flow in the context of real-world locomotion therefore suggests a re-evaluation of the role of optic flow in the control of action during natural behavior.     

Neuronal representation of optic flow experienced by unilaterally blinded flies on their mean walking trajectories

Asymmetries in the optic flow on both eyes may indicate an unintended turn of an animal and evoke compensatory optomotor responses. On a straight path in an evenly structured environment, the optic flow on both eyes is balanced corresponding to a state of optomotor equilibrium. When one eye is occluded an optomotor equilibrium is expected to be reached on a curved path provided that the translatory optic flow component is cancelled by a superimposed rotation. This hypothesis is tested by analysing how the HSE cell, a constituent element of the fly's optomotor system, represents optic flow in behavioural situations. The optic flow as seen on the average trajectory of freely walking monocular flies is reconstructed. This optic flow is used as stimulus of the HSE cell in electrophysiological experiments and as input of a model of the fly's optomotor system. The responses of the HSE cell and of the model fluctuate around the resting potential. On average, they are much smaller than the responses evoked by optic flow experienced on a straight path. These results corroborate the hypothesis that the mean trajectory of monocular flies corresponds to a path of optomotor equilibrium. Accepted: 29 February 2000     

Motion anisotropies and heading detection

 In motion-processing areas of the visual cortex in cats and monkeys, an anisotropic distribution of direction selectivities displays a preference for movements away from the fovea. This ‘centrifugal bias’ has been hypothetically linked to the processing of optic flow fields generated during forward locomotion. In this paper, we show that flow fields induced on the retina in many natural situations of locomotion of higher mammals are indeed qualitatively centrifugal in structure, even when biologically plausible eye movements to stabilize gaze on environmental targets are performed. We propose a network model of heading detection that carries an anisotropy similar to the one found in cat and monkey. In simulations, this model reproduces a number of psychophysical results of human heading detection. It suggests that a recently reported human disability to correctly identify the direction of heading from optic flow when a certain type of eye movement is simulated might be linked to the noncentrifugal structure of the resulting retinal flow field and to the neurophysiological anisotropies. Received: 1 April 1994/Accepted in revised form: 4 August 1994     

A Unified Model of Heading and Path Perception in Primate MSTd

Self-motion, steering, and obstacle avoidance during navigation in the real world require humans to travel along curved paths. Many perceptual models have been proposed that focus on heading, which specifies the direction of travel along straight paths, but not on path curvature, which humans accurately perceive and is critical to everyday locomotion. In primates, including humans, dorsal medial superior temporal area (MSTd) has been implicated in heading perception. However, the majority of MSTd neurons respond optimally to spiral patterns, rather than to the radial expansion patterns associated with heading. No existing theory of curved path perception explains the neural mechanisms by which humans accurately assess path and no functional role for spiral-tuned cells has yet been proposed. Here we present a computational model that demonstrates how the continuum of observed cells (radial to circular) in MSTd can simultaneously code curvature and heading across the neural population. Curvature is encoded through the spirality of the most active cell, and heading is encoded through the visuotopic location of the center of the most active cell''s receptive field. Model curvature and heading errors fit those made by humans. Our model challenges the view that the function of MSTd is heading estimation, based on our analysis we claim that it is primarily concerned with trajectory estimation and the simultaneous representation of both curvature and heading. In our model, temporal dynamics afford time-history in the neural representation of optic flow, which may modulate its structure. This has far-reaching implications for the interpretation of studies that assume that optic flow is, and should be, represented as an instantaneous vector field. Our results suggest that spiral motion patterns that emerge in spatio-temporal optic flow are essential for guiding self-motion along complex trajectories, and that cells in MSTd are specifically tuned to extract complex trajectory estimation from flow.     

Gaze Strategy in the Free Flying Zebra Finch (Taeniopygia guttata)

Fast moving animals depend on cues derived from the optic flow on their retina. Optic flow from translational locomotion includes information about the three-dimensional composition of the environment, while optic flow experienced during a rotational self motion does not. Thus, a saccadic gaze strategy that segregates rotations from translational movements during locomotion will facilitate extraction of spatial information from the visual input. We analysed whether birds use such a strategy by highspeed video recording zebra finches from two directions during an obstacle avoidance task. Each frame of the recording was examined to derive position and orientation of the beak in three-dimensional space. The data show that in all flights the head orientation was shifted in a saccadic fashion and was kept straight between saccades. Therefore, birds use a gaze strategy that actively stabilizes their gaze during translation to simplify optic flow based navigation. This is the first evidence of birds actively optimizing optic flow during flight.     

Optic flow-based collision-free strategies: From insects to robots

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects’ abilities and better understanding their flight.     

A simple strategy for detecting moving objects during locomotion revealed by animal-robot interactions

An important role of visual systems is to detect nearby predators, prey, and potential mates, which may be distinguished in part by their motion. When an animal is at rest, an object moving in any direction may easily be detected by motion-sensitive visual circuits. During locomotion, however, this strategy is compromised because the observer must detect a moving object within the pattern of optic flow created by its own motion through the stationary background. However, objects that move creating back-to-front (regressive) motion may be unambiguously distinguished from stationary objects because forward locomotion creates only front-to-back (progressive) optic flow. Thus, moving animals should exhibit an enhanced sensitivity to regressively moving objects. We explicitly tested this hypothesis by constructing a simple fly-sized robot that was programmed to interact with a real fly. Our measurements indicate that whereas walking female flies freeze in response to a regressively moving object, they ignore a progressively moving one. Regressive motion salience also explains observations of behaviors exhibited by pairs of walking flies. Because the assumptions underlying the regressive motion salience hypothesis are general, we suspect that the behavior we have observed in Drosophila may be widespread among eyed, motile organisms.     

Parietal area VIP neuronal responses to heading stimuli are encoded in head-centered coordinates

The ventral intraparietal area (VIP) is a multimodal parietal area, where visual responses are brisk, directional, and typically selective for complex optic flow patterns. VIP thus could provide signals useful for visual estimation of heading (self-motion direction). A central problem in heading estimation is how observers compensate for eye velocity, which distorts the retinal motion cues upon which perception depends. To find out if VIP could be useful for heading, we measured its responses to simulated trajectories, both with and without eye movements. Our results showed that most VIP neurons very strongly signal heading direction. Furthermore, the tuning of most VIP neurons was remarkably stable in the presence of eye movements. This stability was such that the population of VIP neurons represented heading very nearly in head-centered coordinates. This makes VIP the most robust source of such signals yet described, with properties ideal for supporting perception.     

Retinal scanning and visual inputs in freely walking crickets

ABSTRACT. Freely walking crickets were filmed from above during their visual orientation towards a black stripe. A frame-by-frame analysis enabled head and body movements to be recorded. The animals walk in 200ms bouts (runs) separated by pauses of similar duration. During each run, rotations of the body axis are observed and some corrections of the course direction occur between successive runs. Generally, the crickets do not walk straight ahead but slightly sideways. Because no lateral head movements were observed during visually orientated locomotion, retinal scanning results from both rotations of the body axis and translation of the head. While walking, one of the target edges is maintained by the cricket on a relatively limited area of the retina, generally between 10 and 25 laterally. Thus, the cricket often records three pieces of information about each edge: one in the monocular visual field, and two in the binocular visual field. Nevertheless, between two pauses, the images of each edge shift asymmetrically on the retinae. Such movement could prevent receptor adaptation by modulation of the ommatidial excitation, or by stimulation of the neighbouring ommatidia. It is also suggested that antennal movements are influenced by the positions of the visually fixated target edges.     

Angle of gaze and optic flow direction modulate body sway

Optic flow is a crucial signal in maintaining postural stability. We sought to investigate whether the activity of postural muscles and body sway was modulated by eye position during the view of radial optic flow stimuli. We manipulated the spatial distribution of dot speed and the fixation point position to simulate specific heading directions combined with different gaze positions. The experiments were performed using stabilometry and surface electromyography (EMG) on 24 right-handed young, healthy volunteers. Center of pressure (COP) signals were analyzed considering antero-posterior and medio-lateral oscillation, COP speed, COP area, and the prevalent direction of oscillation of body sway. We found a significant main effect of body side in all COP parameters, with the right body side showing greater oscillations. The different combinations of optic flow and eye position evoked a non-uniform direction of oscillations in females. The EMG analysis showed a significant main effect for muscle and body side. The results showed that the eye position modulated body sway without changing the activity of principal leg postural muscles, suggesting that the extraretinal input regarding the eye position is a crucial signal that needs to be integrated with perceptual optic flow processing in order to control body sway.     

Visual Neuroscience: the brain's interest in natural flow

Optic flow is a key signal for heading perception. A new study has shown that the human brain can dissociate between consistent (natural) and inconsistent flow, revealing what is likely a new hierarchy in visual motion processing.     

Modeling boundary vector cell firing given optic flow as a cue

Boundary vector cells in entorhinal cortex fire when a rat is in locations at a specific distance from walls of an environment. This firing may originate from memory of the barrier location combined with path integration, or the firing may depend upon the apparent visual input image stream. The modeling work presented here investigates the role of optic flow, the apparent change of patterns of light on the retina, as input for boundary vector cell firing. Analytical spherical flow is used by a template model to segment walls from the ground, to estimate self-motion and the distance and allocentric direction of walls, and to detect drop-offs. Distance estimates of walls in an empty circular or rectangular box have a mean error of less than or equal to two centimeters. Integrating these estimates into a visually driven boundary vector cell model leads to the firing patterns characteristic for boundary vector cells. This suggests that optic flow can influence the firing of boundary vector cells.     

Computing the direction of heading from affine image flow

Observers moving through a three-dimensional environment can use optic flow to determine their direction of heading. Existing heading algorithms use cartesian flow fields in which image flow is the displacement of image features over time. I explore a heading algorithm that uses affine flow instead. The affine flow at an image feature is its displacement modulo an affine transformation defined by its neighborhood. Modeling the observer's instantaneous motion by a translation and a rotation about an axis through its eye, affine flow is tangent to the translational field lines on the observer's viewing sphere. These field lines form a radial flow field whose center is the direction of heading. The affine flow heading algorithm has characteristics that can be used to determine whether the human visual system relies on it. The algorithm is immune to observer rotation and arbitrary affine transformations of its input images; its accuracy improves with increasing variation in environmental depth; and it cannot recover heading in an environment consisting of a single plane because affine flow vanishes in this case. Translational field lines can also be approximated through differential cartesian motion. I compare the performance of heading algorithms based on affine flow, differential cartesian flow, and least-squares search.     

Treadmill experience alters treadmill effects on perceived visual motion

Information on ongoing body movements can affect the perception of ambiguous visual motion. Previous studies on "treadmill capture" have shown that treadmill walking biases the perception of ambiguous apparent motion in backward direction in accordance with the optic flow during normal walking, and that long-term treadmill experience changes the effect of treadmill capture. To understand the underlying mechanisms for these phenomena, we conducted Experiment 1 with non-treadmill runners and Experiment 2 with treadmill runners. The participants judged the motion direction of the apparent motion stimuli of horizontal gratings in front of their feet under three conditions: walking on a treadmill, standing on a treadmill, and standing on the floor. The non-treadmill runners showed the presence of downward bias only under the walking condition, indicating that ongoing treadmill walking but not the awareness of being on a treadmill biased the visual directional discrimination. In contrast, the treadmill runners showed no downward bias under any of the conditions, indicating that neither ongoing activity nor the awareness of spatial context produced perception bias. This suggests that the long-term repetitive experience of treadmill walking without optic flow induced the formation of a treadmill-specific locomotor-visual linkage to perceive the complex relationship between self and the environment.     

Visual, auditory, and bimodal activity in the banks of the lateral suprasylvian sulcus in the cat

In addition to visually driven cells we found within the lateral suprasylvian visual cortex of cats a considerable number of auditory and/or bimodal cells. Most of the visually driven cells were direction and orientation selective with responses that were neither highly stimulus time locked nor very stable. Most of the auditory responses were also not very stable, had relatively high thresholds and were readily habituated. Previous studies have suggested that populations of cells within the lateral suprasylvian area are specialized for the analysis of optic flow fields. Given that a remarkable proportion of cells within this area can be also driven by auditory stimuli we hypothesize that the "optic flow" model may be extended to the bimodal domain rather than restricted to visual clues only. This, however, remains to be corroborated experimentally.     

Optimizing visual motion perception during eye movements.

We usually perceive a stationary, stable world and we are able to correctly estimate the direction of heading from optic flow despite coherent visual motion induced by eye movements. This astonishing example of perceptual invariance results from a comparison of visual information with internal reference signals predicting the visual consequences of an eye movement. Here we demonstrate that the reference signal predicting the consequences of smooth-pursuit eye movements is continuously calibrated on the basis of direction-selective interactions between the pursuit motor command and the rotational flow induced by the eye movement, thereby minimizing imperfections of the reference signal and guaranteeing an ecologically optimal interpretation of visual motion.     

The optokinetic response in wild type and white zebra finches

Optic flow is a main source of information about self movement and the three-dimensional composition of the environment during locomotion. It is processed by the accessory optic system in all vertebrates. The optokinetic response is elicited by rotational optic flow, e.g. in a rotating drum lined with vertical stripes. We investigated here the effect of rotational optic flow on the optokinetic response in wild type and white zebra finches. The highest stimulus velocity eliciting an optokinetic response (upper velocity threshold) was dependent on stimulus direction and illumination level, but was not different between the colour morphs. The upper velocity threshold was higher with temporal to nasal movements in monocularly exposed birds and symmetrical with binocular exposure. Its increase with illumination level followed Fechner's law and reached a plateau at about 560 Lux. In bright daylight, white birds did not show optokinetic responses. We conclude that the altered wiring of the visual system of white birds has no influence on accessory optic system function. The unwillingness of white birds to respond with optokinetic response in bright daylight may be due to a substantial lack of inhibition within the visual system as demonstrated earlier, which may enhance the sensibility to glare.