Computer-aided studies of vision in crabs

A system for the back projection of computer-generated visual images onto a screen or screens that cover 240° of the horizontal visual field is described. Its applicability for the study of crab vision is tested by comparing the frequency response of the optokinetic response of the land crab, Cardisoma guanhumi, to sinusoidal oscillation of computer-generated striped patterns and a real striped drum. Significant differences were observed only at the low end of the frequency spectrum. The flexibility of computer-generated visual stimulation and its advantages for the study of optic flow are illustrated by experiments that: (a) demonstrate how well crabs separate the translational and rotational components of optic flow by showing compensatory eye movements to only the latter; (b) show that the ability to compensate for rotation is not impaired by combinations of rotation and translation; (c) show that motion parallax cues are used in addition to previously-described global cues for making the distinction between rotation and translation. Finally, the use of these methods in a successful search for visual interneurones sensitive to optic flow stimuli is demonstrated for the shore crab, Carcinus maenas.     

Computer-aided studies of vision in crabs

A system for the back projection of computer-generated visual images onto a screen or screens that cover 240° of the horizontal visual field is described. Its applicability for the study of crab vision is tested by comparing the frequency response of the optokinetic response of the land crab, Cardisoma guanhumi , to sinusoidal oscillation of computer-generated striped patterns and a real striped drum. Significant differences were observed only at the low end of the frequency spectrum. The flexibility of computer-generated visual stimulation and its advantages for the study of optic flow are illustrated by experiments that: (a) demonstrate how well crabs separate the translational and rotational components of optic flow by showing compensatory eye movements to only the latter; (b) show that the ability to compensate for rotation is not impaired by combinations of rotation and translation; (c) show that motion parallax cues are used in addition to previously-described global cues for making the distinction between rotation and translation. Finally, the use of these methods in a successful search for visual interneurones sensitive to optic flow stimuli is demonstrated for the shore crab, Carcinus maenas .     

To keep on track during flight,fruitflies discount the skyward view

When small flying insects go off their intended course, they use the resulting pattern of motion on their eye, or optic flow, to guide corrective steering. A change in heading generates a unique, rotational motion pattern and a change in position generates a translational motion pattern, and each produces corrective responses in the wingbeats. Any image in the flow field can signal rotation, but owing to parallax, only the images of nearby objects can signal translation. Insects that fly near the ground might therefore respond more strongly to translational optic flow that occurs beneath them, as the nearby ground will produce strong optic flow. In these experiments, rigidly tethered fruitflies steered in response to computer-generated flow fields. When correcting for unintended rotations, flies weight the motion in their upper and lower visual fields equally. However, when correcting for unintended translations, flies weight the motion in the lower visual fields more strongly. These results are consistent with the interpretation that fruitflies stabilize by attending to visual areas likely to contain the strongest signals during natural flight conditions.     

Optimizing visual motion perception during eye movements.

We usually perceive a stationary, stable world and we are able to correctly estimate the direction of heading from optic flow despite coherent visual motion induced by eye movements. This astonishing example of perceptual invariance results from a comparison of visual information with internal reference signals predicting the visual consequences of an eye movement. Here we demonstrate that the reference signal predicting the consequences of smooth-pursuit eye movements is continuously calibrated on the basis of direction-selective interactions between the pursuit motor command and the rotational flow induced by the eye movement, thereby minimizing imperfections of the reference signal and guaranteeing an ecologically optimal interpretation of visual motion.     

Gaze Strategy in the Free Flying Zebra Finch (Taeniopygia guttata)

Fast moving animals depend on cues derived from the optic flow on their retina. Optic flow from translational locomotion includes information about the three-dimensional composition of the environment, while optic flow experienced during a rotational self motion does not. Thus, a saccadic gaze strategy that segregates rotations from translational movements during locomotion will facilitate extraction of spatial information from the visual input. We analysed whether birds use such a strategy by highspeed video recording zebra finches from two directions during an obstacle avoidance task. Each frame of the recording was examined to derive position and orientation of the beak in three-dimensional space. The data show that in all flights the head orientation was shifted in a saccadic fashion and was kept straight between saccades. Therefore, birds use a gaze strategy that actively stabilizes their gaze during translation to simplify optic flow based navigation. This is the first evidence of birds actively optimizing optic flow during flight.     

Visual position stabilization in the hummingbird hawk moth, Macroglossum stellatarum L. I. Behavioural analysis

Optomotor responses of freely flying hawk moths, Macroglossum stellatarum, were characterized while the animals were hovering in front of and feeding on a dummy flower. Compensatory translational and rotational movements of the hawk moth were elicited by vertical grating patterns moving horizontally, mimicking imposed rotational and translational displacements of the animal in the horizontal plane. Oscillatory translational and rotational pattern motion leads to compensatory responses that peak in the frequency range between 2 Hz and 4 Hz. The control systems mediating the translational and rotational components of the optomotor response do not seem to influence each other. The system mediating translational responses is more sensitive in the fronto-lateral part of the visual field than in the lateral part; the opposite is true for the rotational system. The sensitivity of the translational system does not change along the vertical, whereas the rotational system is much more sensitive to motion in the dorsal than in the ventral part of the visual field. These sensitivity gradients may reflect an adaptation to the specific requirements of position stabilization in front of flowers during feeding. Accepted: 13 August 1997     

Visual position stabilization in the hummingbird hawk moth, Macroglossum stellatarum L. II. Electrophysiological analysis of neurons sensitive to wide-field image motion

Response properties of neurons in the cervical connectives of the hummingbird hawk moth, Macroglossum stellatarum L., were determined. All neurons described in this account respond directionally selectively to motion in large parts of the visual field of either eye. They respond maximally to bilateral stimulation, preferring either motion as induced on the eyes during translatory movements of the animal or when it turns around one of its body axes. Cells most sensitive to rotational motion either respond best to rotation of the patterns around the vertical axis of the animal or around its longitudinal body axis. Neurons most sensitive to translational pattern motion respond best to either simulated translations of the animal along its vertical or along an oblique axis. Most types of neurons respond tonically and do not habituate. The sensitivity to motion stimuli is not evenly distributed within the receptive field of any investigated neuron. Part of these neurons might play a role in visual position and course stabilization. Accepted: 13 August 1997     

Optic flow-based collision-free strategies: From insects to robots

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects’ abilities and better understanding their flight.     

A Bio-inspired Collision Avoidance Model Based on Spatial Information Derived from Motion Detectors Leads to Common Routes

Avoiding collisions is one of the most basic needs of any mobile agent, both biological and technical, when searching around or aiming toward a goal. We propose a model of collision avoidance inspired by behavioral experiments on insects and by properties of optic flow on a spherical eye experienced during translation, and test the interaction of this model with goal-driven behavior. Insects, such as flies and bees, actively separate the rotational and translational optic flow components via behavior, i.e. by employing a saccadic strategy of flight and gaze control. Optic flow experienced during translation, i.e. during intersaccadic phases, contains information on the depth-structure of the environment, but this information is entangled with that on self-motion. Here, we propose a simple model to extract the depth structure from translational optic flow by using local properties of a spherical eye. On this basis, a motion direction of the agent is computed that ensures collision avoidance. Flying insects are thought to measure optic flow by correlation-type elementary motion detectors. Their responses depend, in addition to velocity, on the texture and contrast of objects and, thus, do not measure the velocity of objects veridically. Therefore, we initially used geometrically determined optic flow as input to a collision avoidance algorithm to show that depth information inferred from optic flow is sufficient to account for collision avoidance under closed-loop conditions. Then, the collision avoidance algorithm was tested with bio-inspired correlation-type elementary motion detectors in its input. Even then, the algorithm led successfully to collision avoidance and, in addition, replicated the characteristics of collision avoidance behavior of insects. Finally, the collision avoidance algorithm was combined with a goal direction and tested in cluttered environments. The simulated agent then showed goal-directed behavior reminiscent of components of the navigation behavior of insects.     

Self-motion-induced eye movements: effects on visual acuity and navigation

Self-motion disturbs the stability of retinal images by inducing optic flow. Objects of interest need to be fixated or tracked, yet these eye movements can infringe on the experienced retinal flow that is important for visual navigation. Separating the components of optic flow caused by an eye movement from those due to self-motion, as well as using optic flow for visual navigation while simultaneously maintaining visual acuity on near targets, represent key challenges for the visual system. Here we summarize recent advances in our understanding of how the visuomotor and vestibulomotor systems function and interact, given the complex task of compensating for instabilities of retinal images, which typically vary as a function of retinal location and differ for each eye.     

Saccadic flight strategy facilitates collision avoidance: closed-loop performance of a cyberfly

Behavioural and electrophysiological experiments suggest that blowflies employ an active saccadic strategy of flight and gaze control to separate the rotational from the translational optic flow components. As a consequence, this allows motion sensitive neurons to encode during translatory intersaccadic phases of locomotion information about the spatial layout of the environment. So far, it has not been clear whether and how a motor controller could decode the responses of these neurons to prevent a blowfly from colliding with obstacles. Here we propose a simple model of the blowfly visual course control system, named cyberfly, and investigate its performance and limitations. The sensory input module of the cyberfly emulates a pair of output neurons subserving the two eyes of the blowfly visual motion pathway. We analyse two sensory–motor interfaces (SMI). An SMI coupling the differential signal of the sensory neurons proportionally to the yaw rotation fails to avoid obstacles. A more plausible SMI is based on a saccadic controller. Even with sideward drift after saccades as is characteristic of real blowflies, the cyberfly is able to successfully avoid collisions with obstacles. The relative distance information contained in the optic flow during translatory movements between saccades is provided to the SMI by the responses of the visual output neurons. An obvious limitation of this simple mechanism is its strong dependence on the textural properties of the environment.     

Visual gaze control during peering flight manoeuvres in honeybees

As animals travel through the environment, powerful reflexes help stabilize their gaze by actively maintaining head and eyes in a level orientation. Gaze stabilization reduces motion blur and prevents image rotations. It also assists in depth perception based on translational optic flow. Here we describe side-to-side flight manoeuvres in honeybees and investigate how the bees’ gaze is stabilized against rotations during these movements. We used high-speed video equipment to record flight paths and head movements in honeybees visiting a feeder. We show that during their approach, bees generate lateral movements with a median amplitude of about 20 mm. These movements occur with a frequency of up to 7 Hz and are generated by periodic roll movements of the thorax with amplitudes of up to ±60°. During such thorax roll oscillations, the head is held close to horizontal, thereby minimizing rotational optic flow. By having bees fly through an oscillating, patterned drum, we show that head stabilization is based mainly on visual motion cues. Bees exposed to a continuously rotating drum, however, hold their head fixed at an oblique angle. This result shows that although gaze stabilization is driven by visual motion cues, it is limited by other mechanisms, such as the dorsal light response or gravity reception.     

Response to self-motion in waterstriders: visual discrimination between rotation and translation

Waterstriders (Gerris paludum) often try to maintain a nearly stationary position on a moving water surface. Passive motion is restricted to 3 degrees of freedom: yaw, longitudinal, and transverse displacement. They correct for passive rotation and translation by distinct behavioral sequences. The compensatory behavior is predominantly visually controlled.

High-Speed Video-Oculography for Measuring Three-Dimensional Rotation Vectors of Eye Movements in Mice

Background

The mouse is the most commonly used animal model in biomedical research because of recent advances in molecular genetic techniques. Studies related to eye movement in mice are common in fields such as ophthalmology relating to vision, neuro-otology relating to the vestibulo-ocular reflex (VOR), neurology relating to the cerebellum’s role in movement, and psychology relating to attention. Recording eye movements in mice, however, is technically difficult.

Methods

We developed a new algorithm for analyzing the three-dimensional (3D) rotation vector of eye movement in mice using high-speed video-oculography (VOG). The algorithm made it possible to analyze the gain and phase of VOR using the eye’s angular velocity around the axis of eye rotation.

Results

When mice were rotated at 0.5 Hz and 2.5 Hz around the earth’s vertical axis with their heads in a 30° nose-down position, the vertical components of their left eye movements were in phase with the horizontal components. The VOR gain was 0.42 at 0.5 Hz and 0.74 at 2.5 Hz, and the phase lead of the eye movement against the turntable was 16.1° at 0.5 Hz and 4.88° at 2.5 Hz.

Conclusions

To the best of our knowledge, this is the first report of this algorithm being used to calculate a 3D rotation vector of eye movement in mice using high-speed VOG. We developed a technique for analyzing the 3D rotation vector of eye movements in mice with a high-speed infrared CCD camera. We concluded that the technique is suitable for analyzing eye movements in mice. We also include a C++ source code that can calculate the 3D rotation vectors of the eye position from two-dimensional coordinates of the pupil and the iris freckle in the image to this article.     

Visual fields, eye movements, and scanning behavior of a sit-and-wait predator, the black phoebe (Sayornis nigricans)

Foraging mode influences the dominant sensory modality used by a forager and likely the strategies of information gathering used in foraging and anti-predator contexts. We assessed three components of visual information gathering in a sit-and-wait avian predator, the black phoebe (Sayornis nigricans): configuration of the visual field, degree of eye movement, and scanning behavior through head-movement rates. We found that black phoebes have larger lateral visual fields than similarly sized ground-foraging passerines, as well as relatively narrower binocular and blind areas. Black phoebes moved their eyes, but eye movement amplitude was relatively smaller than in other passerines. Black phoebes may compensate for eye movement constraints with head movements. The rate of head movements increased before attacking prey in comparison to non-foraging contexts and before movements between perches. These findings suggest that black phoebes use their lateral visual fields, likely subtended by areas of high acuity in the retina, to track prey items in a three-dimensional space through active head movements. These head movements may increase depth perception, motion detection and tracking. Studying information gathering through head movement changes, rather than body posture changes (head-up, head-down) as generally presented in the literature, may allow us to better understand the mechanisms of information gathering from a comparative perspective.     

Motion parallax contribution to perception of self-motion and depth

The object of this study is to mathematically specify important characteristics of visual flow during translation of the eye for the perception of depth and self-motion. We address various strategies by which the central nervous system may estimate self-motion and depth from motion parallax, using equations for the visual velocity field generated by translation of the eye through space. Our results focus on information provided by the movement and deformation of three-dimensional objects and on local flow behavior around a fixated point. All of these issues are addressed mathematically in terms of definite equations for the optic flow. This formal characterization of the visual information presented to the observer is then considered in parallel with other sensory cues to self-motion in order to see how these contribute to the effective use of visual motion parallax, and how parallactic flow can, conversely, contribute to the sense of self-motion. This article will focus on a central case, for understanding of motion parallax in spacious real-world environments, of monocular visual cues observable during pure horizontal translation of the eye through a stationary environment. We suggest that the global optokinetic stimulus associated with visual motion parallax must converge in significant fashion with vestibular and proprioceptive pathways that carry signals related to self-motion. Suggestions of experiments to test some of the predictions of this study are made.     

Optomotor course control in flies with largely asymmetric visual input

We have studied freely flying and walking flies as well as flies flying in a flight simulator in order to discover how functionally blinding one of the eyes affects the fly's ability to move straight. It is hard to tell just by observing the animals' movements whether they have been deprived of vision in one eye. Statistical analysis is need to show that there are differences in the locomotory paths of monocular and binocular flies: monocular flies tend to turn slightly towards the side of the seeing eye. It is possible that the superimposed translational and rotational optic flow fields, generated on the trajectory of monocular flies, sum to zero net flow. This overall flow over the retina of the open eye might lead to a state of optomotor equilibrium. Accepted: 11 October 1999     

Motion anisotropies and heading detection

 In motion-processing areas of the visual cortex in cats and monkeys, an anisotropic distribution of direction selectivities displays a preference for movements away from the fovea. This ‘centrifugal bias’ has been hypothetically linked to the processing of optic flow fields generated during forward locomotion. In this paper, we show that flow fields induced on the retina in many natural situations of locomotion of higher mammals are indeed qualitatively centrifugal in structure, even when biologically plausible eye movements to stabilize gaze on environmental targets are performed. We propose a network model of heading detection that carries an anisotropy similar to the one found in cat and monkey. In simulations, this model reproduces a number of psychophysical results of human heading detection. It suggests that a recently reported human disability to correctly identify the direction of heading from optic flow when a certain type of eye movement is simulated might be linked to the noncentrifugal structure of the resulting retinal flow field and to the neurophysiological anisotropies. Received: 1 April 1994/Accepted in revised form: 4 August 1994     

Sensory and behavioral properties of neurons in posterior parietal cortex of the awake, trained monkey.

Neurons in posterior parietal cortex of the awake, trained monkey respond to passive visual and/or somatosensory stimuli. In general, the receptive fields of these cells are large and nonspecific. When these neurons are studied during visually guided hand movements and eye movements, most of their activity can be accounted for by passive sensory stimulation. However, for some visual cells, the response to a stimulus is enhanced when it is to be the target for a saccadic eye movement. This enhancement is selective for eye movements into the visual receptive field since it does not occur with eye movements to other parts of the visual field. Cells that discharge in association with a visual fixation task have foveal receptive fields and respond to the spots of light used as fixation targets. Cells discharging selectively in association with different directions of tracking eye movements have directionally selective responses to moving visual stimuli. Every cell in our sample discharging in association with movement could be driven by passive sensory stimuli. We conclude that the activity of neurons in posterior parietal cortex is dependent on and indicative of external stimuli but not predictive of movement.     

Rotational locomotion by the cockroach Blattella germanica

Locomotion on complex substrata can be expressed in a plane by two geometric components of body movement: linear locomotion and rotational locomotion. This study examined pure rotation by analysing the geometry of leg movements and stepping patterns during the courtship turns of male Blattella germanica. Strict rotation or translation by an insect requires that each side of the body cover equal distance with respect to the substrate. There are three mechanisms by which the legs can maintain this equality: frequency of stepping, magnitude of the leg arcs relative to the body and the degree to which legs flex and extend during locomotion. During the courtship behaviour of Blattella germanica selected males executed turns involving body rotation along with leg movements in which the legs on the outside of the turn swung through greater average arcs than those on the inside of the turn. This difference should have resulted in a translation component. However, legs on the inside of the turn compensated by flexion and extension movements which were greater than those of opposing legs. The net effect was that both sides of the body covered equal average ground. These cockroaches used a wide variety of stepping combinations to effect rotation. The frequency of these combinations was compared to an expected frequency distribution of stepping combinations and further to an expected frequency of these stepping combinations used for straight walking. These comparisons demonstrated a similarity between interleg coordination during straight walking and that during turning in place.