Optic flow-based collision-free strategies: From insects to robots

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects’ abilities and better understanding their flight.     

Modulation of whitefly take-off and flight orientation by wind speed and visual cues

The effect of different wind speeds on take-off and flight orientation of the sweetpotato whitefly, Bemisia tabaci Gennadius (Homoptera: Aleyrodidae), was studied in the presence of a green visual stimulus which reflected 550 ± 10 nm light, or a white stimulus of the same intensity. When the white light was present, take-off was negatively correlated with wind speed. Analysis of the flight tracks of whiteflies in 0, 15 and 30 cm/s wind with the white light present showed that flight was not directed toward the stimulus in zero wind, and that insects were carried downwind as the wind increased. Net displacement downwind was significantly slower than the wind speed, indicating that B. tabaci can control its rate of displacement relative to its surroundings, and is not always passively transported by the wind. In the presence of the green visual stimulus, take-off and flight behaviour of B. tabaci was markedly different to that observed in the presence of the white light. Taking off was more likely and whiteflies made upwind orientated flights, landing on the illuminated section of the screen when it reflected green light. At all wind speeds tested, the mean ground speeds of B. tabaci were approximately 20 cm/s whether the insects were flying upwind or downwind. This uniformity of ground speed regardless of the changing effects of wind-induced drift in different directions strongly suggests that whiteflies actively control their ground speed using visual flow fields in a manner similar to all other flying insects examined thus far.     

To what extent can the tiny parasitoid wasps,Eretmocerus mundus,fly upwind?

Body miniaturization in insects is predicted to result in decreased flight speed and therefore limited ability of these insects to fly upwind. Therefore, tiny insects are often regarded as relying on passive dispersal by winds. We tested this assumption in a wind tunnel by measuring the burst speed of Eretmocerus mundus (Mercet), a beneficial parasitoid wasp with body length <1 mm. Insects were filmed flying upwind towards a UV light source in a range of wind speed 0–0.5 m/s. The Insects flew towards the UV light in the absence and presence of wind but increased their flight speed in the presence of wind. They also changed flight direction to be directly upwind and maintained this body orientation even while drifted backwards relative to the ground by stronger winds. Field measurements showed that the average flight speed observed in the wind tunnel (0.3 m/s) is sufficient to allow flying between plants even when the wind speed above the vegetation was 3–5 folds higher. A simulation of the ability of the insects to control their flight trajectory towards a visual target (sticky traps) in winds show that the insects can manipulate their progress relative to the ground even when the wind speed exceeds their flight speed. The main factors determining the ability of the insects to reach the trap were trap diameter and the difference between insect flight speed and wind speed. The simulation also predicts the direction of arrival to the sticky target showing that many of the insects reach the target from the leeward side (i.e. by flight upwind). In light of these results, the notion that miniature insects passively disperse by winds is misleading because it disregards the ability of the insects to control their drift relative to the ground in winds that are faster than their flight speed.     

Control of self-motion in dynamic fluids: fish do it differently from bees

To detect and avoid collisions, animals need to perceive and control the distance and the speed with which they are moving relative to obstacles. This is especially challenging for swimming and flying animals that must control movement in a dynamic fluid without reference from physical contact to the ground. Flying animals primarily rely on optic flow to control flight speed and distance to obstacles. Here, we investigate whether swimming animals use similar strategies for self-motion control to flying animals by directly comparing the trajectories of zebrafish (Danio rerio) and bumblebees (Bombus terrestris) moving through the same experimental tunnel. While moving through the tunnel, black and white patterns produced (i) strong horizontal optic flow cues on both walls, (ii) weak horizontal optic flow cues on both walls and (iii) strong optic flow cues on one wall and weak optic flow cues on the other. We find that the mean speed of zebrafish does not depend on the amount of optic flow perceived from the walls. We further show that zebrafish, unlike bumblebees, move closer to the wall that provides the strongest visual feedback. This unexpected preference for strong optic flow cues may reflect an adaptation for self-motion control in water or in environments where visibility is limited.     

Nocturnal insects use optic flow for flight control

To avoid collisions when navigating through cluttered environments, flying insects must control their flight so that their sensory systems have time to detect obstacles and avoid them. To do this, day-active insects rely primarily on the pattern of apparent motion generated on the retina during flight (optic flow). However, many flying insects are active at night, when obtaining reliable visual information for flight control presents much more of a challenge. To assess whether nocturnal flying insects also rely on optic flow cues to control flight in dim light, we recorded flights of the nocturnal neotropical sweat bee, Megalopta genalis, flying along an experimental tunnel when: (i) the visual texture on each wall generated strong horizontal (front-to-back) optic flow cues, (ii) the texture on only one wall generated these cues, and (iii) horizontal optic flow cues were removed from both walls. We find that Megalopta increase their groundspeed when horizontal motion cues in the tunnel are reduced (conditions (ii) and (iii)). However, differences in the amount of horizontal optic flow on each wall of the tunnel (condition (ii)) do not affect the centred position of the bee within the flight tunnel. To better understand the behavioural response of Megalopta, we repeated the experiments on day-active bumble-bees (Bombus terrestris). Overall, our findings demonstrate that despite the limitations imposed by dim light, Megalopta-like their day-active relatives-rely heavily on vision to control flight, but that they use visual cues in a different manner from diurnal insects.     

Visual ground pattern modulates flight speed of male Oriental fruit moth Grapholita molesta

Insects flying in a horizontal pheromone plume must attend to visual cues to ensure that they make upwind progress. Moreover, it is suggested that flying insects will also modulate their flight speed to maintain a constant retinal angular velocity of terrestrial contrast elements. Evidence from flies and honeybees supports such a hypothesis, although tests with male moths and beetles flying in pheromone plumes are not conclusive. These insects typically fly faster at higher elevations above a high‐contrast ground pattern, as predicted by the hypothesis, although the increase in speed is not sufficient to demonstrate quantitatively that they maintain constant visual angular velocity of the ground pattern. To test this hypothesis more rigorously, the flight speed of male oriental fruit moths (OFM) Grapholita molesta Busck (Lepidoptera: Tortricidae) flying in a sex pheromone plume in a laboratory wind tunnel is measured at various heights (5–40 cm) above patterns of different spatial wavelength (1.8–90°) in the direction of flight. The OFM modulate their flight speed three‐fold over different patterns. They fly fastest when there is no pattern in the tunnel or the contrast elements are too narrow to resolve. When the spatial wavelength of the pattern is sufficiently wide to resolve, moths fly at a speed that tends to maintain a visual contrast frequency of 3.5 ± 3.2 Hz rather than a constant angular velocity, which varies from 57 to 611° s^?1. In addition, for the first time, it is also demonstrated that the width of a contrast pattern perpendicular to the flight direction modulates flight speed.     

Optic flow cues guide flight in birds

Although considerable effort has been devoted to investigating how birds migrate over large distances, surprisingly little is known about how they tackle so successfully the moment-to-moment challenges of rapid flight through cluttered environments [1]. It has been suggested that birds detect and avoid obstacles [2] and control landing maneuvers [3-5] by using cues derived from the image motion that is generated in the eyes during flight. Here we investigate the ability of budgerigars to fly through narrow passages in a collision-free manner, by filming their trajectories during flight in a corridor where the walls are decorated with various visual patterns. The results demonstrate, unequivocally and for the first time, that birds negotiate narrow gaps safely by balancing the speeds of image motion that are experienced by the two eyes and that the speed of flight is regulated by monitoring the speed of image motion that is experienced by the two eyes. These findings have close parallels with those previously reported for flying insects [6-13], suggesting that some principles of visual guidance may be shared by all diurnal, flying animals.     

Eye movements and target fixation during dragonfly prey-interception flights

The capture of flying insects by foraging dragonflies is a highly accurate, visually guided behavior. Rather than simply aiming at the prey’s position, the dragonfly aims at a point in front of the prey, so that the prey is intercepted with a relatively straight flight trajectory. To better understand the neural mechanisms underlying this behavior, we used high-speed video to quantify the head and body orientation of dragonflies (female Erythemis simplicicollis flying in an outdoor flight cage) relative to an artificial prey object before and during pursuit. The results of our frame-by-frame analysis showed that during prey pursuit, the dragonfly adjusts its head orientation to maintain the image of the prey centered on the “crosshairs” formed by the visual midline and the dorsal fovea, a high acuity streak that crosses midline at right angles about 60° above the horizon. The visual response latencies to drifting of the prey image are remarkably short, ca. 25 ms for the head and 30 ms for the wing responses. Our results imply that the control of the prey-interception flight must include a neural pathway that takes head position into account.     

Recognition and avoidance of contaminated flowers by foraging bumblebees (Bombus terrestris)

Bumblebee colonies are founded by a single-mated queen. Due to this life history trait, bumblebees are more susceptible to parasites and diseases than polyandrous and/or polygynous social insects. A greater resistance towards parasites is shown when the genetic variability within a colony is increased. The parasite resistance may be divided into different levels regarding the step of the parasite infection (e.g. parasite uptake, parasite intake, parasite's establishment in the nest, parasite transmission). We investigate the prophylactic behaviour of bumblebees. Bumblebees were observed during their foraging flights on two artificial flowers; one of these was contaminated by Crithidia bombi, a naturally occurring gut parasite of bumblebees (in a control experiment the non-specific pathogen Escherichia coli was used). For C. bombi, bumblebees were preferentially observed feeding on the non-contaminated flower. Whereas for E. coli, the number of visits between flowers was the same, bumblebees spent more time feeding on the non-contaminated flower. These results demonstrate the ability of bumblebees to recognise the contamination of food sources. In addition, bumblebees have a stronger preference for the non-contaminated flower when C. bombi is present in the other flower than with E. coli which might be explained as an adaptive behaviour of bumblebees towards this specific gut parasite. It seems that the more specific the parasite is, the more it reduces the reward of the flower.     

Egomotion estimation with optic flow and air velocity sensors

We develop a method that allows a flyer to estimate its own motion (egomotion), the wind velocity, ground slope, and flight height using only inputs from onboard optic flow and air velocity sensors. Our artificial algorithm demonstrates how it could be possible for flying insects to determine their absolute egomotion using their available sensors, namely their eyes and wind sensitive hairs and antennae. Although many behaviors can be performed by only knowing the direction of travel, behavioral experiments indicate that odor tracking insects are able to estimate the wind direction and control their absolute egomotion (i.e., groundspeed). The egomotion estimation method that we have developed, which we call the opto-aeronautic algorithm, is tested in a variety of wind and ground slope conditions using a video recorded flight of a moth tracking a pheromone plume. Over all test cases that we examined, the algorithm achieved a mean absolute error in height of 7% or less. Furthermore, our algorithm is suitable for the navigation of aerial vehicles in environments where signals from the Global Positioning System are unavailable.     

Visuomotor Response to Object Expansion in Free-Flying Bumble Bees

The flight control systems of flying insects enable many kinds of sophisticated maneuvers, including avoidance of midair collisions. Visuomotor response to an approaching object, received as image expansion on insects’ retina, is a complex event in a dynamic environment where both animals and objects are moving. There are intensive free flight studies on the landing response in which insects receive image expansion by their own movement. However, few studies have been conducted regarding how freely flying insects respond to approaching objects. Here, using common laboratory insects for behavioral research, the bumblebee Bombus ignitus, we examined their visual response to an approaching object in the free-flying condition. While the insect was slowly flying in a free-flight arena, an expanding stripe was projected laterally from one side of the arena with a high-speed digital mirror device projector. Rather than turning away reported before, the bumble bees performed complex flight maneuvers. We synchronized flight trajectories, orientations and wing stroke frequencies with projection parameters of temporal resolution in 0.5 ms, and analyzed the instantaneous relationship between visual input and behavioral output. In their complex behavioral responses, we identified the following two visuomotor behaviors: increasing stroke frequency when the bumble bees confront the stripe expansion, and turning towards (not away) the stripe expansion when it is located laterally to the bee. Our results suggested that the response to object expansion is not a simple and reflexive escape but includes object fixation, presumably for subsequent behavioral choice.     

Navigation without vision: bumblebee orientation in complete darkness

In flight cages, worker bumblebees (Bombus impatiens) spontaneously explored the surroundings of their nest and foraged in complete darkness, by walking instead of flying, from feeders up to 150 cm away from the nest. This behaviour was wholly unexpected in these classically visual foragers. The finding provides a controlled system for dissecting possible non-visual components of navigation used in daylight. It also allows us to isolate navigation mechanisms used in naturally dark situations, such as in the nest. Using infrared video, we mapped walking trails. We found that bumblebees laid odour marks. When such odour cues were eliminated, bees maintained correct directionality, suggesting a magnetic compass. They were also able to assess travel distance correctly, using an internal, non-visual, measure of path length. Path integration was not employed. Presumably, this complex navigational skill requires visual input in bees.     

Free-flight odor tracking in Drosophila is consistent with an optimal intermittent scale-free search

During their trajectories in still air, fruit flies (Drosophila melanogaster) explore their landscape using a series of straight flight paths punctuated by rapid 90 degrees body-saccades [1]. Some saccades are triggered by visual expansion associated with collision avoidance. Yet many saccades are not triggered by visual cues, but rather appear spontaneously. Our analysis reveals that the control of these visually independent saccades and the flight intervals between them constitute an optimal scale-free active searching strategy. Two characteristics of mathematical optimality that are apparent during free-flight in Drosophila are inter-saccade interval lengths distributed according to an inverse square law, which does not vary across landscape scale, and 90 degrees saccade angles, which increase the likelihood that territory will be revisited and thereby reduce the likelihood that near-by targets will be missed. We also show that searching is intermittent, such that active searching phases randomly alternate with relocation phases. Behaviorally, this intermittency is reflected in frequently occurring short, slow speed inter-saccade intervals randomly alternating with rarer, longer, faster inter-saccade intervals. Searching patterns that scale similarly across orders of magnitude of length (i.e., scale-free) have been revealed in animals as diverse as microzooplankton, bumblebees, albatrosses, and spider monkeys, but these do not appear to be optimised with respect to turning angle, whereas Drosophila free-flight search does. Also, intermittent searching patterns, such as those reported here for Drosophila, have been observed in foragers such as planktivorous fish and ground foraging birds. Our results with freely flying Drosophila may constitute the first reported example of searching behaviour that is both scale-free and intermittent.     

Orientation in high-flying migrant insects in relation to flows: mechanisms and strategies

High-flying insect migrants have been shown to display sophisticated flight orientations that can, for example, maximize distance travelled by exploiting tailwinds, and reduce drift from seasonally optimal directions. Here, we provide a comprehensive overview of the theoretical and empirical evidence for the mechanisms underlying the selection and maintenance of the observed flight headings, and the detection of wind direction and speed, for insects flying hundreds of metres above the ground. Different mechanisms may be used—visual perception of the apparent ground movement or mechanosensory cues maintained by intrinsic features of the wind—depending on circumstances (e.g. day or night migrations). In addition to putative turbulence-induced velocity, acceleration and temperature cues, we present a new mathematical analysis which shows that ‘jerks’ (the time-derivative of accelerations) can provide indicators of wind direction at altitude. The adaptive benefits of the different orientation strategies are briefly discussed, and we place these new findings for insects within a wider context by comparisons with the latest research on other flying and swimming organisms.This article is part of the themed issue ‘Moving in a moving medium: new perspectives on flight’.     

Categorization of Natural Dynamic Audiovisual Scenes

This work analyzed the perceptual attributes of natural dynamic audiovisual scenes. We presented thirty participants with 19 natural scenes in a similarity categorization task, followed by a semi-structured interview. The scenes were reproduced with an immersive audiovisual display. Natural scene perception has been studied mainly with unimodal settings, which have identified motion as one of the most salient attributes related to visual scenes, and sound intensity along with pitch trajectories related to auditory scenes. However, controlled laboratory experiments with natural multimodal stimuli are still scarce. Our results show that humans pay attention to similar perceptual attributes in natural scenes, and a two-dimensional perceptual map of the stimulus scenes and perceptual attributes was obtained in this work. The exploratory results show the amount of movement, perceived noisiness, and eventfulness of the scene to be the most important perceptual attributes in naturalistically reproduced real-world urban environments. We found the scene gist properties openness and expansion to remain as important factors in scenes with no salient auditory or visual events. We propose that the study of scene perception should move forward to understand better the processes behind multimodal scene processing in real-world environments. We publish our stimulus scenes as spherical video recordings and sound field recordings in a publicly available database.     

A bio-inspired flying robot sheds light on insect piloting abilities

When insects are flying forward, the image of the ground sweeps backward across their ventral viewfield and forms an "optic flow," which depends on both the groundspeed and the groundheight. To explain how these animals manage to avoid the ground by using this visual motion cue, we suggest that insect navigation hinges on a visual-feedback loop we have called the optic-flow regulator, which controls the vertical lift. To test this idea, we built a micro-helicopter equipped with an optic-flow regulator and a bio-inspired optic-flow sensor. This fly-by-sight micro-robot can perform exacting tasks such as take-off, level flight, and landing. Our control scheme accounts for many hitherto unexplained findings published during the last 70 years on insects' visually guided performances; for example, it accounts for the fact that honeybees descend in a headwind, land with a constant slope, and drown when travelling over mirror-smooth water. Our control scheme explains how insects manage to fly safely without any of the instruments used onboard aircraft to measure the groundheight, groundspeed, and descent speed. An optic-flow regulator is quite simple in terms of its neural implementation and just as appropriate for insects as it would be for aircraft.     

A comparison of distances flown by different visitors to flowers of the same species

Summary The morphologically complex flowers of Delphinium nelsonii, D. barbeyi, and Ipomopsis aggregata are visited by a wide variety of animals. Visitors to each species range from small insects, such as worker bumblebees and solitary bees, to hummingbirds, and thus span roughly an order of magnitude in body mass and metabolic rate while flying; they also differ in type of food collected and in their efficacy as pollinators. Despite these differences, all the visitors to a given plant species fly similar, short distances between successively visited flowers and plants. There are no significant relationships between mean flight distance and metabolic rate or body mass among the visitors to any plant species. Thus there is no evidence that flight characteristics depend on anything as straightforward as whether flower visitors have high or low energetic requirements.     

The effect of optic flow cues on honeybee flight control in wind

To minimize the risk of colliding with the ground or other obstacles, flying animals need to control both their ground speed and ground height. This task is particularly challenging in wind, where head winds require an animal to increase its airspeed to maintain a constant ground speed and tail winds may generate negative airspeeds, rendering flight more difficult to control. In this study, we investigate how head and tail winds affect flight control in the honeybee Apis mellifera, which is known to rely on the pattern of visual motion generated across the eye—known as optic flow—to maintain constant ground speeds and heights. We find that, when provided with both longitudinal and transverse optic flow cues (in or perpendicular to the direction of flight, respectively), honeybees maintain a constant ground speed but fly lower in head winds and higher in tail winds, a response that is also observed when longitudinal optic flow cues are minimized. When the transverse component of optic flow is minimized, or when all optic flow cues are minimized, the effect of wind on ground height is abolished. We propose that the regular sidewards oscillations that the bees make as they fly may be used to extract information about the distance to the ground, independently of the longitudinal optic flow that they use for ground speed control. This computationally simple strategy could have potential uses in the development of lightweight and robust systems for guiding autonomous flying vehicles in natural environments.     

Visual control of prey-capture flight in dragonflies

Interacting with a moving object poses a computational problem for an animal's nervous system. This problem has been elegantly solved by the dragonfly, a formidable visual predator on flying insects. The dragonfly computes an interception flight trajectory and steers to maintain it during its prey-pursuit flight. This review summarizes current knowledge about pursuit behavior and neurons thought to control interception in the dragonfly. When understood, this system has the potential for explaining how a small group of neurons can control complex interactions with moving objects.