Bio-inspired visual ego-rotation sensor for MAVs

Flies are capable of extraordinary flight maneuvers at very high speeds largely due to their highly elaborate visual system. In this work we present a fly-inspired FPGA based sensor system able to visually sense rotations around different body axes, for use on board micro aerial vehicles (MAVs). Rotation sensing is performed analogously to the fly’s VS cell network using zero-crossing detection. An additional key feature of our system is the ease of adding new functionalities akin to the different tasks attributed to the fly’s lobula plate tangential cell network, such as object avoidance or collision detection. Our implementation consists of a modified eneo SC-MVC01 SmartCam module and a custom built circuit board, weighing less than 200 g and consuming less than 4 W while featuring 57,600 individual two-dimensional elementary motion detectors, a 185° field of view and a frame rate of 350 frames per second. This makes our sensor system compact in terms of size, weight and power requirements for easy incorporation into MAV platforms, while autonomously performing all sensing and processing on-board and in real time.     

Light might suppress both types of sound‐evoked antipredator flight in moths

Urbanization exposes wild animals to increased levels of light, affecting particularly nocturnal animals. Artificial light at night might shift the balance of predator–prey interactions, for example, of nocturnal echolocating bats and eared moths. Moths exposed to light show less last‐ditch maneuvers in response to attacking close‐by bats. In contrast, the extent to which negative phonotaxis, moths’ first line of defense against distant bats, is affected by light is unclear. Here, we aimed to quantify the overall effect of light on both types of sound‐evoked antipredator flight, last‐ditch maneuvers and negative phonotaxis. We caught moths at two light traps, which were alternately equipped with loudspeakers that presented ultrasonic playbacks to simulate hunting bats. The light field was omnidirectional to attract moths equally from all directions. In contrast, the sound field was directional and thus, depending on the moth''s approach direction, elicited either only negative phonotaxis, or negative phonotaxis and last‐ditch maneuvers. We did not observe an effect of sound playback on the number of caught moths, suggesting that light might suppress both types of antipredator flight, as either type would have caused a decline in the number of caught moths. As control, we confirmed that our playback was able to elicit evasive flight in moths in a dark flight room. Showing no effect of a treatment, however, is difficult. We discuss potential alternative explanations for our results, and call for further studies to investigate how light interferes with animal behavior.     

Neuroethology of ultrasonic hearing in nocturnal butterflies (Hedyloidea)

Nocturnal Hedyloidea butterflies possess ultrasound-sensitive ears that mediate evasive flight maneuvers. Tympanal ear morphology, auditory physiology and behavioural responses to ultrasound are described for Macrosoma heliconiaria, and evidence for hearing is described for eight other hedylid species. The ear is formed by modifications of the cubital and subcostal veins at the forewing base, where the thin (1–3 μm), ovoid (520 × 220 μm) tympanal membrane occurs in a cavity. The ear is innervated by nerve IIN1c, with three chordotonal organs attaching to separate regions of the tympanal membrane. Extracellular recordings from IIN1c reveal sensory responses to ultrasonic (>20 kHz), but not low frequency (<10 kHz) sounds. Hearing is broadly tuned to frequencies between 40 and 80 kHz, with best thresholds around 60 dB SPL. Free flying butterflies exposed to ultrasound exhibit a variety of evasive maneuvers, characterized by sudden and unpredictable changes in direction, increased velocity, and durations of ∼500 ms. Hedylid hearing is compared to that of several other insects that have independently evolved ears for the same purpose-bat detection. Hedylid hearing may also represent an interesting example of evolutionary divergence, since we demonstrate that the ears are homologous to low frequency ears in some diurnal Nymphalidae butterflies.     

Accounting for Dynamic Fluctuations across Time when Examining fMRI Test-Retest Reliability: Analysis of a Reward Paradigm in the EMBARC Study

Longitudinal investigation of the neural correlates of reward processing in depression may represent an important step in defining effective biomarkers for antidepressant treatment outcome prediction, but the reliability of reward-related activation is not well understood. Thirty-seven healthy control participants were scanned using fMRI while performing a reward-related guessing task on two occasions, approximately one week apart. Two main contrasts were examined: right ventral striatum (VS) activation fMRI BOLD signal related to signed prediction errors (PE) and reward expectancy (RE). We also examined bilateral visual cortex activation coupled to outcome anticipation. Significant VS PE-related activity was observed at the first testing session, but at the second testing session, VS PE-related activation was significantly reduced. Conversely, significant VS RE-related activity was observed at time 2 but not time 1. Increases in VS RE-related activity from time 1 to time 2 were significantly associated with decreases in VS PE-related activity from time 1 to time 2 across participants. Intraclass correlations (ICCs) in VS were very low. By contrast, visual cortex activation had much larger ICCs, particularly in individuals with high quality data. Dynamic changes in brain activation are widely predicted, and failure to account for these changes could lead to inaccurate evaluations of the reliability of functional MRI signals. Conventional measures of reliability cannot distinguish between changes specified by algorithmic models of neural function and noisy signal. Here, we provide evidence for the former possibility: reward-related VS activations follow the pattern predicted by temporal difference models of reward learning but have low ICCs.     

Optic flow cues guide flight in birds

Although considerable effort has been devoted to investigating how birds migrate over large distances, surprisingly little is known about how they tackle so successfully the moment-to-moment challenges of rapid flight through cluttered environments [1]. It has been suggested that birds detect and avoid obstacles [2] and control landing maneuvers [3-5] by using cues derived from the image motion that is generated in the eyes during flight. Here we investigate the ability of budgerigars to fly through narrow passages in a collision-free manner, by filming their trajectories during flight in a corridor where the walls are decorated with various visual patterns. The results demonstrate, unequivocally and for the first time, that birds negotiate narrow gaps safely by balancing the speeds of image motion that are experienced by the two eyes and that the speed of flight is regulated by monitoring the speed of image motion that is experienced by the two eyes. These findings have close parallels with those previously reported for flying insects [6-13], suggesting that some principles of visual guidance may be shared by all diurnal, flying animals.     

How Lovebirds Maneuver Rapidly Using Super-Fast Head Saccades and Image Feature Stabilization

Diurnal flying animals such as birds depend primarily on vision to coordinate their flight path during goal-directed flight tasks. To extract the spatial structure of the surrounding environment, birds are thought to use retinal image motion (optical flow) that is primarily induced by motion of their head. It is unclear what gaze behaviors birds perform to support visuomotor control during rapid maneuvering flight in which they continuously switch between flight modes. To analyze this, we measured the gaze behavior of rapidly turning lovebirds in a goal-directed task: take-off and fly away from a perch, turn on a dime, and fly back and land on the same perch. High-speed flight recordings revealed that rapidly turning lovebirds perform a remarkable stereotypical gaze behavior with peak saccadic head turns up to 2700 degrees per second, as fast as insects, enabled by fast neck muscles. In between saccades, gaze orientation is held constant. By comparing saccade and wingbeat phase, we find that these super-fast saccades are coordinated with the downstroke when the lateral visual field is occluded by the wings. Lovebirds thus maximize visual perception by overlying behaviors that impair vision, which helps coordinate maneuvers. Before the turn, lovebirds keep a high contrast edge in their visual midline. Similarly, before landing, the lovebirds stabilize the center of the perch in their visual midline. The perch on which the birds land swings, like a branch in the wind, and we find that retinal size of the perch is the most parsimonious visual cue to initiate landing. Our observations show that rapidly maneuvering birds use precisely timed stereotypic gaze behaviors consisting of rapid head turns and frontal feature stabilization, which facilitates optical flow based flight control. Similar gaze behaviors have been reported for visually navigating humans. This finding can inspire more effective vision-based autopilots for drones.     

Emotion Separation Is Completed Early and It Depends on Visual Field Presentation

It is now apparent that the visual system reacts to stimuli very fast, with many brain areas activated within 100 ms. It is, however, unclear how much detail is extracted about stimulus properties in the early stages of visual processing. Here, using magnetoencephalography we show that the visual system separates different facial expressions of emotion well within 100 ms after image onset, and that this separation is processed differently depending on where in the visual field the stimulus is presented. Seven right-handed males participated in a face affect recognition experiment in which they viewed happy, fearful and neutral faces. Blocks of images were shown either at the center or in one of the four quadrants of the visual field. For centrally presented faces, the emotions were separated fast, first in the right superior temporal sulcus (STS; 35–48 ms), followed by the right amygdala (57–64 ms) and medial pre-frontal cortex (83–96 ms). For faces presented in the periphery, the emotions were separated first in the ipsilateral amygdala and contralateral STS. We conclude that amygdala and STS likely play a different role in early visual processing, recruiting distinct neural networks for action: the amygdala alerts sub-cortical centers for appropriate autonomic system response for fight or flight decisions, while the STS facilitates more cognitive appraisal of situations and links appropriate cortical sites together. It is then likely that different problems may arise when either network fails to initiate or function properly.     

Visuomotor Response to Object Expansion in Free-Flying Bumble Bees

The flight control systems of flying insects enable many kinds of sophisticated maneuvers, including avoidance of midair collisions. Visuomotor response to an approaching object, received as image expansion on insects’ retina, is a complex event in a dynamic environment where both animals and objects are moving. There are intensive free flight studies on the landing response in which insects receive image expansion by their own movement. However, few studies have been conducted regarding how freely flying insects respond to approaching objects. Here, using common laboratory insects for behavioral research, the bumblebee Bombus ignitus, we examined their visual response to an approaching object in the free-flying condition. While the insect was slowly flying in a free-flight arena, an expanding stripe was projected laterally from one side of the arena with a high-speed digital mirror device projector. Rather than turning away reported before, the bumble bees performed complex flight maneuvers. We synchronized flight trajectories, orientations and wing stroke frequencies with projection parameters of temporal resolution in 0.5 ms, and analyzed the instantaneous relationship between visual input and behavioral output. In their complex behavioral responses, we identified the following two visuomotor behaviors: increasing stroke frequency when the bumble bees confront the stripe expansion, and turning towards (not away) the stripe expansion when it is located laterally to the bee. Our results suggested that the response to object expansion is not a simple and reflexive escape but includes object fixation, presumably for subsequent behavioral choice.     

Variability in odor-modulated flight by moths

Based on previous studies of odor-modulated flight where track parameter data was lumped and averaged, the speed and orientation of the moths' movement along their flight tracks have been said to be controlled to maintain certain “preferred” values. The results from our fine-scaled analysis of this behavior show that none of the track parameters typically measured are held constant. The moths' speed along the flight track is modulated substantially and predictably: fastest along the straight legs and slowest around the turns. In addition, about half of the individuals studied progressively reduced the peak speed along the straight legs as they approached the pheromone source. While most of the track legs between the turns were directed upwind, their orientations were widely distributed, indicating no preferred direction. Small fluctuations of orientation along some straight legs suggest corrective maneuvers to stabilize flight direction about an internal set point. The visual inputs hypothesized to control steering and speed, transverse and longitudinal image flow, changed continuously during upwind flight in pheromone, but no regular relationship between them was observed. We found that the orientation of the longitudinal body axis and the direction of thrust (course angle) were only rarely coincident during upwind flight to the odor source, suggesting that moths receive sensory input which differs quantitatively from that calculated by conventional methods. Our results strongly suggest that the long-accepted hypothetical mechanisms of control for this behavior do not operate in the manner in which they have been proposed. Accepted: 11 July 1997     

Optic flow-based collision-free strategies: From insects to robots

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects’ abilities and better understanding their flight.     

Local and global motion preferences in descending neurons of the fly

For a moving animal, optic flow is an important source of information about its ego-motion. In flies, the processing of optic flow is performed by motion sensitive tangential cells in the lobula plate. Amongst them, cells of the vertical system (VS cells) have receptive fields with similarities to optic flows generated during rotations around different body axes. Their output signals are further processed by pre-motor descending neurons. Here, we investigate the local motion preferences of two descending neurons called descending neurons of the ocellar and vertical system (DNOVS1 and DNOVS2). Using an LED arena subtending 240° × 95° of visual space, we mapped the receptive fields of DNOVS1 and DNOVS2 as well as those of their presynaptic elements, i.e. VS cells 1–10 and V2. The receptive field of DNOVS1 can be predicted in detail from the receptive fields of those VS cells that are most strongly coupled to the cell. The receptive field of DNOVS2 is a combination of V2 and VS cells receptive fields. Predicting the global motion preferences from the receptive field revealed a linear spatial integration in DNOVS1 and a superlinear spatial integration in DNOVS2. In addition, the superlinear integration of V2 output is necessary for DNOVS2 to differentiate between a roll rotation and a lift translation of the fly.     

No neural evidence for dynamic auditory tuning of the A1 receptor in the ear of the noctuid moth, <Emphasis Type="Italic">Noctua pronuba</Emphasis>

By examining the mechanical properties of the tympanum of the noctuid moth, Noctua pronuba, Windmill et al. (2006) suggested that this insect increases (up-tunes) the frequencies of its best hearing when exposed to high intensity sounds (HIS) resembling the echolocation calls of attacking bats. We tested whether this biophysical phenomenon was encoded in the neural responses of this moth’s most sensitive auditory receptor (A1 cell) before and after exposure to HIS. We measured: (1) the number of A1 action potentials (spikes) per stimulus pulse; (2) the proportion of A1 spike periods below that determined to elicit evasive flight maneuvers and, (3) the change in A1 cell firing (spike number, interspike interval, stimulus/spike latency) over a duration of time similar to that in which up-tuning lasts. We observed no significant spiking response changes in the predicted direction to any of the frequencies tested following exposure to HIS and we observed only two of the 24 predicted time-dependent changes to A1 firing. These results indicate that tympanal up-tuning does not result in a change to this moth’s auditory frequency sensitivity and we suggest either sensillar resonances or increases in thoracic muscle tension following exposure to HIS as alternative explanations.     

Wind-evoked evasive responses in flying cockroaches

1. A standing cockroach (Periplaneta americana) responds to the air displacement made by an approaching predator, by turning away and running. The wind receptors on the cerci, two posterior sensory appendages, excite a group of ventral giant interneurons that mediate this response. While flying, these interneurons remain silent, owing to strong inhibition; however, the dorsal giant interneurons respond strongly to wind. Using behavioral and electromyographic analysis, we sought to determine whether flying cockroaches also turn away from air displacement like that produced by an approaching flying predator; and if so, whether the cerci and dorsal giant interneurons mediate this response.
2. When presented with a wind puff from the side, a flying cockroach carries out a variety of maneuvers that would cause a rapid turn away and perhaps a dive. These are not evoked if the cerci are ablated (Figs. 4, 5, 6).
3. This evasive response appears to be mediated by a circuit separate from that mediating escape when the cockroach is standing (Fig. 7).
4. The dorsal giant interneurons respond during flight in a directional manner that is suited to mediate this behavior (Fig. 8).
5. Recordings of the wind produced by a moving model predator (Fig. 9), together with measurements of the behavioral latency of tethered cockroaches, suggest that the evasive response would begin just milliseconds before a predator actually arrives. However, as explained in the Discussion section, under natural conditions, the evasive response may well begin earlier, and could indeed be useful in escaping from predators.
6. If cockroaches had a wind-mediated yaw-correcting behavior, as locusts have, this could conflict with the wind-evoked escape. In fact, cockroaches show the opposite, yaw-enhancing response, mediated by the cerci, that does not present a conflict with escape (Figs. 10–14).

     

Visual search and the importance of time in complex decision making by bees

Psychophysicists studying decision making in animals have overwhelmingly focused on choice accuracy, not speed. Results from human visual search, however, show that there might be a tight link between the two. Here we review both visual-sensory and cognitive mechanisms that affect decision speed in flower visiting bees. We show that decision times are affected by contrast of targets and background, by similarity between targets and distractors, numbers of distractors present in a scene, illuminating light intensity, presence or absence of punishment, and complexity of tasks. We explore between-individual and within-individual speed-accuracy tradeoffs, and show that bees resort to highly dynamic strategies when solving visual search tasks. Where possible, we attempt to link the observed search behaviour to the temporal and spatial properties of neuronal circuits underlying visual object detection. We demonstrate that natural foraging speed may not only be limited by factors such as food item density, flight energetics and scramble competition, as often implied. Our results show that understanding the behavioural ecology of foraging can substantially gain from knowledge about mechanisms of visual information processing.     

Response properties of motion-sensitive visual interneurons in the lobula plate of Drosophila melanogaster

The crystalline-like structure of the optic lobes of the fruit fly Drosophila melanogaster has made them a model system for the study of neuronal cell-fate determination, axonal path finding, and target selection. For functional studies, however, the small size of the constituting visual interneurons has so far presented a formidable barrier. We have overcome this problem by establishing in vivo whole-cell recordings from genetically targeted visual interneurons of Drosophila. Here, we describe the response properties of six motion-sensitive large-field neurons in the lobula plate that form a network consisting of individually identifiable, directionally selective cells most sensitive to vertical image motion (VS cells). Individual VS cell responses to visual motion stimuli exhibit all the characteristics that are indicative of presynaptic input from elementary motion detectors of the correlation type. Different VS cells possess distinct receptive fields that are arranged sequentially along the eye's azimuth, corresponding to their characteristic cellular morphology and position within the retinotopically organized lobula plate. In addition, lateral connections between individual VS cells cause strongly overlapping receptive fields that are wider than expected from their dendritic input. Our results suggest that motion vision in different dipteran fly species is accomplished in similar circuitries and according to common algorithmic rules. The underlying neural mechanisms of population coding within the VS cell network and of elementary motion detection, respectively, can now be analyzed by the combination of electrophysiology and genetic intervention in Drosophila.     

Responses of blowfly motion-sensitive neurons to reconstructed optic flow along outdoor flight paths

The retinal image flow a blowfly experiences in its daily life on the wing is determined by both the structure of the environment and the animal’s own movements. To understand the design of visual processing mechanisms, there is thus a need to analyse the performance of neurons under natural operating conditions. To this end, we recorded flight paths of flies outdoors and reconstructed what they had seen, by moving a panoramic camera along exactly the same paths. The reconstructed image sequences were later replayed on a fast, panoramic flight simulator to identified, motion sensitive neurons of the so-called horizontal system (HS) in the lobula plate of the blowfly, which are assumed to extract self-motion parameters from optic flow. We show that under real life conditions HS-cells not only encode information about self-rotation, but are also sensitive to translational optic flow and, thus, indirectly signal information about the depth structure of the environment. These properties do not require an elaboration of the known model of these neurons, because the natural optic flow sequences generate—at least qualitatively—the same depth-related response properties when used as input to a computational HS-cell model and to real neurons.     

Visual modelling suggests a weak relationship between the evolution of ultraviolet vision and plumage coloration in birds

Birds have sophisticated colour vision mediated by four cone types that cover a wide visual spectrum including ultraviolet (UV) wavelengths. Many birds have modest UV sensitivity provided by violet‐sensitive (VS) cones with sensitivity maxima between 400 and 425 nm. However, some birds have evolved higher UV sensitivity and a larger visual spectrum given by UV‐sensitive (UVS) cones maximally sensitive at 360–370 nm. The reasons for VS–UVS transitions and their relationship to visual ecology remain unclear. It has been hypothesized that the evolution of UVS‐cone vision is linked to plumage colours so that visual sensitivity and feather coloration are ‘matched’. This leads to the specific prediction that UVS‐cone vision enhances the discrimination of plumage colours of UVS birds while such an advantage is absent or less pronounced for VS‐bird coloration. We test this hypothesis using knowledge of the complex distribution of UVS cones among birds combined with mathematical modelling of colour discrimination during different viewing conditions. We find no support for the hypothesis, which, combined with previous studies, suggests only a weak relationship between UVS‐cone vision and plumage colour evolution. Instead, we suggest that UVS‐cone vision generally favours colour discrimination, which creates a nonspecific selection pressure for the evolution of UVS cones.     

Persistence and regression of hearing in the exclusively diurnal moths, Trichodezia albovittata (Geometridae) and Lycomorpha pholus (Arctiidae)

Abstract.  1. Auditory sensitivities and ultrasound avoidance behaviour of two exclusively diurnal moths were examined to test the prediction that total isolation from the predatory effects of echolocating bats will result in the regression of these sensory systems and/or the defences they evoke.
2. The silent geometrid, Trichodezia albovittata , possessed large ears with auditory neural thresholds similar to or better than those of a sympatric, exclusively nocturnal geometrid moth. Trichodezia albovittata readily responded with evasive flight to ultrasound and it is suggested that if this moth has become completely isolated from bats its ears are functionally vestigial, at least in the population studied here.
3. In contrast, while the sound-producing arctiid, Lycomorpha pholus , had low auditory sensitivity based on neural thresholds, it still responded with flight changes to ultrasound. It did not, however, produce sounds when stimulated ultrasonically. It is suggested that the ears of this moth are functionally vestigial for bat-detection purposes but may be used for short-distance social communication.     

Spatial and temporal dynamics of attentional guidance during inefficient visual search

Spotting a prey or a predator is crucial in the natural environment and relies on the ability to extract quickly pertinent visual information. The experimental counterpart of this behavior is visual search (VS) where subjects have to identify a target amongst several distractors. In difficult VS tasks, it has been found that the reaction time (RT) is influenced by salience factors, such as the target-distractor similarity, and this finding is usually regarded as evidence for a guidance of attention by preattentive mechanisms. However, the use of RT measurements, a parameter which depends on multiple factors, allows only very indirect inferences about the underlying attentional mechanisms. The purpose of the present study was to determine the influence of salience factors on attentional guidance during VS, by measuring directly attentional allocation. We studied attention allocation by using a dual covert VS task in subjects who had 1) to detect a target amongst different items and 2) to report letters briefly flashed inside those items at different delays. As predicted, we showed that parallel processes guide attention towards the most relevant item by virtue of both goal-directed and stimulus-driven factors, and we demonstrated that this attentional selection is a prerequisite for target detection. In addition, we show that when the target is characterized by two features (conjunction VS), the goal-directed effects of both features are initially combined into a unique salience value, but at a later stage, grouping phenomena interact with the salience computation, and lead to the selection of a whole group of items. These results, in line with Guided Search Theory, show that efficient and rapid preattentive processes guide attention towards the most salient item, allowing to reduce the number of attentional shifts needed to find the target.     

High duty cycle pulses suppress orientation flights of crambid moths

Bat-and-moth is a good model system for understanding predator–prey interactions resulting from interspecific coevolution. Night-flying insects have been under predation pressure from echolocating bats for 65 Myr, pressuring vulnerable moths to evolve ultrasound detection and evasive maneuvers as counter tactics. Past studies of defensive behaviors against attacking bats have been biased toward noctuoid moth responses to short duration pulses of low-duty-cycle (LDC) bat calls. Depending on the region, however, moths have been exposed to predation pressure from high-duty-cycle (HDC) bats as well. Here, we reveal that long duration pulse of the sympatric HDC bat (e.g., greater horseshoe bat) is easily detected by the auditory nerve of Japanese crambid moths (yellow peach moth and Asian corn borer) and suppress both mate-finding flights of virgin males and host-finding flights of mated females. The hearing sensitivities for the duration of pulse stimuli significantly dropped non-linearly in both the two moth species as the pulse duration shortened. These hearing properties support the energy integrator model; however, the threshold reduction per doubling the duration has slightly larger than those of other moth species hitherto reported. And also, Asian corn borer showed a lower auditory sensitivity and a lower flight suppression to short duration pulse than yellow peach moth did. Therefore, flight disruption of moth might be more frequently achieved by the pulse structure of HDC calls. The combination of long pulses and inter-pulse intervals, which moths can readily continue detecting, will be useful for repelling moth pests.