Optic flow-based collision-free strategies: From insects to robots

Flying insects are able to fly smartly in an unpredictable environment. It has been found that flying insects have smart neurons inside their tiny brains that are sensitive to visual motion also called optic flow. Consequently, flying insects rely mainly on visual motion during their flight maneuvers such as: takeoff or landing, terrain following, tunnel crossing, lateral and frontal obstacle avoidance, and adjusting flight speed in a cluttered environment. Optic flow can be defined as the vector field of the apparent motion of objects, surfaces, and edges in a visual scene generated by the relative motion between an observer (an eye or a camera) and the scene. Translational optic flow is particularly interesting for short-range navigation because it depends on the ratio between (i) the relative linear speed of the visual scene with respect to the observer and (ii) the distance of the observer from obstacles in the surrounding environment without any direct measurement of either speed or distance. In flying insects, roll stabilization reflex and yaw saccades attenuate any rotation at the eye level in roll and yaw respectively (i.e. to cancel any rotational optic flow) in order to ensure pure translational optic flow between two successive saccades. Our survey focuses on feedback-loops which use the translational optic flow that insects employ for collision-free navigation. Optic flow is likely, over the next decade to be one of the most important visual cues that can explain flying insects' behaviors for short-range navigation maneuvers in complex tunnels. Conversely, the biorobotic approach can therefore help to develop innovative flight control systems for flying robots with the aim of mimicking flying insects’ abilities and better understanding their flight.     

Aphid flight-track analysis in three dimensions using video techniques

Abstract. A technique is described for the three-dimensional analysis of the flight paths of small insects using two video cameras placed alongside each other with the optical axes coincident at a point some distance beyond the area of interest. The video signals were mixed and a time base introduced before recording the superimposed images from both cameras on a single VCR. With suitable lighting and a black background, flying aphids appeared on the monitor as double, bright images on a dark background. The distance between the two images was inversely proportional to the distance of the aphid from the camera lenses. A calibration grid was used to insert the correct parameters into software designed to provide the x (vertical), y (horizontal) and z (distance from the cameras) Cartesian co-ordinates for a flying insect and to calculate the distances flown, flight speed and turning parameters. The advantages of the system are that it is designed for a single VCR and monitor, provides automatic synchrony between camera signals and can examine a larger visual arena than screen-splitting methods. It operates with insects as small as aphids, and wind-tunnel studies on the black bean aphid, Aphis fabae Scop., showed that some flight parameters (for the final one-second approach to a visually attractive landing platform) differed according to whether wind was present or not. Thus, ground speed and distance moved differed significantly but turning parameters were unchanged. In addition, flight trajectory on the approach to landing depended upon initial direction of flight and the presence of wind.     

A Computer-Controlled Video System for Real-Time Recording of Insect Flight in Three Dimensions

A computer-controlled video system for real-time recording of insect flight in three dimensions is described. The flight paths of moths were recorded in a flight tunnel using two CCD cameras placed adjacent to each other at angles of 45 and 135° to the flight tunnel axis and separated by a distance of 120 cm. They were connected to two 2⁸-level gray-scale frame grabbers via two external synchronizers. The two-dimensional coordinates of the flying insect were obtained from the two cameras at 40-ms intervals and transferred to host computer for processing and monitor for real-time display. Due to speed limitation in the image acquisition hardware, construction of the three-dimensional file was carried off-line. The flying insect was rendered as a dark spot in a bright background using a homogeneous light source. As the insect enters into the field of view of the two cameras, the light distribution changes, and the frame grabber detects only those variation in the light distribution which results from a flying insect. The target insect can be as small as 3 pixels and can be tracked in a stereoscopic field of view 60 cm long and 50 cm high. A method was developed that allowed for scalar scoring of various pheromone sources to assess their attractiveness using vector flight parameters. This method was applied successfully for optimization of pheromone blend of the grapevine moth, Lobesia botrana.     

Nocturnal insects use optic flow for flight control

To avoid collisions when navigating through cluttered environments, flying insects must control their flight so that their sensory systems have time to detect obstacles and avoid them. To do this, day-active insects rely primarily on the pattern of apparent motion generated on the retina during flight (optic flow). However, many flying insects are active at night, when obtaining reliable visual information for flight control presents much more of a challenge. To assess whether nocturnal flying insects also rely on optic flow cues to control flight in dim light, we recorded flights of the nocturnal neotropical sweat bee, Megalopta genalis, flying along an experimental tunnel when: (i) the visual texture on each wall generated strong horizontal (front-to-back) optic flow cues, (ii) the texture on only one wall generated these cues, and (iii) horizontal optic flow cues were removed from both walls. We find that Megalopta increase their groundspeed when horizontal motion cues in the tunnel are reduced (conditions (ii) and (iii)). However, differences in the amount of horizontal optic flow on each wall of the tunnel (condition (ii)) do not affect the centred position of the bee within the flight tunnel. To better understand the behavioural response of Megalopta, we repeated the experiments on day-active bumble-bees (Bombus terrestris). Overall, our findings demonstrate that despite the limitations imposed by dim light, Megalopta-like their day-active relatives-rely heavily on vision to control flight, but that they use visual cues in a different manner from diurnal insects.     

Visuomotor Response to Object Expansion in Free-Flying Bumble Bees

The flight control systems of flying insects enable many kinds of sophisticated maneuvers, including avoidance of midair collisions. Visuomotor response to an approaching object, received as image expansion on insects’ retina, is a complex event in a dynamic environment where both animals and objects are moving. There are intensive free flight studies on the landing response in which insects receive image expansion by their own movement. However, few studies have been conducted regarding how freely flying insects respond to approaching objects. Here, using common laboratory insects for behavioral research, the bumblebee Bombus ignitus, we examined their visual response to an approaching object in the free-flying condition. While the insect was slowly flying in a free-flight arena, an expanding stripe was projected laterally from one side of the arena with a high-speed digital mirror device projector. Rather than turning away reported before, the bumble bees performed complex flight maneuvers. We synchronized flight trajectories, orientations and wing stroke frequencies with projection parameters of temporal resolution in 0.5 ms, and analyzed the instantaneous relationship between visual input and behavioral output. In their complex behavioral responses, we identified the following two visuomotor behaviors: increasing stroke frequency when the bumble bees confront the stripe expansion, and turning towards (not away) the stripe expansion when it is located laterally to the bee. Our results suggested that the response to object expansion is not a simple and reflexive escape but includes object fixation, presumably for subsequent behavioral choice.     

Active Sensing System with In Situ Adjustable Sensor Morphology

Background

Despite the widespread use of sensors in engineering systems like robots and automation systems, the common paradigm is to have fixed sensor morphology tailored to fulfill a specific application. On the other hand, robotic systems are expected to operate in ever more uncertain environments. In order to cope with the challenge, it is worthy of note that biological systems show the importance of suitable sensor morphology and active sensing capability to handle different kinds of sensing tasks with particular requirements.

Methodology

This paper presents a robotics active sensing system which is able to adjust its sensor morphology in situ in order to sense different physical quantities with desirable sensing characteristics. The approach taken is to use thermoplastic adhesive material, i.e. Hot Melt Adhesive (HMA). It will be shown that the thermoplastic and thermoadhesive nature of HMA enables the system to repeatedly fabricate, attach and detach mechanical structures with a variety of shape and size to the robot end effector for sensing purposes. Via active sensing capability, the robotic system utilizes the structure to physically probe an unknown target object with suitable motion and transduce the arising physical stimuli into information usable by a camera as its only built-in sensor.

Conclusions/Significance

The efficacy of the proposed system is verified based on two results. Firstly, it is confirmed that suitable sensor morphology and active sensing capability enables the system to sense different physical quantities, i.e. softness and temperature, with desirable sensing characteristics. Secondly, given tasks of discriminating two visually indistinguishable objects with respect to softness and temperature, it is confirmed that the proposed robotic system is able to autonomously accomplish them. The way the results motivate new research directions which focus on in situ adjustment of sensor morphology will also be discussed.     

烟粉虱的飞行行为与害虫综合治理策略

烟粉虱具有较强的飞行潜力,飞行高度可以超过150 m,在田间的扩散距离最远可以超过150 km,但在食源丰富地区,较少作远距离的扩散,绝大部分飞行高度在距地面0.5 m左右.烟粉虱具有搜索飞行和迁飞飞行特性,它是寻找适宜寄主和扩大生境的重要方式.烟粉虱不具备“卵子发生与飞行共轭”的典型特征.可见光、温湿度、寄主质量和风等是影响烟粉虱飞行行为的重要生态因子.本文对烟粉虱的飞行能力、飞行生理学和影响飞行的生态因子进行了综述,对非露地越冬区利用烟粉虱的飞行特性实施IPM策略进行了探讨.     

Weight Loading and Reproductive Status Affect the Flight Performance of Pieris napi Butterflies

Weight-induced mobility reductions can have dramatic fitness consequences and winged animals are especially sensitive to the trade-off between mass and locomotion. Data on how natural weight fluctuations influence a flying insect’s ability to take off are scarce. We therefore quantified take-off flight ability in Pieris napi butterflies in relation to reproductive status. Take-off flight ability (velocity and take-off angle) under suboptimal temperature conditions was recorded with a 3D-tracking camera system and was predicted to decrease with relatively larger weight loads. Our results show that relatively larger weight loads generally reduce flight speed in male butterflies and lower take-off angles in females. However, despite having a lower wing loading, mated male butterflies flew slower than unmated males. Our study suggests that retention of weight loads associated with reproduction impairs insect flight performance.     

An analog VLSI implementation of a visual interneuron: enhanced sensory processing through biophysical modeling

Flies are capable of rapid, coordinated flight through unstructured environments. This flight is guided by visual motion information that is extracted from photoreceptors in a robust manner. One feature of the fly's visual processing that adds to this robustness is the saturation of wide-field motion-sensitive neuron responses with increasing pattern size. This makes the cell's responses less dependent on the sparseness of the optical flow field while retaining motion information. By implementing a compartmental neuronal model in silicon, we add this "gain control" to an existing analog VLSI model of fly vision. This results in enhanced performance in a compact, low-power CMOS motion sensor. Our silicon system also demonstrates that modern, biophysically-detailed models of neural sensory processing systems can be instantiated in VLSI hardware.     

On the role of emotion in biological and robotic autonomy

This paper reviews some of the differences between notions of biological and robotic autonomy, and how these differences have been reflected in discussions of embodiment, grounding and other concepts in AI and autonomous robotics. Furthermore, the relations between homeostasis, emotion and embodied cognition are discussed as well as recent proposals to model their interplay in robots, which reflects a commitment to a multi-tiered affectively/emotionally embodied view of mind that takes organismic embodiment more serious than usually done in biologically inspired robotics.     

A Simulation of the Flight Characteristics of the Deployable Hindwings of Beetle

An insect is an excellent biological object for the bio-inspirations to design and develop a MAV.This paper presents the simulation study of the flight characteristics of the deployable hindwings of beetle,Dorcustitanus platymelus.A 3D geometric model of the beetle was obtained using a 3D laser scanning technique.By studying its hindwings and flight mechanism,the mathematical model of the flapping motion of its hindwings was analyzed.Then a simulation analysis was carried out to analyze and evaluate the flapping flying aerodynamic characteristics.After that,the flow of blood in the hindwing veins was studied through simulation to determine the maximum pressure on a vein surface and the minimum blood flow in flight.A number of interesting bio-inspirations were obtained.It is believed that these findings can be used for the design and development of a MAV with similar flying capabilities to a natural beetle.     

Effects of pheromone plume structure and visual stimuli on the pheromone-modulated upwind flight of male gypsy moths (Lymantria dispar) in a Forest (Lepidoptera: Lymantriidae)

The pheromone-modulated upwind flight ofLymantria dispar males responding to different pheromone plume structures and visual stimuli designed to mimic trees was video recorded in a forest. Males flying upwind along pheromone plumes of similar structure generated tracks that were similar in appearance and quantitatively similar in almost all parameters measured, regardless of the experimentally manipulated visual stimuli associated with the pheromone source. Net velocities, ground speeds, and airspeeds of males flying in point-source plumes were slower than those of males flying in the wider, more diffuse plumes issuing from a cylindrical baffle. The mean track angle of males flying in plumes issuing from a point source was greater (oriented more across the wind) than that of males flying in plumes issuing from a transparent cylindrical baffle. Males flying in point-source plumes also turned more frequently and had narrower tracks overall than males responding to plumes from a cylindrical baffle. These data suggest thatL. dispar males orienting to pheromone sources (i.e., calling females) associated with visible vertical cylinders (i.e., trees) use predominantly olfactory cues to locate the source and that the structure of the pheromone plume markedly affects the flight orientation and the resultant track.     

A low‐cost,computer‐controlled robotic flower system for behavioral experiments

Human observations during behavioral studies are expensive, time‐consuming, and error prone. For this reason, automatization of experiments is highly desirable, as it reduces the risk of human errors and workload. The robotic system we developed is simple and cheap to build and handles feeding and data collection automatically. The system was built using mostly off‐the‐shelf components and has a novel feeding mechanism that uses servos to perform refill operations. We used the robotic system in two separate behavioral studies with bumblebees (Bombus terrestris): The system was used both for training of the bees and for the experimental data collection. The robotic system was reliable, with no flight in our studies failing due to a technical malfunction. The data recorded were easy to apply for further analysis. The software and the hardware design are open source. The development of cheap open‐source prototyping platforms during the recent years has opened up many possibilities in designing of experiments. Automatization not only reduces workload, but also potentially allows experimental designs never done before, such as dynamic experiments, where the system responds to, for example, learning of the animal. We present a complete system with hardware and software, and it can be used as such in various experiments requiring feeders and collection of visitation data. Use of the system is not limited to any particular experimental setup or even species.     

Visual behaviour of the greenhouse whitefly, Trialeurodes vaporariorum

ABSTRACT. The behaviour of Trialeurodes vaporariorum (Westwood) (Homoptera, Aleyrodidae) in violet and green light (400 and 550 nm) was examined using several responses. Under 400 nm the whiteflies took-off more readily and walked faster than under 500 nm. In flight, they oriented towards 400 nm when simultaneously illuminated with equal quanta of 550 and 400 nm light. The ecological significance of this behaviour is discussed, and it is concluded that in nature flying adults would orient towards the sky (i.e. c. 400 nm) but would tend to land on a green plant because plants reflect maximally at 550 nm. Once landed on a suitable food-plant the position on that plant where the insect finally feeds and reproduces is probably also determined by visual stimuli, since whiteflies will walk to the shaded side of a leaf regardless of whether that is below or above.     

Behavioural circadian rhythms measured in real-time by automatic image analysis: applications in parasitoid insects

Abstract. A video analysis system was developed to study behavioural circadian rhythms in insects. Technical innovations were performed in image analysis and in the use of robotics. In image analysis, both the automatic detection of the insect and the calculation of several variables (linear and angular speed, locomotor activity, etc.) are carried out in real-time, which saves time and allows long-term studies without any intervention of the operator. The displacement of the camera by robotics allows the simultaneous study of many individuals (i.e. sixty individuals for the study of locomotor activity of adult insects, more than 400 individuals for the study of the emergence of adult parasitoids from their host). Four applications of this new device are presented. They deal with circadian rhythms in parasitoid insects: locomotor activity and time variation in locomotory parameters, daily distribution of the emergence and onset of the locomotor activity in adults. The adaptability of the system to several experimental materials and situations is discussed and the interest of the study of behavioural rhythms in the approach of host-parasitoid associations is emphasized.     

Intrinsic activity in the fly brain gates visual information during behavioral choices

The small insect brain is often described as an input/output system that executes reflex-like behaviors. It can also initiate neural activity and behaviors intrinsically, seen as spontaneous behaviors, different arousal states and sleep. However, less is known about how intrinsic activity in neural circuits affects sensory information processing in the insect brain and variability in behavior. Here, by simultaneously monitoring Drosophila's behavioral choices and brain activity in a flight simulator system, we identify intrinsic activity that is associated with the act of selecting between visual stimuli. We recorded neural output (multiunit action potentials and local field potentials) in the left and right optic lobes of a tethered flying Drosophila, while its attempts to follow visual motion (yaw torque) were measured by a torque meter. We show that when facing competing motion stimuli on its left and right, Drosophila typically generate large torque responses that flip from side to side. The delayed onset (0.1-1 s) and spontaneous switch-like dynamics of these responses, and the fact that the flies sometimes oppose the stimuli by flying straight, make this behavior different from the classic steering reflexes. Drosophila, thus, seem to choose one stimulus at a time and attempt to rotate toward its direction. With this behavior, the neural output of the optic lobes alternates; being augmented on the side chosen for body rotation and suppressed on the opposite side, even though the visual input to the fly eyes stays the same. Thus, the flow of information from the fly eyes is gated intrinsically. Such modulation can be noise-induced or intentional; with one possibility being that the fly brain highlights chosen information while ignoring the irrelevant, similar to what we know to occur in higher animals.     

To what extent can the tiny parasitoid wasps,Eretmocerus mundus,fly upwind?

Body miniaturization in insects is predicted to result in decreased flight speed and therefore limited ability of these insects to fly upwind. Therefore, tiny insects are often regarded as relying on passive dispersal by winds. We tested this assumption in a wind tunnel by measuring the burst speed of Eretmocerus mundus (Mercet), a beneficial parasitoid wasp with body length <1 mm. Insects were filmed flying upwind towards a UV light source in a range of wind speed 0–0.5 m/s. The Insects flew towards the UV light in the absence and presence of wind but increased their flight speed in the presence of wind. They also changed flight direction to be directly upwind and maintained this body orientation even while drifted backwards relative to the ground by stronger winds. Field measurements showed that the average flight speed observed in the wind tunnel (0.3 m/s) is sufficient to allow flying between plants even when the wind speed above the vegetation was 3–5 folds higher. A simulation of the ability of the insects to control their flight trajectory towards a visual target (sticky traps) in winds show that the insects can manipulate their progress relative to the ground even when the wind speed exceeds their flight speed. The main factors determining the ability of the insects to reach the trap were trap diameter and the difference between insect flight speed and wind speed. The simulation also predicts the direction of arrival to the sticky target showing that many of the insects reach the target from the leeward side (i.e. by flight upwind). In light of these results, the notion that miniature insects passively disperse by winds is misleading because it disregards the ability of the insects to control their drift relative to the ground in winds that are faster than their flight speed.     

The High Throughput Screening Infrastructure: The Right Tools for the Task

HTS is a key component of pharmaceutical lead identification process. Over recent years, the pharmaceutical industry has experienced significant increases in the throughput capabilities of its HTS functions. In those companies where HTS has been effectively deployed, it is now possible to screen the entire corporate compound collection against a pharmacological target within a timescale of several weeks to a few months. This capability has been realized, not as a result of the purchase of any one particular piece of hardware, but rather through the development of a truly effective HTS infrastructure that matches the needs of the parent organization. Central to this is the need to understand how to effectively combine the use of the different types of hardware available to the HTS specialist. The use of both modular workstations and single-arm robotic systems have underpinned most HTS groups operations. Recent advances in the field of multiple-arm robotic systems and dedicated automation systems offer even further potential for increasing productivity. This article describes our experience with the use of a dedicated automation system for HTS applications.     

Bio-inspired grasp control in a robotic hand with massive sensorial input

The capability of grasping and lifting an object in a suitable, stable and controlled way is an outstanding feature for a robot, and thus far, one of the major problems to be solved in robotics. No robotic tools able to perform an advanced control of the grasp as, for instance, the human hand does, have been demonstrated to date. Due to its capital importance in science and in many applications, namely from biomedics to manufacturing, the issue has been matter of deep scientific investigations in both the field of neurophysiology and robotics. While the former is contributing with a profound understanding of the dynamics of real-time control of the slippage and grasp force in the human hand, the latter tries more and more to reproduce, or take inspiration by, the nature’s approach, by means of hardware and software technology. On this regard, one of the major constraints robotics has to overcome is the real-time processing of a large amounts of data generated by the tactile sensors while grasping, which poses serious problems to the available computational power. In this paper a bio-inspired approach to tactile data processing has been followed in order to design and test a hardware–software robotic architecture that works on the parallel processing of a large amount of tactile sensing signals. The working principle of the architecture bases on the cellular nonlinear/neural network (CNN) paradigm, while using both hand shape and spatial–temporal features obtained from an array of microfabricated force sensors, in order to control the sensory-motor coordination of the robotic system. Prototypical grasping tasks were selected to measure the system performances applied to a computer-interfaced robotic hand. Successful grasps of several objects, completely unknown to the robot, e.g. soft and deformable objects like plastic bottles, soft balls, and Japanese tofu, have been demonstrated.     

Current capabilities of rehabilitation robots

Predictions are often made of intelligent and independently mobile robots for the disabled, and researchers are continually improving laboratory systems. Reductions in the cost of the technology involved may lead to affordable devices by the end of the decade. Less ambitious goals must be adopted by those projects wishing to distribute robotic aids to the disabled in the next few years. A modest selling price dictates the use of existing components. Even with the advent of more advanced robots, cost considerations may still make simpler devices an attractive alternative. Excessive optimism of future capabilities should be avoided, lest unrealistic expectations of current robotic aids hamper their development. Progress at all levels of rehabilitation robotics is complementary.