Biomimetic Sensors: Active Electrolocation of Weakly Electric Fish as a Model for Active Sensing in Technical Systems

Instead of vision, many nocturnal animals use alternative senses for navigation and object detection in their dark environment. For this purpose, weakly electric mormyrid fish employ active electrolocation, during which they discharge a specialized electric organ in their tail which discharges electrical pulses. Each discharge builds up an electrical field around the fish, which is sensed by cutaneous electroreceptor organs that are distributed over most of the body surface of the fish. Nearby objects distort this electrical field and cause a local alteration in current flow in those electroreceptors that are closest to the object. By constantly monitoring responses of its electroreceptor organs, a fish can detect, localize, and identify environmental objects.Inspired by the remarkable capabilities of weakly electric fish in detecting and recognizing objects, we designed technical sensor systems that can solve similar problems of remote object sensing. We applied the principles of active electrolocation to technical systems by building devices that produce electrical current pulses in a conducting medium (water or ionized gases) and simultaneously sense local current density. Depending on the specific task a sensor was designed for devices could (i) detect an object, (ii) localize it in space, (iii) determine its distance, and (iv) measure properties such as material properties, thickness, or material faults. Our systems proved to be relatively insensitive to environmental disturbances such as heat, pressure, or turbidity. They have a wide range of applications including material identification, quality control, non-contact distance measurements, medical applications and many more. Despite their astonishing capacities, our sensors still lag far behind what electric fish are able to achieve during active electrolocation. The understanding of the neural principles governing electric fish sensory physiology and the corresponding optimization of our sensors to solve certain technical tasks therefore remain ongoing goals of our research.     

Biomimetic Sensors： Active Electrolocation of Weakly Electric Fish as a Model for Active Sensing in Technical Systems

Instead of vision, many nocturnal animals use alternative senses for navigation and object detection in their dark environment. For this purpose, weakly electric mormyrid fish employ active electrolocation, during which they discharge a specialized electric organ in their tail which discharges electrical pulses. Each discharge builds up an electrical field around the fish, which is sensed by cutaneous electroreceptor organs that are distributed over most of the body surface of the fish. Nearby objects distort this electrical field and cause a local alteration in current flow in those electroreceptors that are closest to the object. By constantly monitoring responses of its electroreceptor organs, a fish can detect, localize, and identify environmental objects.Inspired by the remarkable capabilities of weakly electric fish in detecting and recognizing objects, we designed technical sensor systems that can solve similar problems of remote object sensing. We applied the principles of active electrolocation to technical systems by building devices that produce electrical current pulses in a conducting medium (water or ionized gases) and simultaneously sense local current density. Depending on the specific task a sensor was designed for devices could (i) detect an object, (ii) localize it in space, (iii) determine its distance, and (iv) measure properties such as material properties, thickness, or material faults. Our systems proved to be relatively insensitive to environmental disturbances such as heat, pressure, or turbidity. They have a wide range of applications including material identification, quality control, non-contact distance measurements, medical applications and many more. Despite their astonishing capacities, our sensors still lag far behind what electric fish are able to achieve during active electrolocation. The understanding of the neural principles governing electric fish sensory physiology and the corresponding optimization of our sensors to solve certain technical tasks therefore remain ongoing goals of our research.     

Parsimony,Exhaustivity and Balanced Detection in Neocortex

The layout of sensory brain areas is thought to subtend perception. The principles shaping these architectures and their role in information processing are still poorly understood. We investigate mathematically and computationally the representation of orientation and spatial frequency in cat primary visual cortex. We prove that two natural principles, local exhaustivity and parsimony of representation, would constrain the orientation and spatial frequency maps to display a very specific pinwheel-dipole singularity. This is particularly interesting since recent experimental evidences show a dipolar structures of the spatial frequency map co-localized with pinwheels in cat. These structures have important properties on information processing capabilities. In particular, we show using a computational model of visual information processing that this architecture allows a trade-off in the local detection of orientation and spatial frequency, but this property occurs for spatial frequency selectivity sharper than reported in the literature. We validated this sharpening on high-resolution optical imaging experimental data. These results shed new light on the principles at play in the emergence of functional architecture of cortical maps, as well as their potential role in processing information.     

Short-range orientation in fish: How fish map space

One way in which fish can move around efficiently is to learn and remember a spatial map of their environment. This can be a relatively simple process where, for example, sequences of landmarks are learned. However, more complex spatial representations can be generated by integrating multiple pieces of information. In this review, we consider what types of information fish use to generate a spatial map; for instance, beacons (single landmarks) that signal a specific location, or learned geometric relationships between multiple landmarks that allow fish to guide their movements. Owing to the diversity of fish species and the broad range of environments that they inhabit, there is considerable diversity in the maps that they develop and the sensory systems that they use to detect spatial information. This chapter uses a series of examples to investigate the types of spatial information that fish encode, for instance, how they map three-dimensional space, how they make use of different sensory modalities, and where this information might be processed. We also highlight the versatility of short-range orientation in fish, and discuss a number of similarities between the mapping mechanisms used by fish and terrestrial vertebrates.     

Information-processing demands in electrosensory and mechanosensory lateral line systems.

The electrosensory and mechanosensory lateral line systems of fish exhibit many common features in their structural and functional organization, both at the sensory periphery as well as in central processing pathways. These two sensory systems also appear to play similar roles in many behavioral tasks such as prey capture, orientation with respect to external environmental cues, navigation in low-light conditions, and mediation of interactions with nearby animals. In this paper, we briefly review key morphological, physiological, and behavioral aspects of these two closely related sensory systems. We present arguments that the information processing demands associated with spatial processing are likely to be quite similar, due largely to the spatial organization of both systems and the predominantly dipolar nature of many electrosensory and mechanosensory stimulus fields. Demands associated with temporal processing may be quite different, however, due primarily to differences in the physical bases of electrosensory and mechanosensory stimuli (e.g. speed of transmission). With a better sense of the information processing requirements, we turn our attention to an analysis of the functional organization of the associated first-order sensory nuclei in the hindbrain, including the medial octavolateral nucleus (MON), dorsal octavolateral nucleus (DON), and electrosensory lateral line lobe (ELL). One common feature of these systems is a set of neural mechanisms for improving signal-to-noise ratios, including mechanisms for adaptive suppression of reafferent signals. This comparative analysis provides new insights into how the nervous system extracts biologically significant information from dipolar stimulus fields in order to solve a variety of behaviorally relevant problems faced by aquatic animals.     

Sensing scenes with silicon

Scene analysis, the process of converting sensory information from peripheral receptors into a representation of objects in the external world, is central to our human experience of perception. Through our efforts to design systems for object recognition and for robot navigation, we have come to appreciate that a number of common themes apply across the sensory modalities of vision, audition, and olfaction; and many apply across species ranging from invertebrates to mammals. These themes include the need for adaptation in the periphery and trade-offs between selectivity for frequency or molecular structure with resolution in time or space. In addition, neural mechanisms involving coincidence detection are found in many different subsystems that appear to implement cross-correlation or autocorrelation computations.     

Sensory integration in the hydrodynamic world of rainbow trout

Water movements, of both abiotic and biotic origin, provide a wealth of information for fishes. They detect these water movements by arrays of hydrodynamic sensors located on the surface of the body as superficial neuromasts and embedded in subdermal lateral line canals. Recently, the anatomical dichotomy between superficial and canal neuromasts has been matched by demonstrations of a corresponding functional dichotomy. Superficial neuromasts are sensitive to water flows over the surface of the fish and are the sub-modality that participates in orientation to water currents, a behaviour known as rheotaxis. The canal neuromasts are sensitive to water vibration and it is this sub-modality that determines the localization of artificial prey. Recently, however, it has been shown that the complex behaviour of natural prey capture in the dark requires input from both lateral line sensory submodalities and here we show that the ability of trout to hold station behind a stationary object in fast flowing water also requires integration of information from both sub-modalities.     

Crossmodal integration for perception and action.

The integration of information from different sensory modalities has many advantages for human observers, including increase of salience, resolution of perceptual ambiguities, and unified perception of objects and surroundings. Several behavioral, electrophysiological and neuroimaging data collected in various tasks, including localization and detection of spatial events, crossmodal perception of object properties and scene analysis are reviewed here. All the results highlight the multiple faces of crossmodal interactions and provide converging evidence that the brain takes advantages of spatial and temporal coincidence between spatial events in the crossmodal binding of spatial features gathered through different modalities. Furthermore, the elaboration of a multimodal percept appears to be based on an adaptive combination of the contribution of each modality, according to the intrinsic reliability of sensory cue, which itself depends on the task at hand and the kind of perceptual cues involved in sensory processing. Computational models based on bayesian sensory estimation provide valuable explanations of the way perceptual system could perform such crossmodal integration. Recent anatomical evidence suggest that crossmodal interactions affect early stages of sensory processing, and could be mediated through a dynamic recurrent network involving backprojections from multimodal areas as well as lateral connections that can modulate the activity of primary sensory cortices, though future behavioral and neurophysiological studies should allow a better understanding of the underlying mechanisms.     

Object localization with whiskers

Rats use their large facial hairs (whiskers) to detect, localize and identify objects in their proximal three-dimensional (3D) space. Here, we focus on recent evidence of how object location is encoded in the neural sensory pathways of the rat whisker system. Behavioral and neuronal observations have recently converged to the point where object location in 3D appears to be encoded by an efficient orthogonal scheme supported by primary sensory-afferents: each primary-afferent can signal object location by a spatial (labeled-line) code for the vertical axis (along whisker arcs), a temporal code for the horizontal axis (along whisker rows), and an intensity code for the radial axis (from the face out). Neuronal evidence shows that (i) the identities of activated sensory neurons convey information about the vertical coordinate of an object, (ii) the timing of their firing, in relation to other reference signals, conveys information about the horizontal object coordinate, and (iii) the intensity of firing conveys information about the radial object coordinate. Such a triple-coding scheme allows for efficient multiplexing of 3D object location information in the activity of single neurons. Also, this scheme provides redundancy since the same information may be represented in the activity of many neurons. These features of orthogonal coding increase accuracy and reliability. We propose that the multiplexed information is conveyed in parallel to different readout circuits, each decoding a specific spatial variable. Such decoding reduces ambiguity, and simplifies the required decoding algorithms, since different readout circuits can be optimized for a particular variable.     

Spatial sensing of stimulus gradients can be superior to temporal sensing for free-swimming bacteria.

Predictions of the minimal size an organism must have to swim along stimulus gradients were used to compare the relative advantages of sensory systems employing spatial (simultaneous) and temporal (sequential) gradient detection mechanisms for small free-swimming bacteria, leading to the following conclusions: 1) there are environmental conditions where spatial detection mechanisms can function for smaller organisms than can temporal mechanisms, 2) temporal mechanisms are superior (have a smaller size limit) for the difficult conditions of low concentration and shallow gradients, but 3) observed bacterial chemotaxis occurs mostly under conditions where spatial mechanisms have a smaller size limit, and 4) relevant conditions in the natural environment favor temporal mechanisms in some cases and spatial mechanisms in others. Thus, sensory ecology considerations do not preclude free-swimming bacteria from employing spatial detection mechanisms, as has been thought, and microbiologists should be on the lookout for them. If spatial mechanisms do not occur, the explanation should be sought elsewhere.     

The motion aftereffect as a universal phenomenon in sensory systems involved in spatial orientation. III. Aftereffect of motion adaptation in the somatosensory and vestibular systems

The motion aftereffect may be considered as a consequence of visual illusions of self-motion (vection) and the persistence of sensory information processing. There is ample experimental evidence indicating a uniformity of mechanisms that underlie motion aftereffects in different modalities based on the principle of motion detectors. Currently, there is firm ground to believe that the motion aftereffect is intrinsic to all sensory systems involved in spatial orientation, that motion adaptation in one sensory system elicits changes in another one, and that such adaptation is of great adaptive importance for spatial orientation and motion of an organism. This review seeks to substantiate these ideas.     

Cryptically Patterned Moths Perceive Bark Structure When Choosing Body Orientations That Match Wing Color Pattern to the Bark Pattern

Many moths have wing patterns that resemble bark of trees on which they rest. The wing patterns help moths to become camouflaged and to avoid predation because the moths are able to assume specific body orientations that produce a very good match between the pattern on the bark and the pattern on the wings. Furthermore, after landing on a bark moths are able to perceive stimuli that correlate with their crypticity and are able to re-position their bodies to new more cryptic locations and body orientations. However, the proximate mechanisms, i.e. how a moth finds an appropriate resting position and orientation, are poorly studied. Here, we used a geometrid moth Jankowskia fuscaria to examine i) whether a choice of resting orientation by moths depends on the properties of natural background, and ii) what sensory cues moths use. We studied moths’ behavior on natural (a tree log) and artificial backgrounds, each of which was designed to mimic one of the hypothetical cues that moths may perceive on a tree trunk (visual pattern, directional furrow structure, and curvature). We found that moths mainly used structural cues from the background when choosing their resting position and orientation. Our findings highlight the possibility that moths use information from one type of sensory modality (structure of furrows is probably detected through tactile channel) to achieve crypticity in another sensory modality (visual). This study extends our knowledge of how behavior, sensory systems and morphology of animals interact to produce crypsis.     

Optimality in mono- and multisensory map formation

In the struggle for survival in a complex and dynamic environment, nature has developed a multitude of sophisticated sensory systems. In order to exploit the information provided by these sensory systems, higher vertebrates reconstruct the spatio-temporal environment from each of the sensory systems they have at their disposal. That is, for each modality the animal computes a neuronal representation of the outside world, a monosensory neuronal map. Here we present a universal framework that allows to calculate the specific layout of the involved neuronal network by means of a general mathematical principle, viz., stochastic optimality. In order to illustrate the use of this theoretical framework, we provide a step-by-step tutorial of how to apply our model. In so doing, we present a spatial and a temporal example of optimal stimulus reconstruction which underline the advantages of our approach. That is, given a known physical signal transmission and rudimental knowledge of the detection process, our approach allows to estimate the possible performance and to predict neuronal properties of biological sensory systems. Finally, information from different sensory modalities has to be integrated so as to gain a unified perception of reality for further processing, e.g., for distinct motor commands. We briefly discuss concepts of multimodal interaction and how a multimodal space can evolve by alignment of monosensory maps.     

Implicit multisensory associations influence voice recognition

Natural objects provide partially redundant information to the brain through different sensory modalities. For example, voices and faces both give information about the speech content, age, and gender of a person. Thanks to this redundancy, multimodal recognition is fast, robust, and automatic. In unimodal perception, however, only part of the information about an object is available. Here, we addressed whether, even under conditions of unimodal sensory input, crossmodal neural circuits that have been shaped by previous associative learning become activated and underpin a performance benefit. We measured brain activity with functional magnetic resonance imaging before, while, and after participants learned to associate either sensory redundant stimuli, i.e. voices and faces, or arbitrary multimodal combinations, i.e. voices and written names, ring tones, and cell phones or brand names of these cell phones. After learning, participants were better at recognizing unimodal auditory voices that had been paired with faces than those paired with written names, and association of voices with faces resulted in an increased functional coupling between voice and face areas. No such effects were observed for ring tones that had been paired with cell phones or names. These findings demonstrate that brief exposure to ecologically valid and sensory redundant stimulus pairs, such as voices and faces, induces specific multisensory associations. Consistent with predictive coding theories, associative representations become thereafter available for unimodal perception and facilitate object recognition. These data suggest that for natural objects effective predictive signals can be generated across sensory systems and proceed by optimization of functional connectivity between specialized cortical sensory modules.     

Bring your own camera to the trap: An inexpensive,versatile, and portable triggering system tested on wild hummingbirds

The study of animals in the wild offers opportunities to collect relevant information on their natural behavior and abilities to perform ecologically relevant tasks. However, it also poses challenges such as accounting for observer effects, human sensory limitations, and the time intensiveness of this type of research. To meet these challenges, field biologists have deployed camera traps to remotely record animal behavior in the wild. Despite their ubiquity in research, many commercial camera traps have limitations, and the species and behavior of interest may present unique challenges. For example, no camera traps support high‐speed video recording. We present a new and inexpensive camera trap system that increases versatility by separating the camera from the triggering mechanism. Our system design can pair with virtually any camera and allows for independent positioning of a variety of sensors, all while being low‐cost, lightweight, weatherproof, and energy efficient. By using our specialized trigger and customized sensor configurations, many limitations of commercial camera traps can be overcome. We use this system to study hummingbird feeding behavior using high‐speed video cameras to capture fast movements and multiple sensors placed away from the camera to detect small body sizes. While designed for hummingbirds, our application can be extended to any system where specialized camera or sensor features are required, or commercial camera traps are cost‐prohibitive, allowing camera trap use in more research avenues and by more researchers.     

Function of a fly motion-sensitive neuron matches eye movements during free flight

Sensing is often implicitly assumed to be the passive acquisition of information. However, part of the sensory information is generated actively when animals move. For instance, humans shift their gaze actively in a sequence of saccades towards interesting locations in a scene. Likewise, many insects shift their gaze by saccadic turns of body and head, keeping their gaze fixed between saccades. Here we employ a novel panoramic virtual reality stimulator and show that motion computation in a blowfly visual interneuron is tuned to make efficient use of the characteristic dynamics of retinal image flow. The neuron is able to extract information about the spatial layout of the environment by utilizing intervals of stable vision resulting from the saccadic viewing strategy. The extraction is possible because the retinal image flow evoked by translation, containing information about object distances, is confined to low frequencies. This flow component can be derived from the total optic flow between saccades because the residual intersaccadic head rotations are small and encoded at higher frequencies. Information about the spatial layout of the environment can thus be extracted by the neuron in a computationally parsimonious way. These results on neuronal function based on naturalistic, behaviourally generated optic flow are in stark contrast to conclusions based on conventional visual stimuli that the neuron primarily represents a detector for yaw rotations of the animal.     

Visual, haptic and cross-modal recognition of objects and scenes.

In this article we review current literature on cross-modal recognition and present new findings from our studies on object and scene recognition. Specifically, we address the questions of what is the nature of the representation underlying each sensory system that facilitates convergence across the senses and how perception is modified by the interaction of the senses. In the first set of our experiments, the recognition of unfamiliar objects within and across the visual and haptic modalities was investigated under conditions of changes in orientation (0 degrees or 180 degrees ). An orientation change increased recognition errors within each modality but this effect was reduced across modalities. Our results suggest that cross-modal object representations of objects are mediated by surface-dependent representations. In a second series of experiments, we investigated how spatial information is integrated across modalities and viewpoint using scenes of familiar, 3D objects as stimuli. We found that scene recognition performance was less efficient when there was either a change in modality, or in orientation, between learning and test. Furthermore, haptic learning was selectively disrupted by a verbal interpolation task. Our findings are discussed with reference to separate spatial encoding of visual and haptic scenes. We conclude by discussing a number of constraints under which cross-modal integration is optimal for object recognition. These constraints include the nature of the task, and the amount of spatial and temporal congruency of information across the modalities.     

Why do animals have so many receptors? The role of multiple chemosensors in animal perception

Many animals have an abundance and diverse assortment of peripheral sensors, both across and within sensory modalities. Multiple sensors offer many functional advantages to an animal's ability to perceive and respond to environmental signals. Advantages include extending the ability to detect and determine the spatial distribution of stimuli, improving the range and accuracy of discrimination among stimuli of different types and intensities, increasing behavioral sensitivity to stimuli, ensuring continued sensory capabilities when the probability of damage or other loss of function to some sensors is high, maintaining sensory function over the entire sensory surface during development and growth, and increasing the richness of behavioral output to sensory stimulation. In this paper, we use the crustacean chemosensory system as the primary example to discuss these functions of multiple sensors. These principles may be applicable to the function of autonomous robots and should be considered in their design.     

Invertebrate-Inspired sensory-motor systems and autonomous, olfactory-guided exploration

The localization of resources in a natural environment is a multifaceted problem faced by both invertebrate animals and autonomous robots. At a first approximation, locomotion through natural environments must be guided by reliable sensory information. But natural environments can be unpredictable, so from time to time, information from any one sensory modality is likely to become temporarily unreliable. Fortunately, compensating mechanisms ensure that such signals are replaced or disambiguated by information from more reliable modalities. For invertebrates and robots to rely primarily on chemical senses has advantages and pitfalls, and these are discussed. The role of turbulence, which makes tracking a single odor to its source a complex problem, is contrasted with the high-fidelity identification of stimulus quality by the invertebrate chemoreceptor and by artificial sensors.     

Modelling a historic oil-tank fire allows an estimation of the sensitivity of the infrared receptors in pyrophilous Melanophila beetles

Pyrophilous jewel beetles of the genus Melanophila approach forest fires and there is considerable evidence that these beetles can detect fires from great distances of more than 60 km. Because Melanophila beetles are equipped with infrared receptors and are also attracted by hot surfaces it can be concluded that these infrared receptors are used for fire detection.The sensitivity of the IR receptors is still unknown. The lowest threshold published so far is 0.6 W/m(2) which, however, cannot explain the detection of forest fires by IR radiation from distances larger than approximately 10 km. To investigate the possible sensitivity of the IR receptors we assumed that beetles use IR radiation for remote fire detection and we made use of a historic report about a big oil-tank fire in Coalinga, California, in 1924. IR emission of an oil-tank fire can be calculated by "pool fire" simulations which now are used for fire safety and risk analysis. Assuming that beetles were lured to the fire from the nearest forests 25 and 130 km away, our results show that detection from a distance of 25 km requires a threshold of the IR receptors of at least 3×10(-2) W/m(2). According to our investigations most beetles became aware of the fire from a distance of 130 km. In this case the threshold has to be 1.3×10(-4) W/m(2). Because such low IR intensities are buried in thermal noise we suggest that the infrared sensory system of Melanophila beetles utilizes stochastic resonance for the detection of weak IR radiation. Our simulations also suggest that the biological IR receptors might be even more sensitive than uncooled technical IR sensors. Thus a closer look into the mode of operation of the Melanophila IR receptors seems promising for the development of novel IR sensors.