Bayesian multisensory integration and cross-modal spatial links.

Our perception of the word is the result of combining information between several senses, such as vision, audition and proprioception. These sensory modalities use widely different frames of reference to represent the properties and locations of object. Moreover, multisensory cues come with different degrees of reliability, and the reliability of a given cue can change in different contexts. The Bayesian framework--which we describe in this review--provides an optimal solution to deal with this issue of combining cues that are not equally reliable. However, this approach does not address the issue of frames of references. We show that this problem can be solved by creating cross-modal spatial links in basis function networks. Finally, we show how the basis function approach can be combined with the Bayesian framework to yield networks that can perform optimal multisensory combination. On the basis of this theory, we argue that multisensory integration is a dialogue between sensory modalities rather that the convergence of all sensory information onto a supra-modal area.     

Olfactory assessment of predation risk in the aquatic environment

The aquatic environment is well suited for the transmission of chemical information. Aquatic animals have evolved highly sensitive receptors for detecting these cues. Here, I review behavioural evidence for the use of chemical cues by aquatic animals for the assessment of predation risk. Chemical cues are released during detection, attack, capture and ingestion of prey. The nature of the cue released depends on the stage of the predation sequence in which cues are released. Predator odours, disturbance pheromones, injury-released chemical cues and dietary cues all convey chemical information to prey Prey use these cues to minimize their probability of being taken on to the next stage of the sequence. The evolution of specialized epidermal alarm substance cells in fishes in the superorder Ostariophysi represent an amplification of this general phenomenon. These cells carry a significant metabolic cost. The cost is offset by the fitness benefit of the chemical attraction of predators. Attempts of piracy by secondary predators interrupt predation events allowing prey an opportunity for escape. In conclusion, chemical cues are widely used by aquatic prey for risk assessment and this has resulted in the evolution of specialized structures among some taxa.     

The physical nature of life.

Life evolved from the primeval world of physics. Sensory systems inform animals of the natural environment, enabling them to conduct responsively. The discovery of weak, DC bioelectric fields in the vicinity of aquatic organisms and the role they play in guiding sharks and rays to their prey have led to the recognition of fundamental, hitherto less well known, physical aspects of sensory biology. The inferred cybernetic algorithm of electric-field orientation in sharks and rays is highly effective and extremely robust. In orienting to the weak DC electric fields of ocean currents and to the earth's magnetic field, sharks and rays unwittingly practise the motional-electric principles that Einstein had in mind when he introduced the special theory of relativity. At the sense-organ, receptor-membrane, and ion-channel levels, the elasmobranch ampullae of Lorenzini operate on the basis of graded positive feedback driven by negative conductance, supposedly employing voltage-sensitive ion channels as the active, excitable elements. The electric sense of sharks and rays presents an exquisite implementation of the very biophysical principles that also govern the graded, much richer than all-or-none, integrative brain processes of animal and man.     

Largemouth bass (Micropterus salmoides) switch feeding modalities in response to sensory deprivation

Animals use a suite of sensory modalities to precisely locate and capture prey. While numerous studies have examined the effects of sensory deprivation on the behaviors leading to prey capture and while it is generally believed that information in the pre-strike period determines the way fish capture prey, this study is the first to examine the contribution of sensory information to jaw kinematics during capture. Largemouth bass were filmed using high-speed videography while capturing live mosquitofish. Bass were examined intact, with visual deprivation under infrared light, and with lateral line deprivation following treatment with cobalt chloride. Deprived of visual cues, this visual ram-feeding predator switches towards suction-based feeding to successfully capture prey. They approach prey slowly but open their mouths more rapidly, which has been shown to result in greater buccal pressure, causing their prey to move a greater distance at a more rapid velocity as they are being drawn into the predators' mouths. Deprived of lateral line cues, bass have higher forward velocities during capture and capture prey earlier in the gape cycle. This study demonstrates that sensory pre-strike information directly affects the capture modality employed by fishes and that fish can modulate between ram and suction not only by adjusting the amount of ram by increasing or decreasing their movements, but also by actively increasing the amount of suction used. These results suggest that the ability to modulate feeding behavior may allow animals to not only exploit a broader breadth of prey items, but also to be capable of doing so in a wider variety of environments.     

Multisensory integration of colors and scents: insights from bees and flowers

Karl von Frisch’s studies of bees’ color vision and chemical senses opened a window into the perceptual world of a species other than our own. A century of subsequent research on bees’ visual and olfactory systems has developed along two productive but independent trajectories, leaving the questions of how and why bees use these two senses in concert largely unexplored. Given current interest in multimodal communication and recently discovered interplay between olfaction and vision in humans and Drosophila, understanding multisensory integration in bees is an opportunity to advance knowledge across fields. Using a classic ethological framework, we formulate proximate and ultimate perspectives on bees’ use of multisensory stimuli. We discuss interactions between scent and color in the context of bee cognition and perception, focusing on mechanistic and functional approaches, and we highlight opportunities to further explore the development and evolution of multisensory integration. We argue that although the visual and olfactory worlds of bees are perhaps the best-studied of any non-human species, research focusing on the interactions between these two sensory modalities is vitally needed.     

Influence of gut morphology, sensory cues and hunger on feeding behaviour of spangled perch, Leiopotherapon unicolor (Günther, 1859), (Percoidei, Teraponidae)

The morphology of the alimentary canal, feeding technique and responses to sensory cues were investigated in Leiopotherapon unicolor . The small conical teeth on the jaws and pharynx, simple Y-shaped stomach and short gut of L. unicolor are characteristic of opportunistic carnivorous fish. Five types of behaviour, 'basal', 'orientation', 'positioning', 'attack', and 'ingestion', were recognised and are described. Capture of prey small enough to swallow whole was assisted by suction created as the mouth opened to engulf prey. Crustaceans too large to ingest whole were broken into smaller pieces against the substrate. Directional and oscillatory movement were the most effective sensory cues in eliciting feeding behaviour, implicating vision as an important sense for L. unicolor feeding in confined aquaria. Acoustic and olfactory senses are less important in prey capture. The frequency with which spangled perch responded to different sensory cues increased with increasing hunger.     

Knowing the Risk: Crickets Distinguish between Spider Predators of Different Size and Commonness

Predators unintentionally release chemical and other cues into their environment that can be used by prey to assess predator presence. Prey organisms can therefore perform specific antipredator behavior to reduce predation risk, which can strongly shape the outcome of trophic interactions. In contrast to aquatic systems, studies on cue‐driven antipredator behavior in terrestrial arthropods cover only few species to date. Here, we investigated occurrence and strength of antipredator behavior of the wood cricket Nemobius sylvestris toward cues of 14 syntopic spider species that are potential predators of wood crickets. We used two different behavioral arena experiments to investigate the influence of predator cues on wood cricket mobility. We further tested whether changes in wood cricket mobility can be explained by five predator‐specific traits: hunting mode, commonness, diurnal activity, predator–prey body–size ratio, and predator–prey life stage differences. Crickets were singly recorded (1) in separate arenas, either in presence or absence of spider cues, to analyze changes in mobility on filter paper covered with cues compared with normal mobility on filter paper without cues; and (2) in subdivided arenas partly covered with spider cues, where the crickets could choose between cue‐bearing and cue‐less areas to analyze differences in residence time and mobility when crickets are able to avoid cues. Crickets either increased or reduced their mobility in the presence of spider cues. In the experiments with cues and controls in separate arenas, the magnitude of behavioral change increased significantly with increasing predator–prey body size ratio. When crickets could choose between spider cues and control, their mobility was significantly higher in the presence of cues from common spider species than from rare spiders. We therefore conclude that wood crickets distinguish between cues from different predator species and that spiders unintentionally release a species‐specific composition and size‐dependent quantity of cues, which lead to distinct antipredator behavior in wood crickets.     

The role of sense organs in the feeding behaviour of Chinese perch

Experiments were conducted to identify the roles of the individual sense organs in the feeding behaviour of Chinese perch Siniperca chuatsi by determining the consumption of natural food after selective removal or blocking of eyes, lateral lines and olfactory organs, and also by observing the behavioural response to visual, mechanical and chemical stimulation by artificial prey. Chinese perch were able to feed properly on live prey fish when either eyes or lateral lines were intact or functional, but could scarcely feed without these two senses. Chinese perch recognized its prey by vision through the perception of motion and shape, and showed a greater dependence on vision in predation when both visual and mechanical cues were available. Chemical stimulation by natural food could not elicit any feeding response in Chinese perch, and gustation was only important to the fish for the last stage of food discrimination in the oropharyngeal cavity. The sensory basis of Chinese perch in feeding is well adapted to its nocturnal stalking hunting strategy, and also explains its peculiar food habit of accepting live prey fish only and refusing dead prey fish or artificial diets.     

Integration of bimodal looming signals through neuronal coherence in the temporal lobe

The ability to integrate information across multiple sensory systems offers several behavioral advantages, from quicker reaction times and more accurate responses to better detection and more robust learning. At the neural level, multisensory integration requires large-scale interactions between different brain regions--the convergence of information from separate sensory modalities, represented by distinct neuronal populations. The interactions between these neuronal populations must be fast and flexible, so that behaviorally relevant signals belonging to the same object or event can be immediately integrated and integration of unrelated signals can be prevented. Looming signals are a particular class of signals that are behaviorally relevant for animals and that occur in both the auditory and visual domain. These signals indicate the rapid approach of objects and provide highly salient warning cues about impending impact. We show here that multisensory integration of auditory and visual looming signals may be mediated by functional interactions between auditory cortex and the superior temporal sulcus, two areas involved in integrating behaviorally relevant auditory-visual signals. Audiovisual looming signals elicited increased gamma-band coherence between these areas, relative to unimodal or receding-motion signals. This suggests that the neocortex uses fast, flexible intercortical interactions to mediate multisensory integration.     

Experimental Evidence of Threat-Sensitive Collective Avoidance Responses in a Large Wild-Caught Herring School

Aggregation is commonly thought to improve animals'' security. Within aquatic ecosystems, group-living prey can learn about immediate threats using cues perceived directly from predators, or from collective behaviours, for example, by reacting to the escape behaviours of companions. Combining cues from different modalities may improve the accuracy of prey antipredatory decisions. In this study, we explored the sensory modalities that mediate collective antipredatory responses of herring (Clupea harengus) when in a large school (approximately 60 000 individuals). By conducting a simulated predator encounter experiment in a semi-controlled environment (a sea cage), we tested the hypothesis that the collective responses of herring are threat-sensitive. We investigated whether cues from potential threats obtained visually or from the perception of water displacement, used independently or in an additive way, affected the strength of the collective avoidance reactions. We modified the sensory nature of the simulated threat by exposing the herring to 4 predator models differing in shape and transparency. The collective vertical avoidance response was observed and quantified using active acoustics. The combination of sensory cues elicited the strongest avoidance reactions, suggesting that collective antipredator responses in herring are mediated by the sensory modalities involved during threat detection in an additive fashion. Thus, this study provides evidence for magnitude-graded threat responses in a large school of wild-caught herring which is consistent with the “threat-sensitive hypothesis”.     

Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments

Although much traditional sensory research has studied each sensory modality in isolation, there has been a recent explosion of interest in causal interplay between different senses. Various techniques have now identified numerous multisensory convergence zones in the brain. Some convergence may arise surprisingly close to low-level sensory-specific cortex, and some direct connections may exist even between primary sensory cortices. A variety of multisensory phenomena have now been reported in which sensory-specific brain responses and perceptual judgments concerning one sense can be affected by relations with other senses. We survey recent progress in this multisensory field, foregrounding human studies against the background of invasive animal work and highlighting possible underlying mechanisms. These include rapid feedforward integration, possible thalamic influences, and/or feedback from multisensory regions to sensory-specific brain areas. Multisensory interplay is more prevalent than classic modular approaches assumed, and new methods are now available to determine the underlying circuits.     

Ontogenetic dietary shifts and feeding behavior of the tiger shark, Galeocerdo cuvier, in Hawaiian waters

Synopsis Stomach content data from 281 tiger sharks caught during shark control programs in Hawaii between 1967 and 1969, and during 1976 were analyzed to examine feeding habits and ontogenetic shifts in diet. As sharks increased in size, prey diversity and frequency of occurrence of large prey items increased. The percent occurrence of teleosts and cephalopods in stomachs decreased as sharks increased in length, while occurrence of elasmobranchs, turtles, land mammals, crustaceans, and undigestible items increased. Comparisons between the diets of tiger sharks from Hawaii and other locations indicate that ontogenetic shifts are universal in this species and that tiger sharks may be opportunistic feeders that prey heavily on abundant, easy to capture prey. Small tiger sharks may be spatially segregated from medium and large sharks and appear to be primarily nocturnal, bottom feeders. Large tiger sharks feed near the bottom at night, but also feed at the surface during the day. Prey, similar in size to humans, begin to occur in the diet of tiger sharks approximately 230 cm TL, and therefore sharks of this size and larger may pose the greatest threat to humans. Ontogenetic shifts in diet may be attributed to increased size of sharks, expanded range and exploitation of habitats of larger sharks, and/or improved hunting skill of larger sharks.Deceased 1974     

Visual influences on auditory spatial learning

The visual and auditory systems frequently work together to facilitate the identification and localization of objects and events in the external world. Experience plays a critical role in establishing and maintaining congruent visual-auditory associations, so that the different sensory cues associated with targets that can be both seen and heard are synthesized appropriately. For stimulus location, visual information is normally more accurate and reliable and provides a reference for calibrating the perception of auditory space. During development, vision plays a key role in aligning neural representations of space in the brain, as revealed by the dramatic changes produced in auditory responses when visual inputs are altered, and is used throughout life to resolve short-term spatial conflicts between these modalities. However, accurate, and even supra-normal, auditory localization abilities can be achieved in the absence of vision, and the capacity of the mature brain to relearn to localize sound in the presence of substantially altered auditory spatial cues does not require visuomotor feedback. Thus, while vision is normally used to coordinate information across the senses, the neural circuits responsible for spatial hearing can be recalibrated in a vision-independent fashion. Nevertheless, early multisensory experience appears to be crucial for the emergence of an ability to match signals from different sensory modalities and therefore for the outcome of audiovisual-based rehabilitation of deaf patients in whom hearing has been restored by cochlear implantation.     

Multimodal sensory integration in weakly electric fish: a behavioral account.

The ability to integrate multisensory information is a fundamental characteristic of the brain serving to enhance the detection and identification of external stimuli. Weakly electric fish employ multiple senses in their interactions with one another and with their inanimate environment (electric, visual, acoustic, mechanical, chemical, thermal, and hydrostatic pressure) and also generate signals using some of the same stimulus energies (electric, acoustic, visual, mechanical). A brief overview provides background on the sensory and motor channels available to the fish followed by an examination of how weakly electric fish 'benefit' from integrating various stimulus modalities that assist in prey detection, schooling, foraging, courtship, and object location. Depending on environmental conditions, multiple sensory inputs can act synergistically and improve the task at hand, can be redundant or contradictory, and can substitute for one another. Over time, in repeated encounters with familiar surrounds, loss of one modality can be compensated for through learning. Studies of neuronal substrates and an understanding of the computational algorithms that underlie multisensory integration ought to expose the physiological corollaries to widely published concepts such as internal representation, sensory expectation, sensory generalization, and sensory transfer.     

Different adaptation to visual hunting in three ground beetle species of the same genus

Compound eyes and hunting behaviour of three species of the genus Asaphidion de Gozis 1886 (Coleoptera, Carabidae) have been investigated. All three have a fovea and binocular overlap in their frontal fields of vision. In the smallest species A. flavipes, the binocular overlap is largest and the foveal interommatidial angles are narrowest. All three species hunt by visual cues; A. flavipes is the most precise during the approach to the prey and during the attack. The mean size of its approach jerks and its critical distance prior to the attack are shorter than those of A. caraboides, and the scatter of these distances is much smaller. This leads to greater success in capturing fast fleeing prey (Collembola) on the soil surface.     

Unexpected Positive Buoyancy in Deep Sea Sharks,Hexanchus griseus,and a Echinorhinus cookei

We do not expect non air-breathing aquatic animals to exhibit positive buoyancy. Sharks, for example, rely on oil-filled livers instead of gas-filled swim bladders to increase their buoyancy, but are nonetheless ubiquitously regarded as either negatively or neutrally buoyant. Deep-sea sharks have particularly large, oil-filled livers, and are believed to be neutrally buoyant in their natural habitat, but this has never been confirmed. To empirically determine the buoyancy status of two species of deep-sea sharks (bluntnose sixgill sharks, Hexanchus griseus, and a prickly shark, Echinorhinus cookei) in their natural habitat, we used accelerometer-magnetometer data loggers to measure their swimming performance. Both species of deep-sea sharks showed similar diel vertical migrations: they swam at depths of 200–300 m at night and deeper than 500 m during the day. Ambient water temperature was around 15°C at 200–300 m but below 7°C at depths greater than 500 m. During vertical movements, all deep-sea sharks showed higher swimming efforts during descent than ascent to maintain a given swimming speed, and were able to glide uphill for extended periods (several minutes), indicating that these deep-sea sharks are in fact positively buoyant in their natural habitats. This positive buoyancy may adaptive for stealthy hunting (i.e. upward gliding to surprise prey from underneath) or may facilitate evening upward migrations when muscle temperatures are coolest, and swimming most sluggish, after spending the day in deep, cold water. Positive buoyancy could potentially be widespread in fish conducting daily vertical migration in deep-sea habitats.     

Multisensory Oddity Detection as Bayesian Inference

A key goal for the perceptual system is to optimally combine information from all the senses that may be available in order to develop the most accurate and unified picture possible of the outside world. The contemporary theoretical framework of ideal observer maximum likelihood integration (MLI) has been highly successful in modelling how the human brain combines information from a variety of different sensory modalities. However, in various recent experiments involving multisensory stimuli of uncertain correspondence, MLI breaks down as a successful model of sensory combination. Within the paradigm of direct stimulus estimation, perceptual models which use Bayesian inference to resolve correspondence have recently been shown to generalize successfully to these cases where MLI fails. This approach has been known variously as model inference, causal inference or structure inference. In this paper, we examine causal uncertainty in another important class of multi-sensory perception paradigm – that of oddity detection and demonstrate how a Bayesian ideal observer also treats oddity detection as a structure inference problem. We validate this approach by showing that it provides an intuitive and quantitative explanation of an important pair of multi-sensory oddity detection experiments – involving cues across and within modalities – for which MLI previously failed dramatically, allowing a novel unifying treatment of within and cross modal multisensory perception. Our successful application of structure inference models to the new ‘oddity detection’ paradigm, and the resultant unified explanation of across and within modality cases provide further evidence to suggest that structure inference may be a commonly evolved principle for combining perceptual information in the brain.     

Assessment of local predation risk: the role of subthreshold concentrations of chemical alarm cues

The ability to accurately assess local predation risk is criticalto prey individuals, as it allows them to maximize threat-sensitivetrade-offs between predator avoidance and other fitness relatedactivities. A wide range of taxonomically diverse prey (includingmany freshwater fishes) relies on chemical alarm cues (alarmpheromones) as their primary information source for local riskassessment. However, the value of chemical alarm cues has beenquestioned due to the availability of additional sensory inputs(i.e., visual cues) and the lack of an overt antipredator responseunder conditions of low perceived risk. In this paper, we testthe hypothesis that chemical alarm cues at concentrations belowthe point at which they elicit an overt behavioral responsefunction to increase vigilance towards other sensory modalities(i.e., visual alarm cues). Shoals of glowlight tetras (Hemigrammuserythrozonus) exposed to the subthreshold concentration of hypoxanthine-3-N-oxide(the putative Ostariophysan alarm pheromone) did not exhibitan overt antipredator response in the absence of secondary visualcues (not different than the distilled water control). However,when exposed to the sight of a visually alarmed conspecific,they significantly increased the intensity of their antipredatorresponse (not different from shoals exposed to the suprathresholdalarm cue). This study demonstrates that prey may benefit fromresponding to low concentration alarm cues by increasing vigilancetowards secondary cues during local risk assessment, even inthe absence of an overt behavioral response. By increasing vigilancetowards secondary risk assessment cues in the presence of alow risk chemical cue, individuals are likely able to maximizethe threat-sensitive trade-offs between predator avoidance andother fitness related activities.     

A sense of scale: Foraging cetaceans' use of scale‐dependent multimodal sensory systems

Research on cetacean foraging ecology is central to our understanding of their spatial and behavioral ecology. Yet, functional mechanisms by which cetaceans detect prey across different scales remain unclear. Here, I postulate that cetaceans utilize a scale‐dependent, multimodal sensory system to assess and increase prey encounters. I review the literature on cetacean sensory systems related to foraging ecology, and hypothesize the effective scales of each sensory modality to inform foraging opportunities. Next, I build two “scale‐of‐senses” schematics for the general groups of dolphins and baleen whales. These schematics illustrate the hypothetical interchange of sensory modalities used to locate and discriminate prey at spatial scales ranging from 0 m to 1,000 km: (1) vision, (2) audition (sound production and sound reception), (3) chemoreception, (4) magnetoreception, and somatosensory perception of (5) prey, or (6) oceanographic stimuli. The schematics illustrate how a cetacean may integrate sensory modalities to form an adaptive foraging landscape as a function of distance to prey. The scale‐of‐senses schematic is flexible, allowing for case‐specific application and enhancement with improved cetacean sensory data. The framework serves to improve our understanding of functional cetacean foraging ecology, and to develop new hypotheses, methods, and results regarding how cetaceans forage at multiple scales.     

Information-processing demands in electrosensory and mechanosensory lateral line systems.

The electrosensory and mechanosensory lateral line systems of fish exhibit many common features in their structural and functional organization, both at the sensory periphery as well as in central processing pathways. These two sensory systems also appear to play similar roles in many behavioral tasks such as prey capture, orientation with respect to external environmental cues, navigation in low-light conditions, and mediation of interactions with nearby animals. In this paper, we briefly review key morphological, physiological, and behavioral aspects of these two closely related sensory systems. We present arguments that the information processing demands associated with spatial processing are likely to be quite similar, due largely to the spatial organization of both systems and the predominantly dipolar nature of many electrosensory and mechanosensory stimulus fields. Demands associated with temporal processing may be quite different, however, due primarily to differences in the physical bases of electrosensory and mechanosensory stimuli (e.g. speed of transmission). With a better sense of the information processing requirements, we turn our attention to an analysis of the functional organization of the associated first-order sensory nuclei in the hindbrain, including the medial octavolateral nucleus (MON), dorsal octavolateral nucleus (DON), and electrosensory lateral line lobe (ELL). One common feature of these systems is a set of neural mechanisms for improving signal-to-noise ratios, including mechanisms for adaptive suppression of reafferent signals. This comparative analysis provides new insights into how the nervous system extracts biologically significant information from dipolar stimulus fields in order to solve a variety of behaviorally relevant problems faced by aquatic animals.