Eavesdropping on visual secrets

Private communication may benefit signalers by reducing the costs imposed by potential eavesdroppers such as parasites, predators, prey, or rivals. It is likely that private communication channels are influenced by the evolution of signalers, intended receivers, and potential eavesdroppers, but most studies only examine how private communication benefits signalers. Here, we address this shortcoming by examining visual private communication from a potential eavesdropper’s perspective. Specifically, we ask if a signaler would face fitness consequences if a potential eavesdropper could detect its signal more clearly. By integrating studies on private communication with those on the evolution of vision, we suggest that published studies find few taxon-based constraints that could keep potential eavesdroppers from detecting most hypothesized forms of visual private communication. However, we find that private signals may persist over evolutionary time if the benefits of detecting a particular signal do not outweigh the functional costs a potential eavesdropper would suffer from evolving the ability to detect it. We also suggest that all undetectable signals are not necessarily private signals: potential eavesdroppers may not benefit from detecting a signal if it co-occurs with signals in other more detectable sensory modalities. In future work, we suggest that researchers consider how the evolution of potential eavesdroppers’ sensory systems influences private communication. Specifically, we suggest that examining the fitness correlates and evolution of potential eavesdroppers can help (1) determine the likelihood that private communication channels are stable over evolutionary time, and (2) demonstrate that undetectable signals are private signals by showing that signalers benefit from a reduction in detection by potential eavesdroppers.     

Communication in troubled waters: responses of fish communication systems to changing environments

Fish populations are increasingly being subjected to anthropogenic changes to their sensory environments. The impact of these changes on inter- and intra-specific communication, and its evolutionary consequences, has only recently started to receive research attention. A disruption of the sensory environment is likely to impact communication, especially with respect to reproductive interactions that help to maintain species boundaries. Aquatic ecosystems around the world are being threatened by a variety of environmental stressors, causing dramatic losses of biodiversity and bringing urgency to the need to understand how fish respond to rapid environmental changes. Here, we discuss current research on different communication systems (visual, chemical, acoustic, electric) and explore the state of our knowledge of how complex systems respond to environmental stressors using fish as a model. By far the bulk of our understanding comes from research on visual communication in the context of mate selection and competition for mates, while work on other communication systems is accumulating. In particular, it is increasingly acknowledged that environmental effects on one mode of communication may trigger compensation through other modalities. The strength and direction of selection on communication traits may vary if such compensation occurs. However, we find a dearth of studies that have taken a multimodal approach to investigating the evolutionary impact of environmental change on communication in fish. Future research should focus on the interaction between different modes of communication, especially under changing environmental conditions. Further, we see an urgent need for a better understanding of the evolutionary consequences of changes in communication systems on fish diversity.     

Multisensory integration of colors and scents: insights from bees and flowers

Karl von Frisch’s studies of bees’ color vision and chemical senses opened a window into the perceptual world of a species other than our own. A century of subsequent research on bees’ visual and olfactory systems has developed along two productive but independent trajectories, leaving the questions of how and why bees use these two senses in concert largely unexplored. Given current interest in multimodal communication and recently discovered interplay between olfaction and vision in humans and Drosophila, understanding multisensory integration in bees is an opportunity to advance knowledge across fields. Using a classic ethological framework, we formulate proximate and ultimate perspectives on bees’ use of multisensory stimuli. We discuss interactions between scent and color in the context of bee cognition and perception, focusing on mechanistic and functional approaches, and we highlight opportunities to further explore the development and evolution of multisensory integration. We argue that although the visual and olfactory worlds of bees are perhaps the best-studied of any non-human species, research focusing on the interactions between these two sensory modalities is vitally needed.     

Behavioural divergence,interfertility and speciation: A review

Behavioural compatibility between mates is fundamental for maintaining species boundaries and is achieved through appropriate communication between males and females. A breakdown in communication will lead to behavioural divergence and reduced interfertility. In this review, we summarise the current knowledge on male signals and female perception of these signals, integrating the literature from several taxa. We advocate that signaller–perceiver coevolution, which is usually under strong stabilising selection to enable mating, forms the basis of species-specific mate recognition systems. The mechanisms (phylogeny, geography, ecology, biology) shaping signaller–perceiver systems are briefly discussed to demonstrate the factors underpinning the evolution of signaller–perceiver couplings. Since divergence and diversification of communication systems is driven by changes in the mechanical properties of sensory pathways and morphology of sensory organs, we highlight signal modalities (auditory, olfactory, visual, tactile) and their importance in communication, particularly in mate selection. Next, using available examples and generating a stylised model, we suggest how disruption (biological, ecological, stochastic) of signaller–perceiver systems drives behavioural divergence and consequently results in reduced interfertility and speciation. Future studies should adopt an integrative approach, combining multiple parameters (phylogeny, adaptive utility of communication systems, genetics and biomechanical/biochemical properties of signals and perception) to explore how disruption of signaller–perceiver systems results in behavioural divergence and reduced interfertility. Finally, we question the impact that rapid environmental change will have on disruption of communication systems, potentially interfering with signaller–perceiver couplings.     

Biases in signal evolution: learning makes a difference

It is now well established that signal receivers have a key role in the evolution of animal communication: the suite of sensory and cognitive processes by which animals perceive and learn about their environment can have a significant impact on signal design. A crucial property of these information-processing mechanisms is the emergence of 'receiver bias' in the behavioural responses to signals. Whereas most research has focussed on receiver biases in the sensory system, more recent studies show that biases can also arise from learning about signals. Here, we highlight how learning-based biases can arise, and how these differ from biases emerging from sensory systems in their impact on signal evolution.     

Temporal dynamics of auditory and visual bistability reveal common principles of perceptual organization

When dealing with natural scenes, sensory systems have to process an often messy and ambiguous flow of information. A stable perceptual organization nevertheless has to be achieved in order to guide behavior. The neural mechanisms involved can be highlighted by intrinsically ambiguous situations. In such cases, bistable perception occurs: distinct interpretations of the unchanging stimulus alternate spontaneously in the mind of the observer. Bistable stimuli have been used extensively for more than two centuries to study visual perception. Here we demonstrate that bistable perception also occurs in the auditory modality. We compared the temporal dynamics of percept alternations observed during auditory streaming with those observed for visual plaids and the susceptibilities of both modalities to volitional control. Strong similarities indicate that auditory and visual alternations share common principles of perceptual bistability. The absence of correlation across modalities for subject-specific biases, however, suggests that these common principles are implemented at least partly independently across sensory modalities. We propose that visual and auditory perceptual organization could rely on distributed but functionally similar neural competition mechanisms aimed at resolving sensory ambiguities.     

Taxonomic and sensory biases in the mate-choice literature: there are far too few studies of chemical and multimodal communication

The sexual selection literature has grown rapidly in recent years. In an effort to elucidate biases in the focus of current mate-choice research here, I review 297 studies among 230 species. Among taxa, studies of birds were most common (40% of all studies), with insects, fishes, and anurans less well represented (20%, 18%, 14%, respectively). All other taxa were poorly represented in the literature (<3%). Across sensory modalities, studies of visual and acoustic signals were most common (46%, 30%, respectively), with relatively few studies investigating chemical, tactile, and electrical signals in mate choice (3%, 3%, <1%, respectively). Most mate-choice studies of birds and fishes investigated visual signals, while the majority of insect and anuran studies investigated acoustic signals. While these associations may reflect biological realities—birds and anurans tend not to emphasize chemical cues in mate choice; electric communication may indeed be quite uncommon—they may also be grossly misleading: chemical cues are likely critical for mate choice in millions of insect species. Moreover, I suggest that the particularly high vulnerability of chemical communication networks to anthropogenic fouling should provide immediate motivation for many more studies of chemical signals in mate choice. Finally, I find that despite widespread acceptance that male displays are often comprised of multiple elements produced across sensory modalities, studies simultaneously investigating the use of multiple cues in mate choice are rare. While not exhaustive, this review identifies biases in the focus of mate-choice studies, and should serve to identify fruitful directions for future mate-choice research.     

Multimodal sensory integration in the strike-feeding behaviour of predatory fishes

The search for useful model systems for the study of sensory processing in vertebrate nervous systems has resulted in many neuroethological studies investigating the roles played by a single sensory modality in a given behaviour. However, behaviours relying solely upon information from one sensory modality are relatively rare. Animals behaving in a complex, three-dimensional environment receive a large amount of information from external and internal receptor arrays. Clearly, the integration of sensory afference arising from different modalities into a coherent 'gestalt' of the world is essential to the behaviours of most animals. In the last several years our laboratory team has examined the roles played by the visual and lateral line sensory systems in organizing the feeding behaviour of two species of predatory teleost fishes, the largemouth bass, Micropterus salmoides, and the muskellunge, Esox masquinongy. The free-field feeding behaviours of these fishes were studied quantitatively in intact animals and compared to animals in which the lateral line and visual systems had been selectively suppressed. All groups of animals continued to feed successfully, but significant differences were observed between each experimental group, providing strong clues as to the relative role played by each sensory system in the organization of the behaviour. Furthermore, significant differences exist between the two species. The differences in behaviour resulting when an animal is deprived of a given sensory modality reflect the nature of central integrative sensory processes, and these behavioural studies provide a foundation for further neuroanatomical and physiological studies of sensory integration in the vertebrate central nervous system.     

You talkin’ to me? Interactive playback is a powerful yet underused tool in animal communication research

Over the years, playback experiments have helped further our understanding of the wonderful world of animal communication. They have provided fundamental insights into animal behaviour and the function of communicative signals in numerous taxa. As important as these experiments are, however, there is strong evidence to suggest that the information conveyed in a signal may only have value when presented interactively. By their very nature, signalling exchanges are interactive and therefore, an interactive playback design is a powerful tool for examining the function of such exchanges. While researchers working on frog and songbird vocal interactions have long championed interactive playback, it remains surprisingly underused across other taxa. The interactive playback approach is not limited to studies of acoustic signalling, but can be applied to other sensory modalities, including visual, chemical and electrical communication. Here, I discuss interactive playback as a potent yet underused technique in the field of animal behaviour. I present a concise review of studies that have used interactive playback thus far, describe how it can be applied, and discuss its limitations and challenges. My hope is that this review will result in more scientists applying this innovative technique to their own study subjects, as a means of furthering our understanding of the function of signalling interactions in animal communication systems.     

Tactics of evasion: strategies used by signallers to deter eavesdropping enemies from exploiting communication systems

Eavesdropping predators, parasites and parasitoids exploit signals emitted by their prey and hosts for detection, assessment, localization and attack, and in the process impose strong selective pressures on the communication systems of the organisms they exploit. Signallers have evolved numerous anti-eavesdropper strategies to mitigate the trade-off between the costs imposed from signal exploitation and the need for conspecific communication. Eavesdropper strategies fall along a continuum from opportunistic to highly specialized, and the tightness of the eavesdropper–signaller relationship results in differential pressures on communication systems. A wide variety of anti-eavesdropper strategies mitigate the trade-off between eavesdropper exploitation and conspecific communication. Antagonistic selection from eavesdroppers can result in diverse outcomes including modulation of signalling displays, signal structure, and evolutionary loss or gain of a signal from a population. These strategies often result in reduced signal conspicuousness and in decreased signal ornamentation. Eavesdropping enemies, however, can also promote signal ornamentation. While less common, this alternative outcome offers a unique opportunity to dissect the factors that may lead to different evolutionary pathways. In addition, contrary to traditional assumptions, no sensory modality is completely ‘safe’ as eavesdroppers are ubiquitous and have a broad array of sensory filters that allow opportunity for signal exploitation. We discuss how anthropogenic change affects interactions between eavesdropping enemies and their victims as it rapidly modifies signalling environments and community composition. Drawing on diverse research from a range of taxa and sensory modalities, we synthesize current knowledge on anti-eavesdropper strategies, discuss challenges in this field and highlight fruitful new directions for future research. Ultimately, this review offers a conceptual framework to understand the diverse strategies used by signallers to communicate under the pressure imposed by their eavesdropping enemies.     

Modality interactions alter the shape of acoustic mate preference functions in gray treefrogs

Sexual selection takes place in complex environments where females evaluating male mating signals are confronted with stimuli from multiple sources and modalities. The pattern of expression of female preferences may be influenced by interactions between modalities, changing the shape of female preference functions, and thus ultimately altering the selective landscape acting on male signal evolution. We tested the hypothesis that the responses of female gray treefrogs, Hyla versicolor, to acoustic male advertisement calls are affected by interactions with visual stimuli. We measured preference functions for several call traits under two experimental conditions: unimodal (only acoustic signals presented), and multimodal (acoustic signals presented along with a video‐animated calling male). We found that females were more responsive to multimodal stimulus presentations and, compared to unimodal playbacks, had weaker preferences for temporal call characteristics. We compared the preference functions obtained in these two treatments to the distribution of male call characteristics to make inferences on the strength and direction of selection expected to act on male calls. Modality interactions have the potential to influence the course of signal evolution and thus are an important consideration in sexual selection studies.     

Exploring Combinations of Auditory and Visual Stimuli for Gaze-Independent Brain-Computer Interfaces

For Brain-Computer Interface (BCI) systems that are designed for users with severe impairments of the oculomotor system, an appropriate mode of presenting stimuli to the user is crucial. To investigate whether multi-sensory integration can be exploited in the gaze-independent event-related potentials (ERP) speller and to enhance BCI performance, we designed a visual-auditory speller. We investigate the possibility to enhance stimulus presentation by combining visual and auditory stimuli within gaze-independent spellers. In this study with N = 15 healthy users, two different ways of combining the two sensory modalities are proposed: simultaneous redundant streams (Combined-Speller) and interleaved independent streams (Parallel-Speller). Unimodal stimuli were applied as control conditions. The workload, ERP components, classification accuracy and resulting spelling speed were analyzed for each condition. The Combined-speller showed a lower workload than uni-modal paradigms, without the sacrifice of spelling performance. Besides, shorter latencies, lower amplitudes, as well as a shift of the temporal and spatial distribution of discriminative information were observed for Combined-speller. These results are important and are inspirations for future studies to search the reason for these differences. For the more innovative and demanding Parallel-Speller, where the auditory and visual domains are independent from each other, a proof of concept was obtained: fifteen users could spell online with a mean accuracy of 87.7% (chance level <3%) showing a competitive average speed of 1.65 symbols per minute. The fact that it requires only one selection period per symbol makes it a good candidate for a fast communication channel. It brings a new insight into the true multisensory stimuli paradigms. Novel approaches for combining two sensory modalities were designed here, which are valuable for the development of ERP-based BCI paradigms.     

Issues in the classification of multimodal communication signals

Communication involves complex behavior in multiple sensory channels, or "modalities." We provide an overview of multimodal communication and its costs and benefits, place examples of signals and displays from an array of taxa, sensory systems, and functions into our signal classification system, and consider issues surrounding the categorization of multimodal signals. The broadest level of classification is between signals with redundant and nonredundant components, with finer distinctions in each category. We recommend that researchers gather information on responses to each component of a multimodal signal as well as the response to the signal as a whole. We discuss the choice of categories, whether to categorize signals on the basis of the signal or the response, and how to classify signals if data are missing. The choice of behavioral assay may influence the outcome, as may the context of the communicative event. We also consider similarities and differences between multimodal and unimodal composite signals and signals that are sequentially, rather than simultaneously, multimodal.     

Sensory trade-offs predict signal divergence in Surfperch

Unidirectional elaboration of male trait evolution (e.g., larger, brighter males) has been predicted by receiver bias models of sexual selection and empirically tested in a number of different taxa. This study identifies a bidirectional pattern of male trait evolution and suggests that a sensory constraint is driving this divergence. In this system, the inherent trade-off in dichromatic visual detection places limits on the direction that sensory biases may take and thus provides a quantitative test of the sensory drive model. Here I show that sensory systems with trade-offs in detection abilities produce bidirectional biases and that signal design properties match these biases. I combine species-specific measurements and ancestral estimates with visual detection modeling to examine biases in sensory and signaling traits across five fish species occupying optically diverse habitats in the Californian kelp forest. Species-specific divergence in visual pigments correlates with changes in environment and produces different sensory biases--favoring luminance (brightness) detection for some species and chromatic (color) detection for others. Divergence in male signals (spectral reflectance of orange, blue, and silver color elements) is predicted by each species' sensory bias: color divergence favors chromatic detection for species with chromatically biased visual systems, whereas species with luminance sensory biases have signals favoring luminance detection. This quantitative example of coevolution of communication traits varying in a bidirectional pattern governed by the environment is the first demonstration of sensory trade-offs driving signal evolution.     

Seeing sounds: visual and auditory interactions in the brain

Objects and events can often be detected by more than one sensory system. Interactions between sensory systems can offer numerous benefits for the accuracy and completeness of the perception. Recent studies involving visual-auditory interactions have highlighted the perceptual advantages of combining information from these two modalities and have suggested that predominantly unimodal brain regions play a role in multisensory processing.     

Coevolution of visual signals and eye morphology in Polistes paper wasps

To be effective, signals must propagate through the environment and be detected by receivers. As a result, signal form evolves in response to both the constraints imposed by the transmission environment and receiver perceptual abilities. Little work has examined the extent to which signals may act as selective forces on receiver sensory systems to improve the efficacy of communication. If receivers benefit from accurate signal assessment, selection could favour sensory organs that improve discrimination of established signals. Here, we provide evidence that visual resolution coevolves with visual signals in Polistes wasps. Multiple Polistes species have variable facial patterns that function as social signals, whereas other species lack visual signals. Analysis of 19 Polistes species shows that maximum eye facet size is positively associated with both eye size and presence of visual signals. Relatively larger facets within the eye''s acute zone improve resolution of small images, such as wasp facial signals. Therefore, sensory systems may evolve to optimize signal assessment. Sensory adaptations to facilitate signal detection may represent an overlooked area of the evolution of animal communication.     

Seeing and Feeling Motion: Canonical Computations in Vision and Touch

While the different sensory modalities are sensitive to different stimulus energies, they are often charged with extracting analogous information about the environment. Neural systems may thus have evolved to implement similar algorithms across modalities to extract behaviorally relevant stimulus information, leading to the notion of a canonical computation. In both vision and touch, information about motion is extracted from a spatiotemporal pattern of activation across a sensory sheet (in the retina and in the skin, respectively), a process that has been extensively studied in both modalities. In this essay, we examine the processing of motion information as it ascends the primate visual and somatosensory neuraxes and conclude that similar computations are implemented in the two sensory systems.     

From uni- to multimodality: towards an integrative view on anuran communication

Undeniably, acoustic signals are the predominant mode of communication in frogs and toads. Acoustically active species are found throughout the vast diversity of anuran families. However, additional or alternative signal modalities have gained increasing attention. In several anurans, seismic, visual and chemical communications have convergently evolved due to ecological constraints such as noisy environments. The production of a visual cue, like the inevitably moving vocal sac of acoustically advertising males, is emphasized by conspicuously coloured throats. Limb movements accompanied by dynamic displays of bright colours are additional examples of striking visual signals independent of vocalizations. In some multimodal anuran communication systems, the acoustic component acts as an alert signal, which alters the receiver attention to the following visual display. Recent findings of colourful glands on vocal sacs, producing volatile species-specific scent bouquets suggest the possibility of integration of acoustic, visual and chemical cues in species recognition and mate choice. The combination of signal components facilitates a broadened display repertoire in challenging environmental conditions. Thus, the complexity of the communication systems of frogs and toads may have been underestimated.     

Inhibition, not excitation, is the key to multimodal sensory integration

Multimodal neuronal maps, combining input from two or more sensory systems, play a key role in the processing of sensory and motor information. For such maps to be of any use, the input from all participating modalities must be calibrated so that a stimulus at a specific spatial location is represented at an unambiguous position in the multimodal map. Here we discuss two methods based on supervised spike-timing-dependent plasticity (STDP) to gauge input from different sensory modalities so as to ensure a proper map alignment. The first uses an excitatory teacher input. It is therefore called excitation-mediated learning. The second method is based on an inhibitory teacher signal, as found in the barn owl, and is called inhibition-mediated learning. Using detailed analytical calculations and numerical simulations, we demonstrate that inhibitory teacher input is essential if high-quality multimodal integration is to be learned rapidly. Furthermore, we show that the quality of the resulting map is not so much limited by the quality of the teacher signal but rather by the accuracy of the input from other sensory modalities.