Extrinsic embryonic sensory stimulation alters multimodal behavior and cellular activation

Embryonic vision is generated and maintained by spontaneous neuronal activation patterns, yet extrinsic stimulation also sculpts sensory development. Because the sensory and motor systems are interconnected in embryogenesis, how extrinsic sensory activation guides multimodal differentiation is an important topic. Further, it is unknown whether extrinsic stimulation experienced near sensory sensitivity onset contributes to persistent brain changes, ultimately affecting postnatal behavior. To determine the effects of extrinsic stimulation on multimodal development, we delivered auditory stimulation to bobwhite quail groups during early, middle, or late embryogenesis, and then tested postnatal behavioral responsiveness to auditory or visual cues. Auditory preference tendencies were more consistently toward the conspecific stimulus for animals stimulated during late embryogenesis. Groups stimulated during middle or late embryogenesis showed altered postnatal species-typical visual responsiveness, demonstrating a persistent multimodal effect. We also examined whether auditory-related brain regions are receptive to extrinsic input during middle embryogenesis by measuring postnatal cellular activation. Stimulated birds showed a greater number of ZENK-immunopositive cells per unit volume of brain tissue in deep optic tectum, a midbrain region strongly implicated in multimodal function. We observed similar results in the medial and caudomedial nidopallia in the telencephalon. There were no ZENK differences between groups in inferior colliculus or in caudolateral nidopallium, avian analog to prefrontal cortex. To our knowledge, these are the first results linking extrinsic stimulation delivered so early in embryogenesis to changes in postnatal multimodal behavior and cellular activation. The potential role of competitive interactions between the sensory and motor systems is discussed.     

Pollution going multimodal: the complex impact of the human-altered sensory environment on animal perception and performance

Anthropogenic sensory pollution is affecting ecosystems worldwide. Human actions generate acoustic noise, emanate artificial light and emit chemical substances. All of these pollutants are known to affect animals. Most studies on anthropogenic pollution address the impact of pollutants in unimodal sensory domains. High levels of anthropogenic noise, for example, have been shown to interfere with acoustic signals and cues. However, animals rely on multiple senses, and pollutants often co-occur. Thus, a full ecological assessment of the impact of anthropogenic activities requires a multimodal approach. We describe how sensory pollutants can co-occur and how covariance among pollutants may differ from natural situations. We review how animals combine information that arrives at their sensory systems through different modalities and outline how sensory conditions can interfere with multimodal perception. Finally, we describe how sensory pollutants can affect the perception, behaviour and endocrinology of animals within and across sensory modalities. We conclude that sensory pollution can affect animals in complex ways due to interactions among sensory stimuli, neural processing and behavioural and endocrinal feedback. We call for more empirical data on covariance among sensory conditions, for instance, data on correlated levels in noise and light pollution. Furthermore, we encourage researchers to test animal responses to a full-factorial set of sensory pollutants in the presence or the absence of ecologically important signals and cues. We realize that such approach is often time and energy consuming, but we think this is the only way to fully understand the multimodal impact of sensory pollution on animal performance and perception.     

Inhibition, not excitation, is the key to multimodal sensory integration

Multimodal neuronal maps, combining input from two or more sensory systems, play a key role in the processing of sensory and motor information. For such maps to be of any use, the input from all participating modalities must be calibrated so that a stimulus at a specific spatial location is represented at an unambiguous position in the multimodal map. Here we discuss two methods based on supervised spike-timing-dependent plasticity (STDP) to gauge input from different sensory modalities so as to ensure a proper map alignment. The first uses an excitatory teacher input. It is therefore called excitation-mediated learning. The second method is based on an inhibitory teacher signal, as found in the barn owl, and is called inhibition-mediated learning. Using detailed analytical calculations and numerical simulations, we demonstrate that inhibitory teacher input is essential if high-quality multimodal integration is to be learned rapidly. Furthermore, we show that the quality of the resulting map is not so much limited by the quality of the teacher signal but rather by the accuracy of the input from other sensory modalities.     

A framework for biomedical figure segmentation towards image-based document retrieval

The figures included in many of the biomedical publications play an important role in understanding the biological experiments and facts described within. Recent studies have shown that it is possible to integrate the information that is extracted from figures in classical document classification and retrieval tasks in order to improve their accuracy. One important observation about the figures included in biomedical publications is that they are often composed of multiple subfigures or panels, each describing different methodologies or results. The use of these multimodal figures is a common practice in bioscience, as experimental results are graphically validated via multiple methodologies or procedures. Thus, for a better use of multimodal figures in document classification or retrieval tasks, as well as for providing the evidence source for derived assertions, it is important to automatically segment multimodal figures into subfigures and panels. This is a challenging task, however, as different panels can contain similar objects (i.e., barcharts and linecharts) with multiple layouts. Also, certain types of biomedical figures are text-heavy (e.g., DNA sequences and protein sequences images) and they differ from traditional images. As a result, classical image segmentation techniques based on low-level image features, such as edges or color, are not directly applicable to robustly partition multimodal figures into single modal panels.In this paper, we describe a robust solution for automatically identifying and segmenting unimodal panels from a multimodal figure. Our framework starts by robustly harvesting figure-caption pairs from biomedical articles. We base our approach on the observation that the document layout can be used to identify encoded figures and figure boundaries within PDF files. Taking into consideration the document layout allows us to correctly extract figures from the PDF document and associate their corresponding caption. We combine pixel-level representations of the extracted images with information gathered from their corresponding captions to estimate the number of panels in the figure. Thus, our approach simultaneously identifies the number of panels and the layout of figures.In order to evaluate the approach described here, we applied our system on documents containing protein-protein interactions (PPIs) and compared the results against a gold standard that was annotated by biologists. Experimental results showed that our automatic figure segmentation approach surpasses pure caption-based and image-based approaches, achieving a 96.64% accuracy. To allow for efficient retrieval of information, as well as to provide the basis for integration into document classification and retrieval systems among other, we further developed a web-based interface that lets users easily retrieve panels containing the terms specified in the user queries.     

How do high-level specifications of the brain relate to variational approaches?

High-level specification of how the brain represents and categorizes the causes of its sensory input allows to link "what is to be done" (perceptual task) with "how to do it" (neural network calculation). In this article, we describe how the variational framework, which encountered a large success in modeling computer vision tasks, has some interesting relationships, at a mesoscopic scale, with computational neuroscience. We focus on cortical map computations such that "what is to be done" can be represented as a variational approach, i.e., an optimization problem defined over a continuous functional space. In particular, generalizing some existing results, we show how a general variational approach can be solved by an analog neural network with a given architecture and conversely. Numerical experiments are provided as an illustration of this general framework, which is a promising framework for modeling macro-behaviors in computational neuroscience.     

Thermodynamic Costs of Information Processing in Sensory Adaptation

Biological sensory systems react to changes in their surroundings. They are characterized by fast response and slow adaptation to varying environmental cues. Insofar as sensory adaptive systems map environmental changes to changes of their internal degrees of freedom, they can be regarded as computational devices manipulating information. Landauer established that information is ultimately physical, and its manipulation subject to the entropic and energetic bounds of thermodynamics. Thus the fundamental costs of biological sensory adaptation can be elucidated by tracking how the information the system has about its environment is altered. These bounds are particularly relevant for small organisms, which unlike everyday computers, operate at very low energies. In this paper, we establish a general framework for the thermodynamics of information processing in sensing. With it, we quantify how during sensory adaptation information about the past is erased, while information about the present is gathered. This process produces entropy larger than the amount of old information erased and has an energetic cost bounded by the amount of new information written to memory. We apply these principles to the E. coli''s chemotaxis pathway during binary ligand concentration changes. In this regime, we quantify the amount of information stored by each methyl group and show that receptors consume energy in the range of the information-theoretic minimum. Our work provides a basis for further inquiries into more complex phenomena, such as gradient sensing and frequency response.     

Communication in troubled waters: responses of fish communication systems to changing environments

Fish populations are increasingly being subjected to anthropogenic changes to their sensory environments. The impact of these changes on inter- and intra-specific communication, and its evolutionary consequences, has only recently started to receive research attention. A disruption of the sensory environment is likely to impact communication, especially with respect to reproductive interactions that help to maintain species boundaries. Aquatic ecosystems around the world are being threatened by a variety of environmental stressors, causing dramatic losses of biodiversity and bringing urgency to the need to understand how fish respond to rapid environmental changes. Here, we discuss current research on different communication systems (visual, chemical, acoustic, electric) and explore the state of our knowledge of how complex systems respond to environmental stressors using fish as a model. By far the bulk of our understanding comes from research on visual communication in the context of mate selection and competition for mates, while work on other communication systems is accumulating. In particular, it is increasingly acknowledged that environmental effects on one mode of communication may trigger compensation through other modalities. The strength and direction of selection on communication traits may vary if such compensation occurs. However, we find a dearth of studies that have taken a multimodal approach to investigating the evolutionary impact of environmental change on communication in fish. Future research should focus on the interaction between different modes of communication, especially under changing environmental conditions. Further, we see an urgent need for a better understanding of the evolutionary consequences of changes in communication systems on fish diversity.     

Predicting human perceptual decisions by decoding neuronal information profiles

Perception relies on the response of populations of neurons in sensory cortex. How the response profile of a neuronal population gives rise to perception and perceptual discrimination has been conceptualized in various ways. Here we suggest that neuronal population responses represent information about our environment explicitly as Fisher information (FI), which is a local measure of the variance estimate of the sensory input. We show how this sensory information can be read out and combined to infer from the available information profile which stimulus value is perceived during a fine discrimination task. In particular, we propose that the perceived stimulus corresponds to the stimulus value that leads to the same information for each of the alternative directions, and compare the model prediction to standard models considered in the literature (population vector, maximum likelihood, maximum-a-posteriori Bayesian inference). The models are applied to human performance in a motion discrimination task that induces perceptual misjudgements of a target direction of motion by task irrelevant motion in the spatial surround of the target stimulus (motion repulsion). By using the neurophysiological insight that surround motion suppresses neuronal responses to the target motion in the center, all models predicted the pattern of perceptual misjudgements. The variation of discrimination thresholds (error on the perceived value) was also explained through the changes of the total FI content with varying surround motion directions. The proposed FI decoding scheme incorporates recent neurophysiological evidence from macaque visual cortex showing that perceptual decisions do not rely on the most active neurons, but rather on the most informative neuronal responses. We statistically compare the prediction capability of the FI decoding approach and the standard decoding models. Notably, all models reproduced the variation of the perceived stimulus values for different surrounds, but with different neuronal tuning characteristics underlying perception. Compared to the FI approach the prediction power of the standard models was based on neurons with far wider tuning width and stronger surround suppression. Our study demonstrates that perceptual misjudgements can be based on neuronal populations encoding explicitly the available sensory information, and provides testable neurophysiological predictions on neuronal tuning characteristics underlying human perceptual decisions.     

Anticipatory Functions, Digital-Analog Forms and Biosemiotics: Integrating the Tools to Model Information and Normativity in Autonomous Biological Agents

We argue that living systems process information such that functionality emerges in them on a continuous basis. We then provide a framework that can explain and model the normativity of biological functionality. In addition we offer an explanation of the anticipatory nature of functionality within our overall approach. We adopt a Peircean approach to Biosemiotics, and a dynamical approach to Digital-Analog relations and to the interplay between different levels of functionality in autonomous systems, taking an integrative approach. We then apply the underlying biosemiotic logic to a particular biological system, giving a model of the B-Cell Receptor signaling system, in order to demonstrate how biosemiotic concepts can be used to build an account of biological information and functionality. Next we show how this framework can be used to explain and model more complex aspects of biological normativity, for example, how cross-talk between different signaling pathways can be avoided. Overall, we describe an integrated theoretical framework for the emergence of normative functions and, consequently, for the way information is transduced across several interconnected organizational levels in an autonomous system, and we demonstrate how this can be applied in real biological phenomena. Our aim is to open the way towards realistic tools for the modeling of information and normativity in autonomous biological agents.     

Small is beautiful: models of small neuronal networks

Modeling has contributed a great deal to our understanding of how individual neurons and neuronal networks function. In this review, we focus on models of the small neuronal networks of invertebrates, especially rhythmically active CPG networks. Models have elucidated many aspects of these networks, from identifying key interacting membrane properties to pointing out gaps in our understanding, for example missing neurons. Even the complex CPGs of vertebrates, such as those that underlie respiration, have been reduced to small network models to great effect. Modeling of these networks spans from simplified models, which are amenable to mathematical analyses, to very complicated biophysical models. Some researchers have now adopted a population approach, where they generate and analyze many related models that differ in a few to several judiciously chosen free parameters; often these parameters show variability across animals and thus justify the approach. Models of small neuronal networks will continue to expand and refine our understanding of how neuronal networks in all animals program motor output, process sensory information and learn.     

Issues in the classification of multimodal communication signals

Communication involves complex behavior in multiple sensory channels, or "modalities." We provide an overview of multimodal communication and its costs and benefits, place examples of signals and displays from an array of taxa, sensory systems, and functions into our signal classification system, and consider issues surrounding the categorization of multimodal signals. The broadest level of classification is between signals with redundant and nonredundant components, with finer distinctions in each category. We recommend that researchers gather information on responses to each component of a multimodal signal as well as the response to the signal as a whole. We discuss the choice of categories, whether to categorize signals on the basis of the signal or the response, and how to classify signals if data are missing. The choice of behavioral assay may influence the outcome, as may the context of the communicative event. We also consider similarities and differences between multimodal and unimodal composite signals and signals that are sequentially, rather than simultaneously, multimodal.     

Active touch sensing

Active sensing systems are purposive and information-seeking sensory systems. Active sensing usually entails sensor movement, but more fundamentally, it involves control of the sensor apparatus, in whatever manner best suits the task, so as to maximize information gain. In animals, active sensing is perhaps most evident in the modality of touch. In this theme issue, we look at active touch across a broad range of species from insects, terrestrial and marine mammals, through to humans. In addition to analysing natural touch, we also consider how engineering is beginning to exploit physical analogues of these biological systems so as to endow robots with rich tactile sensing capabilities. The different contributions show not only the varieties of active touch--antennae, whiskers and fingertips--but also their commonalities. They explore how active touch sensing has evolved in different animal lineages, how it serves to provide rapid and reliable cues for controlling ongoing behaviour, and even how it can disintegrate when our brains begin to fail. They demonstrate that research on active touch offers a means both to understand this essential and primary sensory modality, and to investigate how animals, including man, combine movement with sensing so as to make sense of, and act effectively in, the world.     

Engineering entrainment and adaptation in limit cycle systems

Periodic behavior is key to life and is observed in multiple instances and at multiple time scales in our metabolism, our natural environment, and our engineered environment. A natural way of modeling or generating periodic behavior is done by using oscillators, i.e., dynamical systems that exhibit limit cycle behavior. While there is extensive literature on methods to analyze such dynamical systems, much less work has been done on methods to synthesize an oscillator to exhibit some specific desired characteristics. The goal of this article is twofold: (1) to provide a framework for characterizing and designing oscillators and (2) to review how classes of well-known oscillators can be understood and related to this framework. The basis of the framework is to characterize oscillators in terms of their fundamental temporal and spatial behavior and in terms of properties that these two behaviors can be designed to exhibit. This focus on fundamental properties is important because it allows us to systematically compare a large variety of oscillators that might at first sight appear very different from each other. We identify several specifications that are useful for design, such as frequency-locking behavior, phase-locking behavior, and specific output signal shape. We also identify two classes of design methods by which these specifications can be met, namely offline methods and online methods. By relating these specifications to our framework and by presenting several examples of how oscillators have been designed in the literature, this article provides a useful methodology and toolbox for designing oscillators for a wide range of purposes. In particular, the focus on synthesis of limit cycle dynamical systems should be useful both for engineering and for computational modeling of physical or biological phenomena.     

Tuning curves, neuronal variability, and sensory coding

Tuning curves are widely used to characterize the responses of sensory neurons to external stimuli, but there is an ongoing debate as to their role in sensory processing. Commonly, it is assumed that a neuron's role is to encode the stimulus at the tuning curve peak, because high firing rates are the neuron's most distinct responses. In contrast, many theoretical and empirical studies have noted that nearby stimuli are most easily discriminated in high-slope regions of the tuning curve. Here, we demonstrate that both intuitions are correct, but that their relative importance depends on the experimental context and the level of variability in the neuronal response. Using three different information-based measures of encoding applied to experimentally measured sensory neurons, we show how the best-encoded stimulus can transition from high-slope to high-firing-rate regions of the tuning curve with increasing noise level. We further show that our results are consistent with recent experimental findings that correlate neuronal sensitivities with perception and behavior. This study illustrates the importance of the noise level in determining the encoding properties of sensory neurons and provides a unified framework for interpreting how the tuning curve and neuronal variability relate to the overall role of the neuron in sensory encoding.     

Recognizing Sequences of Sequences

The brain''s decoding of fast sensory streams is currently impossible to emulate, even approximately, with artificial agents. For example, robust speech recognition is relatively easy for humans but exceptionally difficult for artificial speech recognition systems. In this paper, we propose that recognition can be simplified with an internal model of how sensory input is generated, when formulated in a Bayesian framework. We show that a plausible candidate for an internal or generative model is a hierarchy of ‘stable heteroclinic channels’. This model describes continuous dynamics in the environment as a hierarchy of sequences, where slower sequences cause faster sequences. Under this model, online recognition corresponds to the dynamic decoding of causal sequences, giving a representation of the environment with predictive power on several timescales. We illustrate the ensuing decoding or recognition scheme using synthetic sequences of syllables, where syllables are sequences of phonemes and phonemes are sequences of sound-wave modulations. By presenting anomalous stimuli, we find that the resulting recognition dynamics disclose inference at multiple time scales and are reminiscent of neuronal dynamics seen in the real brain.     

Measuring Information-Transfer Delays

In complex networks such as gene networks, traffic systems or brain circuits it is important to understand how long it takes for the different parts of the network to effectively influence one another. In the brain, for example, axonal delays between brain areas can amount to several tens of milliseconds, adding an intrinsic component to any timing-based processing of information. Inferring neural interaction delays is thus needed to interpret the information transfer revealed by any analysis of directed interactions across brain structures. However, a robust estimation of interaction delays from neural activity faces several challenges if modeling assumptions on interaction mechanisms are wrong or cannot be made. Here, we propose a robust estimator for neuronal interaction delays rooted in an information-theoretic framework, which allows a model-free exploration of interactions. In particular, we extend transfer entropy to account for delayed source-target interactions, while crucially retaining the conditioning on the embedded target state at the immediately previous time step. We prove that this particular extension is indeed guaranteed to identify interaction delays between two coupled systems and is the only relevant option in keeping with Wiener’s principle of causality. We demonstrate the performance of our approach in detecting interaction delays on finite data by numerical simulations of stochastic and deterministic processes, as well as on local field potential recordings. We also show the ability of the extended transfer entropy to detect the presence of multiple delays, as well as feedback loops. While evaluated on neuroscience data, we expect the estimator to be useful in other fields dealing with network dynamics.     

Frequency and intensity of facilitation reveal opposing patterns along a stress gradient

Disentangling the different processes structuring ecological communities is a long‐standing challenge. In species‐rich ecosystems, most emphasis has so far been given to environmental filtering and competition processes, while facilitative interactions between species remain insufficiently studied. Here, we propose an analysis framework that not only allows for identifying pairs of facilitating and facilitated species, but also estimates the strength of facilitation and its variation along environmental gradients. Our framework combines the analysis of both co‐occurrence and co‐abundance patterns using a moving window approach along environmental gradients to control for potentially confounding effects of environmental filtering in the co‐abundance analysis. We first validate our new approach against community assembly simulations, and exemplify its potential on a large 1,134 plant community plots dataset. Our results generally show that facilitation intensity was strongest under cold stress, whereas the proportion of facilitating and facilitated species was higher under drought stress. Moreover, the functional distance between individual facilitated species and their facilitating species significantly changed along the temperature–moisture gradient, and seemed to influence facilitation intensity, although no general positive or general negative trend was discernible among species. The main advantages of our robust framework are as follows: It enables detecting facilitating and facilitated species in species‐rich systems, and it allows identifying the directionality and intensity of facilitation in species pairs as well as its variation across long environmental gradients. It thus opens numerous opportunities for incorporating functional (and phylogenetic) information in the analysis of facilitation patterns. Our case study indicated high complexity in facilitative interactions across the stress gradient and revealed new evidence that facilitation, similarly to competition, can operate between functionally similar and dissimilar species. Extending the analyses to other taxa and ecosystems will foster our understanding how complex interspecific interactions promote biodiversity.     

Technological advances in mosquito olfaction neurogenetics

Gene-editing technologies have revolutionized the field of mosquito sensory biology. These technologies have been used to knock in reporter genes in-frame with neuronal genes and tag specific mosquito neurons to detect their activities using binary expression systems. Despite these advances, novel tools still need to be developed to elucidate the transmission of olfactory signals from the periphery to the brain. Here, we propose the development of a set of tools, including novel driver lines as well as sensors of neuromodulatory activities, which can advance our knowledge of how sensory input triggers behavioral outputs. This information can change our understanding of mosquito neurobiology and lead to the development of strategies for mosquito behavioral manipulation to reduce bites and disease transmission.     

Fast coding of orientation in primary visual cortex

Understanding how populations of neurons encode sensory information is a major goal of systems neuroscience. Attempts to answer this question have focused on responses measured over several hundred milliseconds, a duration much longer than that frequently used by animals to make decisions about the environment. How reliably sensory information is encoded on briefer time scales, and how best to extract this information, is unknown. Although it has been proposed that neuronal response latency provides a major cue for fast decisions in the visual system, this hypothesis has not been tested systematically and in a quantitative manner. Here we use a simple 'race to threshold' readout mechanism to quantify the information content of spike time latency of primary visual (V1) cortical cells to stimulus orientation. We find that many V1 cells show pronounced tuning of their spike latency to stimulus orientation and that almost as much information can be extracted from spike latencies as from firing rates measured over much longer durations. To extract this information, stimulus onset must be estimated accurately. We show that the responses of cells with weak tuning of spike latency can provide a reliable onset detector. We find that spike latency information can be pooled from a large neuronal population, provided that the decision threshold is scaled linearly with the population size, yielding a processing time of the order of a few tens of milliseconds. Our results provide a novel mechanism for extracting information from neuronal populations over the very brief time scales in which behavioral judgments must sometimes be made.