Multisensory integration: attending to seen and felt hands

The neglect of one side of space exhibited by some brain-damaged patients can be ameliorated by cueing the patient to the neglected side of space. A related effect has been found to depend on the hand being seen and felt at the same time. The results add to a growing literature on somatosensory-visual interactions.     

Multisensory integration: strategies for synchronization

Recent studies have shown that our ability to match the timing of inputs from objects that can be both seen and heard is highly adaptable and takes into account target depth and the relatively slow speed of sound.     

Multisensory integration: a late bloomer

Under many circumstances, human adults integrate information from different sensory modalities, such as vision and hearing, in a statistically optimal fashion. New results suggest that optimal multisensory integration only develops in middle childhood.     

Multisensory integration: maintaining the perception of synchrony

We are rarely aware of differences in the arrival time of inputs to each of our senses. New research suggests that this is explained by a 'moveable window' for multisensory integration and by a 'temporal ventriloquism' effect.     

Multisensory integration: space, time and superadditivity

The superior colliculus generates and controls eye and head movements based on signals from different senses. The latest research on this structure enhances our understanding of the mechanisms of multisensory integration in the brain.     

Multisensory integration: perceptual grouping by eye and ear

Recent studies using modern functional imaging methods have provided insights into the brain areas involved in combining information across the senses, and the manner in which sensory signals are integrated within them. These findings are remarkably consistent with the results from single-unit recording in animals.     

Arches of the hand in reach to grasp

Topographically, the hand is described by its anterior (palmar) and posterior (dorsal) surfaces that encompass a hollow cavity that changes its shape during hand preshaping and grasping according to the object to be grasped. The hollow cavity has been described as consisting of three arches that run in different directions: transverse, longitudinal and oblique, spanning the anterior surface of the hand. Although described anatomically, the modulation in the palmar arches has not been investigated kinematically during actual grasping. In this study, we describe and compare biomechanical formulations of the palmar arch, specifically, the distal transverse and the oblique arches. In addition, we introduce another biomechanical formulation of the palmar arch, called the kinematic transverse arch that takes account of the thenar and hypothenar involvement in arch formation. Hand shape modulation during two natural power-grasping tasks was studied in eight healthy adults. Results showed a significant influence of the overall contribution of thenar and hypothenar movement during hand shape modulation. While there was relatively more thenar contribution during transport shaping, more hypothenar contribution was evident during preshaping and contact shaping-the two phases of grasping during which the hand establishes contact with the object. The advantage of the new formulation is that it better described the contributions from thenar and hypothenar movement to palmar arch formation which may be a more accurate depiction of hand preshaping during grasping.     

Multisensory integration: how visual experience shapes spatial perception

The localisation of auditory and tactile events is strongly affected by visual information, reflecting the dominant role of vision in spatial perception. New research suggests that early visual experience is critical for the establishment of such multisensory links.     

Multisensory integration in the superior colliculus: a neural network model

Neurons in the superior colliculus (SC) are known to integrate stimuli of different modalities (e.g., visual and auditory) following specific properties. In this work, we present a mathematical model of the integrative response of SC neurons, in order to suggest a possible physiological mechanism underlying multisensory integration in SC. The model includes three distinct neural areas: two unimodal areas (auditory and visual) are devoted to a topological representation of external stimuli, and communicate via synaptic connections with a third downstream area (in the SC) responsible for multisensory integration. The present simulations show that the model, with a single set of parameters, can mimic various responses to different combinations of external stimuli including the inverse effectiveness, both in terms of multisensory enhancement and contrast, the existence of within- and cross-modality suppression between spatially disparate stimuli, a reduction of network settling time in response to cross-modal stimuli compared with individual stimuli. The model suggests that non-linearities in neural responses and synaptic (excitatory and inhibitory) connections can explain several aspects of multisensory integration.     

Multisensory integration: current issues from the perspective of the single neuron

For thousands of years science philosophers have been impressed by how effectively the senses work together to enhance the salience of biologically meaningful events. However, they really had no idea how this was accomplished. Recent insights into the underlying physiological mechanisms reveal that, in at least one circuit, this ability depends on an intimate dialogue among neurons at multiple levels of the neuraxis; this dialogue cannot take place until long after birth and might require a specific kind of experience. Understanding the acquisition and usage of multisensory integration in the midbrain and cerebral cortex of mammals has been aided by a multiplicity of approaches. Here we examine some of the fundamental advances that have been made and some of the challenging questions that remain.     

Multisensory integration: what you see is where you hear

Recent studies of multisensory integration compel a redefinition of fundamental sensory processes, including, but not limited to, how visual inputs influence the localization of sounds and suppression of their echoes.     

Multisensory integration: methodological approaches and emerging principles in the human brain.

Understanding the conditions under which the brain integrates the different sensory streams and the mechanisms supporting this phenomenon is now a question at the forefront of neuroscience. In this paper, we discuss the opportunities for investigating these multisensory processes using modern imaging techniques, the nature of the information obtainable from each method and their benefits and limitations. Despite considerable variability in terms of paradigm design and analysis, some consistent findings are beginning to emerge. The detection of brain activity in human neuroimaging studies that resembles multisensory integration responses at the cellular level in other species, suggests similar crossmodal binding mechanisms may be operational in the human brain. These mechanisms appear to be distributed across distinct neuronal networks that vary depending on the nature of the shared information between different sensory cues. For example, differing extents of correspondence in time, space or content seem to reliably bias the involvement of different integrative networks which code for these cues. A combination of data obtained from haemodynamic and electromagnetic methods, which offer high spatial or temporal resolution respectively, are providing converging evidence of multisensory interactions at both "early" and "late" stages of processing--suggesting a cascade of synergistic processes operating in parallel at different levels of the cortex.     

Multisensory integration in cortex: shedding light on prickly issues

Interactions between different sensory modalities can be observed in unimodal areas of the cortex, as revealed by recent neuroimaging studies. A new report by Macaluso and colleagues ( [this issue of Neuron]) shows that crossmodal effects of tactile stimulation in visual cortex critically depend on the spatial congruence of multisensory inputs. This work is discussed in relation to neural and computational models of multisensory integration.     

Multisensory integration of colors and scents: insights from bees and flowers

Karl von Frisch’s studies of bees’ color vision and chemical senses opened a window into the perceptual world of a species other than our own. A century of subsequent research on bees’ visual and olfactory systems has developed along two productive but independent trajectories, leaving the questions of how and why bees use these two senses in concert largely unexplored. Given current interest in multimodal communication and recently discovered interplay between olfaction and vision in humans and Drosophila, understanding multisensory integration in bees is an opportunity to advance knowledge across fields. Using a classic ethological framework, we formulate proximate and ultimate perspectives on bees’ use of multisensory stimuli. We discuss interactions between scent and color in the context of bee cognition and perception, focusing on mechanistic and functional approaches, and we highlight opportunities to further explore the development and evolution of multisensory integration. We argue that although the visual and olfactory worlds of bees are perhaps the best-studied of any non-human species, research focusing on the interactions between these two sensory modalities is vitally needed.     

Multisensory integration of natural odors and sounds in the auditory cortex

Motherhood is associated with different forms of physiological alterations including transient hormonal changes and brain plasticity. The underlying impact of these changes on the emergence of maternal behaviors and sensory processing within the mother's brain are largely unknown. By using in?vivo cell-attached recordings in the primary auditory cortex of female mice, we discovered that exposure to pups' body odor reshapes neuronal responses to pure tones and natural auditory stimuli. This olfactory-auditory interaction appeared naturally in lactating mothers shortly after parturition and was long lasting. Naive virgins that had experience with the pups also showed an appearance of olfactory-auditory integration in A1, suggesting that multisensory integration may be experience dependent. Neurons from lactating mothers were more sensitive to sounds as compared to those from experienced mice, independent of the odor effects. These uni- and multisensory cortical changes may facilitate the detection and discrimination of pup distress calls and strengthen the bond between mothers and their neonates. VIDEO ABSTRACT:     

Multisensory integration and attention in autism spectrum disorder: evidence from event-related potentials

Successful integration of various simultaneously perceived perceptual signals is crucial for social behavior. Recent findings indicate that this multisensory integration (MSI) can be modulated by attention. Theories of Autism Spectrum Disorders (ASDs) suggest that MSI is affected in this population while it remains unclear to what extent this is related to impairments in attentional capacity. In the present study Event-related potentials (ERPs) following emotionally congruent and incongruent face-voice pairs were measured in 23 high-functioning, adult ASD individuals and 24 age- and IQ-matched controls. MSI was studied while the attention of the participants was manipulated. ERPs were measured at typical auditory and visual processing peaks, namely, P2 and N170. While controls showed MSI during divided attention and easy selective attention tasks, individuals with ASD showed MSI during easy selective attention tasks only. It was concluded that individuals with ASD are able to process multisensory emotional stimuli, but this is differently modulated by attention mechanisms in these participants, especially those associated with divided attention. This atypical interaction between attention and MSI is also relevant to treatment strategies, with training of multisensory attentional control possibly being more beneficial than conventional sensory integration therapy.