Modality distribution of sensory neurons in the feline caudate nucleus and the substantia nigra

Despite extensive analysis of the motor functions of the basal ganglia and the fact that multisensory information processing appears critical for the execution of their behavioral action, little is known concerning the sensory functions of the caudate nucleus (CN) and the substantia nigra (SN). In the present study, we set out to describe the sensory modality distribution and to determine the proportions of multisensory units within the CN and the SN. The separate single sensory modality tests demonstrated that a majority of the neurons responded to only one modality, so that they seemed to be unimodal. In contrast with these findings, a large proportion of these neurons exhibited significant multisensory cross-modal interactions. Thus, these neurons should also be classified as multisensory. Our results suggest that a surprisingly high proportion of sensory neurons in the basal ganglia are multisensory, and demonstrate that an analysis without a consideration of multisensory cross-modal interactions may strongly underrepresent the number of multisensory units. We conclude that a majority of the sensory neurons in the CN and SN process multisensory information and only a minority of these units are clearly unimodal.     

Neuronal oscillations and multisensory interaction in primary auditory cortex

Recent anatomical, physiological, and neuroimaging findings indicate multisensory convergence at early, putatively unisensory stages of cortical processing. The objective of this study was to confirm somatosensory-auditory interaction in A1 and to define both its physiological mechanisms and its consequences for auditory information processing. Laminar current source density and multiunit activity sampled during multielectrode penetrations of primary auditory area A1 in awake macaques revealed clear somatosensory-auditory interactions, with a novel mechanism: somatosensory inputs appear to reset the phase of ongoing neuronal oscillations, so that accompanying auditory inputs arrive during an ideal, high-excitability phase, and produce amplified neuronal responses. In contrast, responses to auditory inputs arriving during the opposing low-excitability phase tend to be suppressed. Our findings underscore the instrumental role of neuronal oscillations in cortical operations. The timing and laminar profile of the multisensory interactions in A1 indicate that nonspecific thalamic systems may play a key role in the effect.     

Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model

The superior colliculus (SC) integrates relevant sensory information (visual, auditory, somatosensory) from several cortical and subcortical structures, to program orientation responses to external events. However, this capacity is not present at birth, and it is acquired only through interactions with cross-modal events during maturation. Mathematical models provide a quantitative framework, valuable in helping to clarify the specific neural mechanisms underlying the maturation of the multisensory integration in the SC. We extended a neural network model of the adult SC (Cuppini et?al., Front Integr Neurosci 4:1?C15, 2010) to describe the development of this phenomenon starting from an immature state, based on known or suspected anatomy and physiology, in which: (1) AES afferents are present but weak, (2) Responses are driven from non-AES afferents, and (3) The visual inputs have a marginal spatial tuning. Sensory experience was modeled by repeatedly presenting modality-specific and cross-modal stimuli. Synapses in the network were modified by simple Hebbian learning rules. As a consequence of this exposure, (1) Receptive fields shrink and come into spatial register, and (2) SC neurons gained the adult characteristic integrative properties: enhancement, depression, and inverse effectiveness. Importantly, the unique architecture of the model guided the development so that integration became dependent on the relationship between the cortical input and the SC. Manipulations of the statistics of the experience during the development changed the integrative profiles of the neurons, and results matched well with the results of physiological studies.     

Dynamic characteristics of multisensory facilitation and inhibition

Multimodal integration, which mainly refers to multisensory facilitation and multisensory inhibition, is the process of merging multisensory information in the human brain. However, the neural mechanisms underlying the dynamic characteristics of multimodal integration are not fully understood. The objective of this study is to investigate the basic mechanisms of multimodal integration by assessing the intermodal influences of vision, audition, and somatosensory sensations (the influence of multisensory background events to the target event). We used a timed target detection task, and measured both behavioral and electroencephalographic responses to visual target events (green solid circle), auditory target events (2 kHz pure tone) and somatosensory target events (1.5 ± 0.1 mA square wave pulse) from 20 normal participants. There were significant differences in both behavior performance and ERP components when comparing the unimodal target stimuli with multimodal (bimodal and trimodal) target stimuli for all target groups. Significant correlation among reaction time and P3 latency was observed across all target conditions. The perceptual processing of auditory target events (A) was inhibited by the background events, while the perceptual processing of somatosensory target events (S) was facilitated by the background events. In contrast, the perceptual processing of visual target events (V) remained impervious to multisensory background events.     

Cross-Modal Stimulus Conflict: The Behavioral Effects of Stimulus Input Timing in a Visual-Auditory Stroop Task

Cross-modal processing depends strongly on the compatibility between different sensory inputs, the relative timing of their arrival to brain processing components, and on how attention is allocated. In this behavioral study, we employed a cross-modal audio-visual Stroop task in which we manipulated the within-trial stimulus-onset-asynchronies (SOAs) of the stimulus-component inputs, the grouping of the SOAs (blocked vs. random), the attended modality (auditory or visual), and the congruency of the Stroop color-word stimuli (congruent, incongruent, neutral) to assess how these factors interact within a multisensory context. One main result was that visual distractors produced larger incongruency effects on auditory targets than vice versa. Moreover, as revealed by both overall shorter response times (RTs) and relative shifts in the psychometric incongruency-effect functions, visual-information processing was faster and produced stronger and longer-lasting incongruency effects than did auditory. When attending to either modality, stimulus incongruency from the other modality interacted with SOA, yielding larger effects when the irrelevant distractor occurred prior to the attended target, but no interaction with SOA grouping. Finally, relative to neutral-stimuli, and across the wide range of the SOAs employed, congruency led to substantially more behavioral facilitation than did incongruency to interference, in contrast to findings that within-modality stimulus-compatibility effects tend to be more evenly split between facilitation and interference. In sum, the present findings reveal several key characteristics of how we process the stimulus compatibility of cross-modal sensory inputs, reflecting stimulus processing patterns that are critical for successfully navigating our complex multisensory world.     

Sensorimotor experience influences recovery of forelimb abilities but not tissue loss after focal cortical compression in adult rats

Sensorimotor activity has been shown to play a key role in functional outcome after extensive brain damage. This study was aimed at assessing the influence of sensorimotor experience through subject-environment interactions on the time course of both lesion and gliosis volumes as well as on the recovery of forelimb sensorimotor abilities following focal cortical injury. The lesion consisted of a cortical compression targeting the forepaw representational area within the primary somatosensory cortex of adult rats. After the cortical lesion, rats were randomly subjected to various postlesion conditions: unilateral C5-C6 dorsal root transection depriving the contralateral cortex from forepaw somatosensory inputs, standard housing or an enriched environment promoting sensorimotor experience and social interactions. Behavioral tests were used to assess forelimb placement during locomotion, forelimb-use asymmetry, and forepaw tactile sensitivity. For each group, the time course of tissue loss was described and the gliosis volume over the first postoperative month was evaluated using an unbiased stereological method. Consistent with previous studies, recovery of behavioral abilities was found to depend on post-injury experience. Indeed, increased sensorimotor activity initiated early in an enriched environment induced a rapid and more complete behavioral recovery compared with standard housing. In contrast, severe deprivation of peripheral sensory inputs led to a delayed and only partial sensorimotor recovery. The dorsal rhizotomy was found to increase the perilesional gliosis in comparison to standard or enriched environments. These findings provide further evidence that early sensory experience has a beneficial influence on the onset and time course of functional recovery after focal brain injury.     

Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex

BACKGROUND: Integrating information from the different senses markedly enhances the detection and identification of external stimuli. Compared with unimodal inputs, semantically and/or spatially congruent multisensory cues speed discrimination and improve reaction times. Discordant inputs have the opposite effect, reducing performance and slowing responses. These behavioural features of crossmodal processing appear to have parallels in the response properties of multisensory cells in the superior colliculi and cerebral cortex of non-human mammals. Although spatially concordant multisensory inputs can produce a dramatic, often multiplicative, increase in cellular activity, spatially disparate cues tend to induce a profound response depression. RESULTS: Using functional magnetic resonance imaging (fMRI), we investigated whether similar indices of crossmodal integration are detectable in human cerebral cortex, and for the synthesis of complex inputs relating to stimulus identity. Ten human subjects were exposed to varying epochs of semantically congruent and incongruent audio-visual speech and to each modality in isolation. Brain activations to matched and mismatched audio-visual inputs were contrasted with the combined response to both unimodal conditions. This strategy identified an area of heteromodal cortex in the left superior temporal sulcus that exhibited significant supra-additive response enhancement to matched audio-visual inputs and a corresponding sub-additive response to mismatched inputs. CONCLUSIONS: The data provide fMRI evidence of crossmodal binding by convergence in the human heteromodal cortex. They further suggest that response enhancement and depression may be a general property of multisensory integration operating at different levels of the neuroaxis and irrespective of the purpose for which sensory inputs are combined.     

Paving the way forward: integrating the senses through phase-resetting of cortical oscillations

Most, if not all, of the neocortex is multisensory, but the mechanisms by which different cortical areas - association versus sensory, for instance - integrate multisensory inputs are not known. The study by Lakatos et al. reveals that, in the primary auditory cortex, the phase of neural oscillations is reset by somatosensory inputs, and subsequent auditory inputs are enhanced or suppressed, depending on their timing relative to the oscillatory cycle.     

Somatosensory Event-related Potentials from Orofacial Skin Stretch Stimulation

Cortical processing associated with orofacial somatosensory function in speech has received limited experimental attention due to the difficulty of providing precise and controlled stimulation. This article introduces a technique for recording somatosensory event-related potentials (ERP) that uses a novel mechanical stimulation method involving skin deformation using a robotic device. Controlled deformation of the facial skin is used to modulate kinesthetic inputs through excitation of cutaneous mechanoreceptors. By combining somatosensory stimulation with electroencephalographic recording, somatosensory evoked responses can be successfully measured at the level of the cortex. Somatosensory stimulation can be combined with the stimulation of other sensory modalities to assess multisensory interactions. For speech, orofacial stimulation is combined with speech sound stimulation to assess the contribution of multi-sensory processing including the effects of timing differences. The ability to precisely control orofacial somatosensory stimulation during speech perception and speech production with ERP recording is an important tool that provides new insight into the neural organization and neural representations for speech.     

Behavioral and neuroplastic changes in the blind: evidence for functionally relevant cross-modal interactions.

The study of blind individuals provides insight into the brain re-organization and behavioral compensations that occur following sensory deprivation. While behavioral studies have yielded conflicting results in terms of performance levels within the remaining senses, deafferentation of visual cortical areas through peripheral blindness results in clear neuroplastic changes. Most striking is the activation of occipital cortex in response to auditory and tactile stimulation. Indeed, parts of the "unimodal" visual cortex are recruited by other sensory modalities to process sensory information in a functionally relevant manner. In addition, a larger area of the sensorimotor cortex is devoted to the representation of the reading finger in blind Braille readers. The "visual" function of the deafferented occipital cortex is also altered, where transcranial magnetic stimulation-induced phosphenes can be elicited in only 20% of blind subjects. The neural mechanisms underlying these changes remain elusive but recent data showing rapid cross-modal plasticity in blindfolded, sighted subjects argue against the establishment of new connections to explain cross-modal interactions in the blind. Rather, latent pathways that participate in multisensory percepts in sighted subjects might be unmasked and may be potentiated in the event of complete loss of visual input. These issues have important implications for the development of visual prosthesis aimed at restoring some degree of vision in the blind.     

Visuo-tactile representation of near-the-body space.

Here we report findings from neuropsychological investigations showing the existence, in humans, of intersensory integrative systems representing space through the multisensory coding of visual and tactile events. In addition, these findings show that visuo-tactile integration may take place in a privileged manner within a limited sector of space closely surrounding the body surface, i.e., the near-peripersonal space. They also demonstrate that the representation of near-peripersonal space is not static, as objects in the out-of-reach space can be processed as nearer, depending upon the (illusory) visual information about hand position in space, and the use of tools as physical extensions of the reachable space. Finally, new evidence is provided suggesting the multisensory coding of peripersonal space can be achieved through bottom-up processing that, at least in some instances, is not necessarily modulated by more "cognitive" top-down processing, such as the expectation regarding the possibility of being touched. These findings are entirely consistent with the functional properties of multisensory neuronal structures coding near-peripersonal space in monkeys, as well as with behavioral, and neuroimaging evidence for the cross-modal coding of space in normal subjects. This high level of convergence ultimately favors the idea that multisensory space coding is achieved through similar multimodal structures in both humans and non-human primates.     

Integration of bimodal looming signals through neuronal coherence in the temporal lobe

The ability to integrate information across multiple sensory systems offers several behavioral advantages, from quicker reaction times and more accurate responses to better detection and more robust learning. At the neural level, multisensory integration requires large-scale interactions between different brain regions--the convergence of information from separate sensory modalities, represented by distinct neuronal populations. The interactions between these neuronal populations must be fast and flexible, so that behaviorally relevant signals belonging to the same object or event can be immediately integrated and integration of unrelated signals can be prevented. Looming signals are a particular class of signals that are behaviorally relevant for animals and that occur in both the auditory and visual domain. These signals indicate the rapid approach of objects and provide highly salient warning cues about impending impact. We show here that multisensory integration of auditory and visual looming signals may be mediated by functional interactions between auditory cortex and the superior temporal sulcus, two areas involved in integrating behaviorally relevant auditory-visual signals. Audiovisual looming signals elicited increased gamma-band coherence between these areas, relative to unimodal or receding-motion signals. This suggests that the neocortex uses fast, flexible intercortical interactions to mediate multisensory integration.     

Integrating information from different senses in the auditory cortex

Multisensory integration was once thought to be the domain of brain areas high in the cortical hierarchy, with early sensory cortical fields devoted to unisensory processing of inputs from their given set of sensory receptors. More recently, a wealth of evidence documenting visual and somatosensory responses in auditory cortex, even as early as the primary fields, has changed this view of cortical processing. These multisensory inputs may serve to enhance responses to sounds that are accompanied by other sensory cues, effectively making them easier to hear, but may also act more selectively to shape the receptive field properties of auditory cortical neurons to the location or identity of these events. We discuss the new, converging evidence that multiplexing of neural signals may play a key role in informatively encoding and integrating signals in auditory cortex across multiple sensory modalities. We highlight some of the many open research questions that exist about the neural mechanisms that give rise to multisensory integration in auditory cortex, which should be addressed in future experimental and theoretical studies.     

Theta band zero-lag long-range cortical synchronization via hippocampal dynamical relaying

Growing evidence suggests that synchronization among distributed neuronal networks underlie functional integration in the brain. Neural synchronization is typically revealed by a consistent phase delay between neural responses generated in two separated sources. But the influence of a third neuronal assembly in that synchrony pattern remains largely unexplored. We investigate here the potential role of the hippocampus in determining cortico-cortical theta synchronization in different behavioral states during motor quiescent and while animals actively explore the environment. To achieve this goal, the two states were modeled with a recurrent network involving the hippocampus, as a relay element, and two distant neocortical sites. We found that cortico-cortical neural coupling accompanied higher hippocampal theta oscillations in both behavioral states, although the highest level of synchronization between cortical regions emerged during motor exploration. Local field potentials recorded from the same brain regions qualitatively confirm these findings in the two behavioral states. These results suggest that zero-lag long-range cortico-cortical synchronization is likely mediated by hippocampal theta oscillations in lower mammals as a function of cognitive demands and motor acts.     

An fMRI Study of Audiovisual Speech Perception Reveals Multisensory Interactions in Auditory Cortex

Research on the neural basis of speech-reading implicates a network of auditory language regions involving inferior frontal cortex, premotor cortex and sites along superior temporal cortex. In audiovisual speech studies, neural activity is consistently reported in posterior superior temporal Sulcus (pSTS) and this site has been implicated in multimodal integration. Traditionally, multisensory interactions are considered high-level processing that engages heteromodal association cortices (such as STS). Recent work, however, challenges this notion and suggests that multisensory interactions may occur in low-level unimodal sensory cortices. While previous audiovisual speech studies demonstrate that high-level multisensory interactions occur in pSTS, what remains unclear is how early in the processing hierarchy these multisensory interactions may occur. The goal of the present fMRI experiment is to investigate how visual speech can influence activity in auditory cortex above and beyond its response to auditory speech. In an audiovisual speech experiment, subjects were presented with auditory speech with and without congruent visual input. Holding the auditory stimulus constant across the experiment, we investigated how the addition of visual speech influences activity in auditory cortex. We demonstrate that congruent visual speech increases the activity in auditory cortex.     

Audio-Tactile Integration and the Influence of Musical Training

Perception of our environment is a multisensory experience; information from different sensory systems like the auditory, visual and tactile is constantly integrated. Complex tasks that require high temporal and spatial precision of multisensory integration put strong demands on the underlying networks but it is largely unknown how task experience shapes multisensory processing. Long-term musical training is an excellent model for brain plasticity because it shapes the human brain at functional and structural levels, affecting a network of brain areas. In the present study we used magnetoencephalography (MEG) to investigate how audio-tactile perception is integrated in the human brain and if musicians show enhancement of the corresponding activation compared to non-musicians. Using a paradigm that allowed the investigation of combined and separate auditory and tactile processing, we found a multisensory incongruency response, generated in frontal, cingulate and cerebellar regions, an auditory mismatch response generated mainly in the auditory cortex and a tactile mismatch response generated in frontal and cerebellar regions. The influence of musical training was seen in the audio-tactile as well as in the auditory condition, indicating enhanced higher-order processing in musicians, while the sources of the tactile MMN were not influenced by long-term musical training. Consistent with the predictive coding model, more basic, bottom-up sensory processing was relatively stable and less affected by expertise, whereas areas for top-down models of multisensory expectancies were modulated by training.     

The modification of cortical reorganization and chronic pain by sensory feedback

Recent neuroscientific evidence has revealed that the adult brain is capable of substantial plastic change in areas such as the primary somatosensory cortex that were formerly thought to be modifiable only during early experience. We discuss research on phantom limb pain as well as chronic back pain that revealed functional reorganization in both the somatosensory and the motor system in these chronic pain states. In phantom limb pain patients, cortical reorganization is correlated with the amount of phantom limb pain; in low back pain patients the amount of reorganizational change increases with chronicity. We present a model of the development of chronic pain that assumes an important role of somatosensory pain memories. In phantom limb pain, we propose that those patients who experienced intense pain prior to the amputation will later likely develop enhanced cortical reorganization and phantom limb pain. We show that cortical plasticity related to chronic pain can be reduced by behavioral interventions that provide feedback to the brain areas that were altered by somatosensory pain memories.     

Remembrance of odors past: human olfactory cortex in cross-modal recognition memory

Episodic memory is often imbued with multisensory richness, such that the recall of an event can be endowed with the sights, sounds, and smells of its prior occurrence. While hippocampus and related medial temporal structures are implicated in episodic memory retrieval, the participation of sensory-specific cortex in representing the qualities of an episode is less well established. We combined functional magnetic resonance imaging (fMRI) with a cross-modal paradigm, where objects were presented with odors during memory encoding. We then examined the effect of odor context on neural responses at retrieval when these same objects were presented alone. Primary olfactory (piriform) cortex, as well as anterior hippocampus, was activated during the successful retrieval of old (compared to new) objects. Our findings indicate that sensory features of the original engram are preserved in unimodal olfactory cortex. We suggest that reactivation of memory traces distributed across modality-specific brain areas underpins the sensory qualities of episodic memories.     

Multisensory integration in macaque visual cortex depends on cue reliability

Responses of multisensory neurons to combinations of sensory cues are generally enhanced or depressed relative to single cues presented alone, but the rules that govern these interactions have remained unclear. We examined integration of visual and vestibular self-motion cues in macaque area MSTd in response to unimodal as well as congruent and conflicting bimodal stimuli in order to evaluate hypothetical combination rules employed by multisensory neurons. Bimodal responses were well fit by weighted linear sums of unimodal responses, with weights typically less than one (subadditive). Surprisingly, our results indicate that weights change with the relative reliabilities of the two cues: visual weights decrease and vestibular weights increase when visual stimuli are degraded. Moreover, both modulation depth and neuronal discrimination thresholds improve for matched bimodal compared to unimodal stimuli, which might allow for increased neural sensitivity during multisensory stimulation. These findings establish important new constraints for neural models of cue integration.     

Multisensory integration in cortex: shedding light on prickly issues

Interactions between different sensory modalities can be observed in unimodal areas of the cortex, as revealed by recent neuroimaging studies. A new report by Macaluso and colleagues ( [this issue of Neuron]) shows that crossmodal effects of tactile stimulation in visual cortex critically depend on the spatial congruence of multisensory inputs. This work is discussed in relation to neural and computational models of multisensory integration.