Layer-specific fMRI reflects different neuronal computations at different depths in human V1

Recent work has established that cerebral blood flow is regulated at a spatial scale that can be resolved by high field fMRI to show cortical columns in humans. While cortical columns represent a cluster of neurons with similar response properties (spanning from the pial surface to the white matter), important information regarding neuronal interactions and computational processes is also contained within a single column, distributed across the six cortical lamina. A basic understanding of underlying neuronal circuitry or computations may be revealed through investigations of the distribution of neural responses at different cortical depths. In this study, we used T(2)-weighted imaging with 0.7 mm (isotropic) resolution to measure fMRI responses at different depths in the gray matter while human subjects observed images with either recognizable or scrambled (physically impossible) objects. Intact and scrambled images were partially occluded, resulting in clusters of activity distributed across primary visual cortex. A subset of the identified clusters of voxels showed a preference for scrambled objects over intact; in these clusters, the fMRI response in middle layers was stronger during the presentation of scrambled objects than during the presentation of intact objects. A second experiment, using stimuli targeted at either the magnocellular or the parvocellular visual pathway, shows that laminar profiles in response to parvocellular-targeted stimuli peak in more superficial layers. These findings provide new evidence for the differential sensitivity of high-field fMRI to modulations of the neural responses at different cortical depths.     

Attention controls multisensory perception via two distinct mechanisms at different levels of the cortical hierarchy

To form a percept of the multisensory world, the brain needs to integrate signals from common sources weighted by their reliabilities and segregate those from independent sources. Previously, we have shown that anterior parietal cortices combine sensory signals into representations that take into account the signals’ causal structure (i.e., common versus independent sources) and their sensory reliabilities as predicted by Bayesian causal inference. The current study asks to what extent and how attentional mechanisms can actively control how sensory signals are combined for perceptual inference. In a pre- and postcueing paradigm, we presented observers with audiovisual signals at variable spatial disparities. Observers were precued to attend to auditory or visual modalities prior to stimulus presentation and postcued to report their perceived auditory or visual location. Combining psychophysics, functional magnetic resonance imaging (fMRI), and Bayesian modelling, we demonstrate that the brain moulds multisensory inference via two distinct mechanisms. Prestimulus attention to vision enhances the reliability and influence of visual inputs on spatial representations in visual and posterior parietal cortices. Poststimulus report determines how parietal cortices flexibly combine sensory estimates into spatial representations consistent with Bayesian causal inference. Our results show that distinct neural mechanisms control how signals are combined for perceptual inference at different levels of the cortical hierarchy.

A combination of psychophysics, computational modelling and fMRI reveals novel insights into how the brain controls the binding of information across the senses, such as the voice and lip movements of a speaker.     

Unveiling functions of the visual cortex using task-specific deep neural networks

The human visual cortex enables visual perception through a cascade of hierarchical computations in cortical regions with distinct functionalities. Here, we introduce an AI-driven approach to discover the functional mapping of the visual cortex. We related human brain responses to scene images measured with functional MRI (fMRI) systematically to a diverse set of deep neural networks (DNNs) optimized to perform different scene perception tasks. We found a structured mapping between DNN tasks and brain regions along the ventral and dorsal visual streams. Low-level visual tasks mapped onto early brain regions, 3-dimensional scene perception tasks mapped onto the dorsal stream, and semantic tasks mapped onto the ventral stream. This mapping was of high fidelity, with more than 60% of the explainable variance in nine key regions being explained. Together, our results provide a novel functional mapping of the human visual cortex and demonstrate the power of the computational approach.     

Playing the electric light orchestra—how electrical stimulation of visual cortex elucidates the neural basis of perception

Vision research has the potential to reveal fundamental mechanisms underlying sensory experience. Causal experimental approaches, such as electrical microstimulation, provide a unique opportunity to test the direct contributions of visual cortical neurons to perception and behaviour. But in spite of their importance, causal methods constitute a minority of the experiments used to investigate the visual cortex to date. We reconsider the function and organization of visual cortex according to results obtained from stimulation techniques, with a special emphasis on electrical stimulation of small groups of cells in awake subjects who can report their visual experience. We compare findings from humans and monkeys, striate and extrastriate cortex, and superficial versus deep cortical layers, and identify a number of revealing gaps in the ‘causal map′ of visual cortex. Integrating results from different methods and species, we provide a critical overview of the ways in which causal approaches have been used to further our understanding of circuitry, plasticity and information integration in visual cortex. Electrical stimulation not only elucidates the contributions of different visual areas to perception, but also contributes to our understanding of neuronal mechanisms underlying memory, attention and decision-making.     

Long-range, pattern-dependent contextual effects in early human visual cortex

The standard view of neurons in early visual cortex is that they behave like localized feature detectors. Here we demonstrate that processing in early visual areas goes beyond feature detection by showing that neural responses are greater when a feature deviates from its context compared to when it does not deviate from its context. Using psychophysics, fMRI, and electroencephalography methodologies, we measured neural responses to an oriented Gabor ("target") embedded in various visual patterns as defined by the relative orientation of flanking stimuli. We first show using psychophysical contrast adaptation and fMRI that a target that differs from its context results in more neural activity compared to a target that is contained within an alternating sequence, suggesting that neurons in early visual cortex are sensitive to large-scale orientation patterns. Next, we use event-related potentials to show that orientation deviations affect the earliest sensory components of the target response. Finally, we use forced-choice classification of "noise" stimuli to show that we are more likely to "see" orientations that deviate from the context. Our results suggest that early visual cortex is sensitive to global patterns in images in a way that is markedly different from the predictions of standard models of cortical visual processing.     

Blinking suppresses the neural response to unchanging retinal stimulation

Blinks profoundly interrupt visual input but are rarely noticed, perhaps because of blink suppression, a visual-sensitivity loss that begins immediately prior to blink onset. Blink suppression is thought to result from an extra-retinal signal that is associated with the blink motor command and may act to attenuate the sensory consequences of the motor action. However, the neural mechanisms underlying this phenomenon remain unclear. They are challenging to study because any brain-activity changes resulting from an extra-retinal signal associated with the blink motor command are potentially masked by profound neural-activity changes caused by the retinal-illumination reduction that results from occlusion of the pupil by the eyelid. Here, we distinguished direct top-down effects of blink-associated motor signals on cortical activity from purely mechanical or optical effects of blinking on visual input by combining pupil-independent retinal stimulation with functional MRI (fMRI) in humans. Even though retinal illumination was kept constant during blinks, we found that blinking nevertheless suppressed activity in visual cortex and in areas of parietal and prefrontal cortex previously associated with awareness of environmental change. Our findings demonstrate active top-down modulation of visual processing during blinking, suggesting a possible mechanism by which blinks go unnoticed.     

Encoding of Natural Sounds at Multiple Spectral and Temporal Resolutions in the Human Auditory Cortex

Functional neuroimaging research provides detailed observations of the response patterns that natural sounds (e.g. human voices and speech, animal cries, environmental sounds) evoke in the human brain. The computational and representational mechanisms underlying these observations, however, remain largely unknown. Here we combine high spatial resolution (3 and 7 Tesla) functional magnetic resonance imaging (fMRI) with computational modeling to reveal how natural sounds are represented in the human brain. We compare competing models of sound representations and select the model that most accurately predicts fMRI response patterns to natural sounds. Our results show that the cortical encoding of natural sounds entails the formation of multiple representations of sound spectrograms with different degrees of spectral and temporal resolution. The cortex derives these multi-resolution representations through frequency-specific neural processing channels and through the combined analysis of the spectral and temporal modulations in the spectrogram. Furthermore, our findings suggest that a spectral-temporal resolution trade-off may govern the modulation tuning of neuronal populations throughout the auditory cortex. Specifically, our fMRI results suggest that neuronal populations in posterior/dorsal auditory regions preferably encode coarse spectral information with high temporal precision. Vice-versa, neuronal populations in anterior/ventral auditory regions preferably encode fine-grained spectral information with low temporal precision. We propose that such a multi-resolution analysis may be crucially relevant for flexible and behaviorally-relevant sound processing and may constitute one of the computational underpinnings of functional specialization in auditory cortex.     

Neural activities in v1 create a bottom-up saliency map

The bottom-up contribution to the allocation of exogenous attention is a saliency map, whose neural substrate is hard to identify because of possible contamination by top-down signals. We obviated this possibility using stimuli that observers could not perceive, but that nevertheless, through orientation contrast between foreground and background regions, attracted attention to improve a localized visual discrimination. When orientation contrast increased, so did the degree of attraction, and two physiological measures: the amplitude of the earliest (C1) component of the ERP, which is associated with primary visual cortex, and fMRI BOLD signals in areas V1-V4 (but not the intraparietal sulcus). Significantly, across observers, the degree of attraction correlated with the C1 amplitude and just the V1 BOLD signal. These findings strongly support the proposal that a bottom-up saliency map is created in V1, challenging the dominant view that the saliency map is generated in the parietal cortex.     

Less is more: expectation sharpens representations in the primary visual cortex

Prior expectations about the visual world facilitate perception by allowing us to quickly deduce plausible interpretations from noisy and ambiguous data. The neural mechanisms of this facilitation remain largely unclear. Here, we used functional magnetic resonance imaging (fMRI) and multivariate pattern analysis (MVPA) techniques to measure both the amplitude and representational content of neural activity in the early visual cortex of human volunteers. We find that while perceptual expectation reduces the neural response amplitude in the primary visual cortex (V1), it improves the stimulus representation in this area, as revealed by MVPA. This informational improvement was independent of attentional modulations by task relevance. Finally, the informational improvement in V1 correlated with subjects' behavioral improvement when the expected stimulus feature was relevant. These data suggest that expectation facilitates perception by sharpening sensory representations.     

Attentional enhancement via selection and pooling of early sensory responses in human visual cortex

The computational processes by which attention improves behavioral performance were characterized by measuring visual cortical activity with functional magnetic resonance imaging as humans performed a contrast-discrimination task with focal and distributed attention. Focal attention yielded robust improvements in behavioral performance accompanied by increases in cortical responses. Quantitative analysis revealed that if performance were limited only by the sensitivity of the measured sensory signals, the improvements in behavioral performance would have corresponded to an unrealistically large reduction in response variability. Instead, behavioral performance was well characterized by a pooling and selection process for which the largest sensory responses, those most strongly modulated by attention, dominated the perceptual decision. This characterization predicts that high-contrast distracters that evoke large responses should negatively impact behavioral performance. We tested and confirmed this prediction. We conclude that attention enhanced behavioral performance predominantly by enabling efficient selection of the behaviorally relevant sensory signals.     

Developmental Switch in Neurovascular Coupling in the Immature Rodent Barrel Cortex

Neurovascular coupling (NVC) in the adult central nervous system (CNS) is a mechanism that provides regions of the brain with more oxygen and glucose upon increased levels of neural activation. Hemodynamic changes that go along with neural activation evoke a blood oxygen level-dependent (BOLD) signal in functional magnetic resonance imaging (fMRI) that can be used to study brain activity non-invasively. A correct correlation of the BOLD signal to neural activity is pivotal to understand this signal in neuronal development, health and disease. However, the function of NVC during development is largely unknown. The rodent whisker-to-barrel cortex is an experimentally well established model to study neurovascular interdependences. Using extracellular multi-electrode recordings and laser-Doppler-flowmetry (LDF) we show in the murine barrel cortex of postnatal day 7 (P7) and P30 mice in vivo that NVC undergoes a physiological shift during the first month of life. In the mature CNS it is well accepted that cortical sensory processing results in a rise in regional cerebral blood flow (rCBF). We show in P7 animals that rCBF decreases during prolonged multi-whisker stimulation and goes along with multi unit activity (MUA) fatigue. In contrast at P30, MUA remains stable during repetitive stimulation and is associated with an increase in rCBF. Further we characterize in both age groups the responses in NVC to single sensory stimuli. We suggest that the observed shift in NVC is an important process in cortical development that may be of high relevance for the correct interpretation of brain activity e.g. in fMRI studies of the immature central nervous system (CNS).     

A Functional Hypothesis for LGN-V1-TRN Connectivities Suggested by Computer Simulation

We employ computer simulation to investigate the function of neural circuitries between thalamic sensory relay nuclei, primary sensory cortices, and the thalamic reticular nucleus (TRN). Computational similarities exist between these circuits and the architecture of a simple artificial neural network. We impose processing parameters on this network architecture in keeping with anatomical and physiological details of the mammalian geniculo-cortical visual pathway, and then run the simulation on a task involving multiple simultaneous inputs from the simulated visual field. After two to three loops through the simulation, activity in cortical and thalamic units whose receptive fields include the stronger stimulus remains constant, while activity in other cortical and thalamic units activated by weaker stimuli declines toward resting values. These results suggest that the modeled neural circuitry functions to prime selective attentional mechanisms further up the visual streams toward specific portions of the total visual stimulus. Besides extending existing models and evidence about the function of these neural circuits, our results also provide physiologists with predicted activity profiles of thalamic and cortical elements of the modeled neural system for a task not yet studied experimentally.     

Modulation of activity in human visual area V1 during memory masking

Neurons in the primary visual cortex, V1, are specialized for the processing of elemental features of the visual stimulus, such as orientation and spatial frequency. Recent fMRI evidence suggest that V1 neurons are also recruited in visual perceptual memory; a number of studies using multi-voxel pattern analysis have successfully decoded stimulus-specific information from V1 activity patterns during the delay phase in memory tasks. However, consistent fMRI signal modulations reflecting the memory process have not yet been demonstrated. Here, we report evidence, from three subjects, that the low V1 BOLD activity during retention of low-level visual features is caused by competing interactions between neural populations coding for different values along the spectrum of the dimension remembered. We applied a memory masking paradigm in which the memory representation of a masker stimulus interferes with a delayed spatial frequency discrimination task when its frequency differs from the discriminanda with ±1 octave and found that impaired behavioral performance due to masking is reflected in weaker V1 BOLD signals. This cross-channel inhibition in V1 only occurs with retinotopic overlap between the masker and the sample stimulus of the discrimination task. The results suggest that memory for spatial frequency is a local process in the retinotopically organized visual cortex.     

Negative BOLD fMRI response in the visual cortex carries precise stimulus-specific information

Sustained positive BOLD (blood oxygen level-dependent) activity is employed extensively in functional magnetic resonance imaging (fMRI) studies as evidence for task or stimulus-specific neural responses. However, the presence of sustained negative BOLD activity (i.e., sustained responses that are lower than the fixation baseline) has remained more difficult to interpret. Some studies suggest that it results from local "blood stealing" wherein blood is diverted to neurally active regions without a concomitant change of neural activity in the negative BOLD regions. However, other evidence suggests that negative BOLD is a result of local neural suppression. In both cases, regions of negative BOLD response are usually interpreted as carrying relatively little, if any, stimulus-specific information (hence the predominant reliance on positive BOLD activity in fMRI). Here we show that the negative BOLD response resulting from visual stimulation can carry high information content that is stimulus-specific. Using a general linear model (GLM), we contrasted standard flickering stimuli to a fixation baseline and found regions of the visual cortex that displayed a sustained negative BOLD response, consistent with several previous studies. Within these negative BOLD regions, we compared patterns of fMRI activity generated by flickering Gabors that were systematically shifted in position. As the Gabors were shifted further from each other, the correlation in the spatial pattern of activity across a population of voxels (such as the population of V1 voxels that displayed a negative BOLD response) decreased significantly. Despite the fact that the BOLD signal was significantly negative (lower than fixation baseline), these regions were able to discriminate objects separated by less than 0.5 deg (at approximately 10 deg eccentricity). The results suggest that meaningful, stimulus-specific processing occurs even in regions that display a strong negative BOLD response.     

Distribution of neuropeptide Y immunoreactivity in human visual cortex and underlying white matter

Immunocytochemical techniques have been used to study neuropeptide Y (NPY) distribution in the human visual cortex (Brodman's areas 17, 18 and 19) NYP cell bodies belong mostly to inhibitory (multipolar and bitufted) but also to excitatory (bipolar and some pyramidal) neuronal types. Their distribution is similar in the three cortical areas studied: 20 to 40% of the NPY perikarya are located in the cortical gray matter, mostly in the deep layers, while the remaining 60 to 80% are located in the underlying white matter. Immunoreactive NPY processes form a rich network of intersecting fibers throughout the entire visual cortex. A superficial plexus (layers I and II) and a deep plexus (deep layer V and layer VI) of NPY fibers are present in areas 17, 18 and 19. In area 17, an additional well developed plexus is present in layers IVb and IVc. These plexuses receive branches from long parallel fibers arising from deep cortical layers or underlying white matter and terminating in superficial layers. Local or extrinsic NPY terminals wind around vessels in the cortex as well as in the white matter, and either penetrate them or form clusters of club endings on their walls. Our results suggest a role for NPY in human visual circuitry and in cortical blood flow regulation.     

Remapping for visual stability

Visual perception is based on both incoming sensory signals and information about ongoing actions. Recordings from single neurons have shown that corollary discharge signals can influence visual representations in parietal, frontal and extrastriate visual cortex, as well as the superior colliculus (SC). In each of these areas, visual representations are remapped in conjunction with eye movements. Remapping provides a mechanism for creating a stable, eye-centred map of salient locations. Temporal and spatial aspects of remapping are highly variable from cell to cell and area to area. Most neurons in the lateral intraparietal area remap stimulus traces, as do many neurons in closely allied areas such as the frontal eye fields the SC and extrastriate area V3A. Remapping is not purely a cortical phenomenon. Stimulus traces are remapped from one hemifield to the other even when direct cortico-cortical connections are removed. The neural circuitry that produces remapping is distinguished by significant plasticity, suggesting that updating of salient stimuli is fundamental for spatial stability and visuospatial behaviour. These findings provide new evidence that a unified and stable representation of visual space is constructed by redundant circuitry, comprising cortical and subcortical pathways, with a remarkable capacity for reorganization.     

On the perception of probable things: neural substrates of associative memory, imagery, and perception

Perception is influenced both by the immediate pattern of sensory inputs and by memories acquired through prior experiences with the world. Throughout much of its illustrious history, however, study of the cellular basis of perception has focused on neuronal structures and events that underlie the detection and discrimination of sensory stimuli. Relatively little attention has been paid to the means by which memories interact with incoming sensory signals. Building upon recent neurophysiological/behavioral studies of the cortical substrates of visual associative memory, I propose a specific functional process by which stored information about the world supplements sensory inputs to yield neuronal signals that can account for visual perceptual experience. This perspective represents a significant shift in the way we think about the cellular bases of perception.     

Integrating information from different senses in the auditory cortex

Multisensory integration was once thought to be the domain of brain areas high in the cortical hierarchy, with early sensory cortical fields devoted to unisensory processing of inputs from their given set of sensory receptors. More recently, a wealth of evidence documenting visual and somatosensory responses in auditory cortex, even as early as the primary fields, has changed this view of cortical processing. These multisensory inputs may serve to enhance responses to sounds that are accompanied by other sensory cues, effectively making them easier to hear, but may also act more selectively to shape the receptive field properties of auditory cortical neurons to the location or identity of these events. We discuss the new, converging evidence that multiplexing of neural signals may play a key role in informatively encoding and integrating signals in auditory cortex across multiple sensory modalities. We highlight some of the many open research questions that exist about the neural mechanisms that give rise to multisensory integration in auditory cortex, which should be addressed in future experimental and theoretical studies.     

The Y cell visual pathway implements a demodulating nonlinearity

Neural encoding of sensory signals involves both linear and nonlinear processes. Determining which nonlinear operations are implemented by neural systems is crucial to understanding sensory processing. Here, we ask if demodulation, the process used to decode AM radio signals, describes how Y cells in the cat LGN nonlinearly encode the visual scene. In response to visual AM signals across?a wide range of carrier frequencies, Y cells were found to transmit a demodulated signal, with the firing rate of single-units fluctuating at the envelope frequency but not the carrier frequency. A comparison of temporal frequency tuning properties between LGN Y cells and neurons in two primary cortical areas suggests that Y cells initiate a distinct pathway that carries a demodulated representation of the visual scene to cortex. The nonlinear signal processing carried out by the Y cell pathway simplifies the neural representation of complex visual features and allows high spatiotemporal frequencies to drive cortical responses.     

Spike suppression in a local cortical circuit induced by transcranial magnetic stimulation

Transcranial magnetic stimulation (TMS) noninvasively interferes with human cortical function, and is widely used as an effective technique for probing causal links between neural activity and cognitive function. However, the physiological mechanisms underlying TMS-induced effects on neural activity remain unclear. We examined the mechanism by which TMS disrupts neural activity in a local circuit in early visual cortex using a computational model consisting of conductance-based spiking neurons with excitatory and inhibitory synaptic connections. We found that single-pulse TMS suppressed spiking activity in a local circuit model, disrupting the population response. Spike suppression was observed when TMS was applied to the local circuit within a limited time window after the local circuit received sensory afferent input, as observed in experiments investigating suppression of visual perception with TMS targeting early visual cortex. Quantitative analyses revealed that the magnitude of suppression was significantly larger for synaptically-connected neurons than for isolated individual neurons, suggesting that intracortical inhibitory synaptic coupling also plays an important role in TMS-induced suppression. A conventional local circuit model of early visual cortex explained only the early period of visual suppression observed in experiments. However, models either involving strong recurrent excitatory synaptic connections or sustained excitatory input were able to reproduce the late period of visual suppression. These results suggest that TMS targeting early visual cortex disrupts functionally distinct neural signals, possibly corresponding to feedforward and recurrent information processing, by imposing inhibitory effects through intracortical inhibitory synaptic connections.