Modulation of activity in human visual area V1 during memory masking

Neurons in the primary visual cortex, V1, are specialized for the processing of elemental features of the visual stimulus, such as orientation and spatial frequency. Recent fMRI evidence suggest that V1 neurons are also recruited in visual perceptual memory; a number of studies using multi-voxel pattern analysis have successfully decoded stimulus-specific information from V1 activity patterns during the delay phase in memory tasks. However, consistent fMRI signal modulations reflecting the memory process have not yet been demonstrated. Here, we report evidence, from three subjects, that the low V1 BOLD activity during retention of low-level visual features is caused by competing interactions between neural populations coding for different values along the spectrum of the dimension remembered. We applied a memory masking paradigm in which the memory representation of a masker stimulus interferes with a delayed spatial frequency discrimination task when its frequency differs from the discriminanda with ±1 octave and found that impaired behavioral performance due to masking is reflected in weaker V1 BOLD signals. This cross-channel inhibition in V1 only occurs with retinotopic overlap between the masker and the sample stimulus of the discrimination task. The results suggest that memory for spatial frequency is a local process in the retinotopically organized visual cortex.     

Different Roles of COMT and HTR2A Genotypes in Working Memory Subprocesses

Working memory is linked to the functions of the frontal areas, in which neural activity is mediated by dopaminergic and serotonergic tones. However, there is no consensus regarding how the dopaminergic and serotonergic systems influence working memory subprocesses. The present study used an imaging genetics approach to examine the interaction between neurochemical functions and working memory performance. We focused on functional polymorphisms of the catechol-O-methyltransferase (COMT) Val¹⁵⁸Met and serotonin 2A receptor (HTR2A) -1438G/A genes, and devised a delayed recognition task to isolate the encoding, retention, and retrieval processes for visual information. The COMT genotypes affected recognition accuracy, whereas the HTR2A genotypes were associated with recognition response times. Activations specifically related to working memory were found in the right frontal and parietal areas, such as the middle frontal gyrus (MFG), inferior frontal gyrus (IFG), anterior cingulate cortex (ACC), and inferior parietal lobule (IPL). MFG and ACC/IPL activations were sensitive to differences between the COMT genotypes and between the HTR2A genotypes, respectively. Structural equation modeling demonstrated that stronger connectivity in the ACC-MFG and ACC-IFG networks is related to better task performance. The behavioral and fMRI results suggest that the dopaminergic and serotonergic systems play different roles in the working memory subprocesses and modulate closer cooperation between lateral and medial frontal activations.     

Cognitive Control in Auditory Working Memory Is Enhanced in Musicians

Musical competence may confer cognitive advantages that extend beyond processing of familiar musical sounds. Behavioural evidence indicates a general enhancement of both working memory and attention in musicians. It is possible that musicians, due to their training, are better able to maintain focus on task-relevant stimuli, a skill which is crucial to working memory. We measured the blood oxygenation-level dependent (BOLD) activation signal in musicians and non-musicians during working memory of musical sounds to determine the relation among performance, musical competence and generally enhanced cognition. All participants easily distinguished the stimuli. We tested the hypothesis that musicians nonetheless would perform better, and that differential brain activity would mainly be present in cortical areas involved in cognitive control such as the lateral prefrontal cortex. The musicians performed better as reflected in reaction times and error rates. Musicians also had larger BOLD responses than non-musicians in neuronal networks that sustain attention and cognitive control, including regions of the lateral prefrontal cortex, lateral parietal cortex, insula, and putamen in the right hemisphere, and bilaterally in the posterior dorsal prefrontal cortex and anterior cingulate gyrus. The relationship between the task performance and the magnitude of the BOLD response was more positive in musicians than in non-musicians, particularly during the most difficult working memory task. The results confirm previous findings that neural activity increases during enhanced working memory performance. The results also suggest that superior working memory task performance in musicians rely on an enhanced ability to exert sustained cognitive control. This cognitive benefit in musicians may be a consequence of focused musical training.     

The neural basis of centre-surround interactions in visual motion processing

Perception of a moving visual stimulus can be suppressed or enhanced by surrounding context in adjacent parts of the visual field. We studied the neural processes underlying such contextual modulation with fMRI. We selected motion selective regions of interest (ROI) in the occipital and parietal lobes with sufficiently well defined topography to preclude direct activation by the surround. BOLD signal in the ROIs was suppressed when surround motion direction matched central stimulus direction, and increased when it was opposite. With the exception of hMT+/V5, inserting a gap between the stimulus and the surround abolished surround modulation. This dissociation between hMT+/V5 and other motion selective regions prompted us to ask whether motion perception is closely linked to processing in hMT+/V5, or reflects the net activity across all motion selective cortex. The motion aftereffect (MAE) provided a measure of motion perception, and the same stimulus configurations that were used in the fMRI experiments served as adapters. Using a linear model, we found that the MAE was predicted more accurately by the BOLD signal in hMT+/V5 than it was by the BOLD signal in other motion selective regions. However, a substantial improvement in prediction accuracy could be achieved by using the net activity across all motion selective cortex as a predictor, suggesting the overall conclusion that visual motion perception depends upon the integration of activity across different areas of visual cortex.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.     

How Attention Can Create Synaptic Tags for the Learning of Working Memories in Sequential Tasks

Intelligence is our ability to learn appropriate responses to new stimuli and situations. Neurons in association cortex are thought to be essential for this ability. During learning these neurons become tuned to relevant features and start to represent them with persistent activity during memory delays. This learning process is not well understood. Here we develop a biologically plausible learning scheme that explains how trial-and-error learning induces neuronal selectivity and working memory representations for task-relevant information. We propose that the response selection stage sends attentional feedback signals to earlier processing levels, forming synaptic tags at those connections responsible for the stimulus-response mapping. Globally released neuromodulators then interact with tagged synapses to determine their plasticity. The resulting learning rule endows neural networks with the capacity to create new working memory representations of task relevant information as persistent activity. It is remarkably generic: it explains how association neurons learn to store task-relevant information for linear as well as non-linear stimulus-response mappings, how they become tuned to category boundaries or analog variables, depending on the task demands, and how they learn to integrate probabilistic evidence for perceptual decisions.     

Cortical Resonance Frequencies Emerge from Network Size and Connectivity

Neural oscillations occur within a wide frequency range with different brain regions exhibiting resonance-like characteristics at specific points in the spectrum. At the microscopic scale, single neurons possess intrinsic oscillatory properties, such that is not yet known whether cortical resonance is consequential to neural oscillations or an emergent property of the networks that interconnect them. Using a network model of loosely-coupled Wilson-Cowan oscillators to simulate a patch of cortical sheet, we demonstrate that the size of the activated network is inversely related to its resonance frequency. Further analysis of the parameter space indicated that the number of excitatory and inhibitory connections, as well as the average transmission delay between units, determined the resonance frequency. The model predicted that if an activated network within the visual cortex increased in size, the resonance frequency of the network would decrease. We tested this prediction experimentally using the steady-state visual evoked potential where we stimulated the visual cortex with different size stimuli at a range of driving frequencies. We demonstrate that the frequency corresponding to peak steady-state response inversely correlated with the size of the network. We conclude that although individual neurons possess resonance properties, oscillatory activity at the macroscopic level is strongly influenced by network interactions, and that the steady-state response can be used to investigate functional networks.     

Different dynamics of performance and brain activation in the time course of perceptual learning

Perceptual learning is regarded as a manifestation of experience-dependent plasticity in the sensory systems, yet the underlying neural mechanisms remain unclear. We measured the dynamics of performance on a visual task and brain activation in the human primary visual cortex (V1) across the time course of perceptual learning. Within the first few weeks of training, brain activation in a V1 subregion corresponding to the trained visual field quadrant and task performance both increased. However, while performance levels then saturated and were maintained at a constant level, brain activation in the corresponding areas decreased to the level observed before training. These findings indicate that there are distinct temporal phases in the time course of perceptual learning, related to differential dynamics of BOLD activity in visual cortex.     

Backward signal from medial temporal lobe in neural circuit reorganization of primate inferotemporal cortex

Neuropsychological theories proposed a critical role of the interaction between the medial temporal lobe and neocortex in the formation of long-term memory for facts and events, which has often been tested by learning of a series of paired words or figures in humans. We identify neural mechanisms of this long-term memory formation process by single-unit recording and molecular biological methods in an animal model of visual pair-association task in monkeys. In our previous studies, we found a group of neurons that manifested selective responses to both of the paired associates (pair-coding neuron) in the anterior inferior temporal (IT) cortex. It provides strong evidence that single IT neurons acquire the response-selectivity through associative learning, and suggests that the reorganized neural circuits for the pair-coding neurons serve as the memory engram of the pair-association learning. In this article, we investigated further mechanisms of the neural circuit reorganization. First, we tested the role of the backward connections from the medial temporal lobe to IT cortex. lbotenic acid was injected unilaterally into the entorhinal and perirhinal cortex which provided massive backward projections ipsilaterally to IT cortex. We found that the limbic lesion disrupted the associative code of the IT neurons between the paired associates, without impairing the visual response to each stimulus. Second, we ask why the limbic-neocortical interactions are so important. We hypothesize that limbic neurons would undergo rapid modification of synaptic connectivity and provide backward signals that guide reorganization of neocortical neural circuits. We then investigated the molecular basis of such rapid synaptic modifiability by detecting the expression of immediate-early genes. We found strong expression of zif268 during the learning of a new set of paired associates, most intensively in area 36 of the perirhinal cortex. All these results with visual pair-association task support our hypothesis, and demonstrate that the ‘consolidation’ process, which was first proposed on the basis of clinico-psychological evidence, can now be examined in the primate with neurophysiolocical and molecular biological approaches.     

A model for integrating elementary neural functions into delayed-response behavior

It is well established that various cortical regions can implement a wide array of neural processes, yet the mechanisms which integrate these processes into behavior-producing, brain-scale activity remain elusive. We propose that an important role in this respect might be played by executive structures controlling the traffic of information between the cortical regions involved. To illustrate this hypothesis, we present a neural network model comprising a set of interconnected structures harboring stimulus-related activity (visual representation, working memory, and planning), and a group of executive units with task-related activity patterns that manage the information flowing between them. The resulting dynamics allows the network to perform the dual task of either retaining an image during a delay (delayed-matching to sample task), or recalling from this image another one that has been associated with it during training (delayed-pair association task). The model reproduces behavioral and electrophysiological data gathered on the inferior temporal and prefrontal cortices of primates performing these same tasks. It also makes predictions on how neural activity coding for the recall of the image associated with the sample emerges and becomes prospective during the training phase. The network dynamics proves to be very stable against perturbations, and it exhibits signs of scale-invariant organization and cooperativity. The present network represents a possible neural implementation for active, top-down, prospective memory retrieval in primates. The model suggests that brain activity leading to performance of cognitive tasks might be organized in modular fashion, simple neural functions becoming integrated into more complex behavior by executive structures harbored in prefrontal cortex and/or basal ganglia.     

Basic mechanisms for graded persistent activity: discrete attractors,continuous attractors,and dynamic representations

Persistent neural activity is observed in many systems, and is thought to be a neural substrate for holding memories over time delays of a few seconds. Recent work has addressed two issues. First, how can networks of neurons robustly hold such an active memory? Computer systems obtain significant robustness to noise by approximating analogue quantities with discrete digital representations. In a similar manner, theoretical models of persistent activity in spiking neurons have shown that the most robust and stable way to store the short-term memory of a continuous parameter is to approximate it with a discrete representation. This general idea applies very broadly to mechanisms that range from biochemical networks to single cells and to large circuits of neurons. Second, why is it commonly observed that persistent activity in the cortex can be strongly time-varying? This observation is almost ubiquitous, and therefore must be taken into account in our models and our understanding of how short-term memories are held in the cortex.     

Complex Visual Search in Children and Adolescents: Effects of Age and Performance on fMRI Activation

Complex visuospatial processing relies on distributed neural networks involving occipital, parietal and frontal brain regions. Effects of physiological maturation (during normal brain development) and proficiency on tasks requiring complex visuospatial processing have not yet been studied extensively, as they are almost invariably interrelated. We therefore aimed at dissociating the effects of age and performance on functional MRI (fMRI) activation in a complex visual search task. In our cross-sectional study, healthy children and adolescents (n = 43, 19 females, 7-17 years) performed a complex visual search task during fMRI. Resulting activation was analysed with regard to the differential effects of age and performance. Our results are compatible with an increase in the neural network''s efficacy with age: within occipital and parietal cortex, the core regions of the visual exploration network, activation increased with age, and more so in the right than in the left hemisphere. Further, activation outside the visual search network decreased with age, mainly in left inferior frontal, middle temporal, and inferior parietal cortex. High-performers had stronger activation in right superior parietal cortex, suggesting a more mature visual search network. We could not see effects of age or performance in frontal cortex. Our results show that effects of physiological maturation and effects of performance, while usually intertwined, can be successfully disentangled and investigated using fMRI in children and adolescents.     

Stronger Neural Modulation by Visual Motion Intensity in Autism Spectrum Disorders

Theories of autism spectrum disorders (ASD) have focused on altered perceptual integration of sensory features as a possible core deficit. Yet, there is little understanding of the neuronal processing of elementary sensory features in ASD. For typically developed individuals, we previously established a direct link between frequency-specific neural activity and the intensity of a specific sensory feature: Gamma-band activity in the visual cortex increased approximately linearly with the strength of visual motion. Using magnetoencephalography (MEG), we investigated whether in individuals with ASD neural activity reflect the coherence, and thus intensity, of visual motion in a similar fashion. Thirteen adult participants with ASD and 14 control participants performed a motion direction discrimination task with increasing levels of motion coherence. A polynomial regression analysis revealed that gamma-band power increased significantly stronger with motion coherence in ASD compared to controls, suggesting excessive visual activation with increasing stimulus intensity originating from motion-responsive visual areas V3, V6 and hMT/V5. Enhanced neural responses with increasing stimulus intensity suggest an enhanced response gain in ASD. Response gain is controlled by excitatory-inhibitory interactions, which also drive high-frequency oscillations in the gamma-band. Thus, our data suggest that a disturbed excitatory-inhibitory balance underlies enhanced neural responses to coherent motion in ASD.     

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition performance even in brief presentations, and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations, such as the amount of noise, the number of neural recording sites, and the number of trials, and computational limitations, such as the complexity of the decoding classifier and the number of classifier training examples. In this work, we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of “kernel analysis” that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT, and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.     

The functional neuroanatomy of implicit-motion perception or representational momentum

BACKGROUND: When we view static scenes that imply motion - such as an object dropping off a shelf - recognition memory for the position of the object is extrapolated forward. It is as if the object in our mind's eye comes alive and continues on its course. This phenomenon is known as representational momentum and results in a distortion of recognition memory in the implied direction of motion. Representational momentum is modifiable; simply labelling a drawing of a pointed object as 'rocket' will facilitate the effect, whereas the label 'steeple' will impede it. We used functional magnetic resonance imaging (fMRI) to explore the neural substrate for representational momentum. RESULTS: Subjects participated in two experiments. In the first, they were presented with video excerpts of objects in motion (versus the same objects in a resting position). This identified brain areas responsible for motion perception. In the second experiment, they were presented with still photographs of the same target items, only some of which implied motion (representational momentum stimuli). When viewing still photographs of scenes implying motion, activity was revealed in secondary visual cortical regions that overlap with areas responsible for the perception of actual motion. Additional bilateral activity was revealed within a posterior satellite of V5 for the representational momentum stimuli. Activation was also engendered in the anterior cingulate cortex. CONCLUSIONS: Considering the implicit nature of representational momentum and its modifiability, the findings suggest that higher-order semantic information can act on secondary visual cortex to alter perception without explicit awareness.     

A neural computation for visual acuity in the presence of eye movements

Humans can distinguish visual stimuli that differ by features the size of only a few photoreceptors. This is possible despite the incessant image motion due to fixational eye movements, which can be many times larger than the features to be distinguished. To perform well, the brain must identify the retinal firing patterns induced by the stimulus while discounting similar patterns caused by spontaneous retinal activity. This is a challenge since the trajectory of the eye movements, and consequently, the stimulus position, are unknown. We derive a decision rule for using retinal spike trains to discriminate between two stimuli, given that their retinal image moves with an unknown random walk trajectory. This algorithm dynamically estimates the probability of the stimulus at different retinal locations, and uses this to modulate the influence of retinal spikes acquired later. Applied to a simple orientation-discrimination task, the algorithm performance is consistent with human acuity, whereas naive strategies that neglect eye movements perform much worse. We then show how a simple, biologically plausible neural network could implement this algorithm using a local, activity-dependent gain and lateral interactions approximately matched to the statistics of eye movements. Finally, we discuss evidence that such a network could be operating in the primary visual cortex.     

Working Memory Cells' Behavior May Be Explained by Cross-Regional Networks with Synaptic Facilitation

Neurons in the cortex exhibit a number of patterns that correlate with working memory. Specifically, averaged across trials of working memory tasks, neurons exhibit different firing rate patterns during the delay of those tasks. These patterns include: 1) persistent fixed-frequency elevated rates above baseline, 2) elevated rates that decay throughout the tasks memory period, 3) rates that accelerate throughout the delay, and 4) patterns of inhibited firing (below baseline) analogous to each of the preceding excitatory patterns. Persistent elevated rate patterns are believed to be the neural correlate of working memory retention and preparation for execution of behavioral/motor responses as required in working memory tasks. Models have proposed that such activity corresponds to stable attractors in cortical neural networks with fixed synaptic weights. However, the variability in patterned behavior and the firing statistics of real neurons across the entire range of those behaviors across and within trials of working memory tasks are typical not reproduced. Here we examine the effect of dynamic synapses and network architectures with multiple cortical areas on the states and dynamics of working memory networks. The analysis indicates that the multiple pattern types exhibited by cells in working memory networks are inherent in networks with dynamic synapses, and that the variability and firing statistics in such networks with distributed architectures agree with that observed in the cortex.     

Cholinergic enhancement of visual attention and neural oscillations in the human brain

Cognitive processes such as visual perception and selective attention induce specific patterns of brain oscillations. The neurochemical bases of these spectral changes in neural activity are largely unknown, but neuromodulators are thought to regulate processing. The cholinergic system is linked to attentional function in vivo, whereas separate in vitro studies show that cholinergic agonists induce high-frequency oscillations in slice preparations. This has led to theoretical proposals that cholinergic enhancement of visual attention might operate via gamma oscillations in visual cortex, although low-frequency alpha/beta modulation may also play a key role. Here we used MEG to record cortical oscillations in the context of administration of a cholinergic agonist (physostigmine) during a spatial visual attention task in humans. This cholinergic agonist enhanced spatial attention effects on low-frequency alpha/beta oscillations in visual cortex, an effect correlating with a drug-induced speeding of performance. By contrast, the cholinergic agonist did not alter high-frequency gamma oscillations in visual cortex. Thus, our findings show that cholinergic neuromodulation enhances attentional selection via an impact on oscillatory synchrony in visual cortex, for low rather than high frequencies. We discuss this dissociation between high- and low-frequency oscillations in relation to proposals that lower-frequency oscillations are generated by feedback pathways within visual cortex.     

A Laterally Interconnected Neural Architecture in MST Accounts for Psychophysical Discrimination of Complex Motion Patterns

The complex patterns of visual motion formed across the retina during self-motion, often referred to as optic flow, provide a rich source of information describing our dynamic relationship within the environment. Psychophysical studies indicate the existence of specialized detectors for component motion patterns (radial, circular, planar) that are consistent with the visual motion properties of cells in the medial superior temporal area (MST) of nonhuman primates. Here we use computational modeling and psychophysics to investigate the structural and functional role of these specialized detectors in performing a graded motion pattern (GMP) discrimination task. In the psychophysical task perceptual discrimination varied significantly with the type of motion pattern presented, suggesting perceptual correlates to the preferred motion bias reported in MST. Simulated perceptual discrimination in a population of independent MST-like neural responses showed inconsistent psychophysical performance that varied as a function of the visual motion properties within the population code. Robust psychophysical performance was achieved by fully interconnecting neural populations such that they inhibited nonpreferred units. Taken together, these results suggest that robust processing of the complex motion patterns associated with self-motion and optic flow may be mediated by an inhibitory structure of neural interactions in MST.