Paying attention to neurons with discriminating taste

Traditional theories of attention rely on the idea that when we search for a target in a visual display the brain boosts the activity of neurons optimally tuned for the target features. In this issue of Neuron, Navalpakkam and Itti take a computational approach to show that this strategy is actually very inefficient when the target is surrounded by distractors with similar features. Instead, the optimal strategy is to boost the activity of neurons that best discriminate between target and distractors, while essentially ignoring the neurons that respond best to the target.     

Very Slow Search and Reach: Failure to Maximize Expected Gain in an Eye-Hand Coordination Task

We examined an eye-hand coordination task where optimal visual search and hand movement strategies were inter-related. Observers were asked to find and touch a target among five distractors on a touch screen. Their reward for touching the target was reduced by an amount proportional to how long they took to locate and reach to it. Coordinating the eye and the hand appropriately would markedly reduce the search-reach time. Using statistical decision theory we derived the sequence of interrelated eye and hand movements that would maximize expected gain and we predicted how hand movements should change as the eye gathered further information about target location. We recorded human observers'' eye movements and hand movements and compared them with the optimal strategy that would have maximized expected gain. We found that most observers failed to adopt the optimal search-reach strategy. We analyze and describe the strategies they did adopt.     

How the value of the environment controls persistence in visual search

Classic foraging theory predicts that humans and animals aim to gain maximum reward per unit time. However, in standard instrumental conditioning tasks individuals adopt an apparently suboptimal strategy: they respond slowly when the expected value is low. This reward-related bias is often explained as reduced motivation in response to low rewards. Here we present evidence this behavior is associated with a complementary increased motivation to search the environment for alternatives. We trained monkeys to search for reward-related visual targets in environments with different values. We found that the reward-related bias scaled with environment value, was consistent with persistent searching after the target was already found, and was associated with increased exploratory gaze to objects in the environment. A novel computational model of foraging suggests that this search strategy could be adaptive in naturalistic settings where both environments and the objects within them provide partial information about hidden, uncertain rewards.     

Attention to stimulus features shifts spectral tuning of V4 neurons during natural vision

Previous neurophysiological studies suggest that attention can alter the baseline or gain of neurons in extrastriate visual areas but that it cannot change tuning. This suggests that neurons in visual cortex function as labeled lines whose meaning does not depend on task demands. To test this common assumption, we used a system identification approach to measure spatial frequency and orientation tuning in area V4 during two attentionally demanding visual search tasks, one that required fixation and one that allowed free viewing during search. We found that spatial attention modulates response baseline and gain but does not alter tuning, consistent with previous reports. In contrast, feature-based attention often shifts neuronal tuning. These tuning shifts are inconsistent with the labeled-line model and tend to enhance responses to stimulus features that distinguish the search target. Our data suggest that V4 neurons behave as matched filters that are dynamically tuned to optimize visual search.     

Cortical Surround Interactions and Perceptual Salience via Natural Scene Statistics

Spatial context in images induces perceptual phenomena associated with salience and modulates the responses of neurons in primary visual cortex (V1). However, the computational and ecological principles underlying contextual effects are incompletely understood. We introduce a model of natural images that includes grouping and segmentation of neighboring features based on their joint statistics, and we interpret the firing rates of V1 neurons as performing optimal recognition in this model. We show that this leads to a substantial generalization of divisive normalization, a computation that is ubiquitous in many neural areas and systems. A main novelty in our model is that the influence of the context on a target stimulus is determined by their degree of statistical dependence. We optimized the parameters of the model on natural image patches, and then simulated neural and perceptual responses on stimuli used in classical experiments. The model reproduces some rich and complex response patterns observed in V1, such as the contrast dependence, orientation tuning and spatial asymmetry of surround suppression, while also allowing for surround facilitation under conditions of weak stimulation. It also mimics the perceptual salience produced by simple displays, and leads to readily testable predictions. Our results provide a principled account of orientation-based contextual modulation in early vision and its sensitivity to the homogeneity and spatial arrangement of inputs, and lends statistical support to the theory that V1 computes visual salience.     

How do we select perceptions and actions? Human brain imaging studies.

The selective nature of human perception and action implies a modulatory interaction between sensorimotor processes and attentional processes. This paper explores the use of functional imaging in humans to explore the mechanisms of perceptual selection and the fate of irrelevant stimuli that are not selected. Experiments with positron emission tomography show that two qualitatively different patterns of modulation of cerebral blood flow can be observed in experiments where non-spatial visual attention and auditory attention are manipulated. These patterns of modulation of cerebral blood flow modulation can be described as gain control and bias signal mechanisms. In visual and auditory cortex, the dominant change in cerebral blood flow associated with attention to either modality is related to a bias signal. The relation of these patterns of modulation to attentional effects that have been observed in single neurons is discussed. The existence of mechanisms for selective perception raises the more general question of whether irrelevant ignored stimuli are nevertheless perceived. Lavie''s theory of attention proposes that the degree to which ignored stimuli are processed varies depending on the perceptual load of the current task. Evidence from behavioural and functional magnetic resonance imaging studies of ignored visual motion processing is presented in support of this proposal.     

Feature-based attention increases the selectivity of population responses in primate visual cortex

BACKGROUND: Attending to the spatial location or to nonspatial features of visual stimuli can modulate neuronal responses in primate visual cortex. The modulation by spatial attention changes the gain of sensory neurons and strengthens the representation of attended locations without changing neuronal selectivities such as directionality, i.e., the ratio of responses to preferred and anti-preferred directions of motion. Whether feature-based attention acts in a similar manner is unknown. RESULTS: To clarify this issue, we recorded the responses of 135 direction-selective neurons in the middle temporal area (MT) of two macaques to an unattended moving random dot pattern (the distractor) positioned inside a neuron's receptive field while the animals attended to a second moving pattern positioned in the opposite hemifield. Responses to different directions of the distractor were modulated by the same factor (approximately 12%) as long as the attended direction remained unchanged. On the other hand, systematically changing the attended direction from a neuron's preferred to its anti-preferred direction caused a systematic change of the attentional modulation from an enhancement to a suppression, increasing directionality by about 20%. CONCLUSIONS: The results show that (1) feature-based attention exerts a multiplicative modulation upon neuronal responses and that the strength of this modulation depends on the similarity between the attended feature and the cell's preferred feature, in line with the feature-similarity gain model, and (2) at the level of the neuronal population, feature-based attention increases the selectivity for attended features by increasing the responses of neurons preferring this feature value while decreasing responses of neurons tuned to the opposite feature value.     

A Bayesian Attractor Model for Perceptual Decision Making

Even for simple perceptual decisions, the mechanisms that the brain employs are still under debate. Although current consensus states that the brain accumulates evidence extracted from noisy sensory information, open questions remain about how this simple model relates to other perceptual phenomena such as flexibility in decisions, decision-dependent modulation of sensory gain, or confidence about a decision. We propose a novel approach of how perceptual decisions are made by combining two influential formalisms into a new model. Specifically, we embed an attractor model of decision making into a probabilistic framework that models decision making as Bayesian inference. We show that the new model can explain decision making behaviour by fitting it to experimental data. In addition, the new model combines for the first time three important features: First, the model can update decisions in response to switches in the underlying stimulus. Second, the probabilistic formulation accounts for top-down effects that may explain recent experimental findings of decision-related gain modulation of sensory neurons. Finally, the model computes an explicit measure of confidence which we relate to recent experimental evidence for confidence computations in perceptual decision tasks.     

Efficient “Communication through Coherence” Requires Oscillations Structured to Minimize Interference between Signals

The ‘communication through coherence’ (CTC) hypothesis proposes that selective communication among neural networks is achieved by coherence between firing rate oscillation in a sending region and gain modulation in a receiving region. Although this hypothesis has stimulated extensive work, it remains unclear whether the mechanism can in principle allow reliable and selective information transfer. Here we use a simple mathematical model to investigate how accurately coherent gain modulation can filter a population-coded target signal from task-irrelevant distracting inputs. We show that selective communication can indeed be achieved, although the structure of oscillatory activity in the target and distracting networks must satisfy certain previously unrecognized constraints. Firstly, the target input must be differentiated from distractors by the amplitude, phase or frequency of its oscillatory modulation. When distracting inputs oscillate incoherently in the same frequency band as the target, communication accuracy is severely degraded because of varying overlap between the firing rate oscillations of distracting inputs and the gain modulation in the receiving region. Secondly, the oscillatory modulation of the target input must be strong in order to achieve a high signal-to-noise ratio relative to stochastic spiking of individual neurons. Thus, whilst providing a quantitative demonstration of the power of coherent oscillatory gain modulation to flexibly control information flow, our results identify constraints imposed by the need to avoid interference between signals, and reveal a likely organizing principle for the structure of neural oscillations in the brain.     

A Normalization Model of Attentional Modulation of Single Unit Responses

Although many studies have shown that attention to a stimulus can enhance the responses of individual cortical sensory neurons, little is known about how attention accomplishes this change in response. Here, we propose that attention-based changes in neuronal responses depend on the same response normalization mechanism that adjusts sensory responses whenever multiple stimuli are present. We have implemented a model of attention that assumes that attention works only through this normalization mechanism, and show that it can replicate key effects of attention. The model successfully explains how attention changes the gain of responses to individual stimuli and also why modulation by attention is more robust and not a simple gain change when multiple stimuli are present inside a neuron''s receptive field. Additionally, the model accounts well for physiological data that measure separately attentional modulation and sensory normalization of the responses of individual neurons in area MT in visual cortex. The proposal that attention works through a normalization mechanism sheds new light a broad range of observations on how attention alters the representation of sensory information in cerebral cortex.     

Dynamic Excitatory and Inhibitory Gain Modulation Can Produce Flexible,Robust and Optimal Decision-making

Behavioural and neurophysiological studies in primates have increasingly shown the involvement of urgency signals during the temporal integration of sensory evidence in perceptual decision-making. Neuronal correlates of such signals have been found in the parietal cortex, and in separate studies, demonstrated attention-induced gain modulation of both excitatory and inhibitory neurons. Although previous computational models of decision-making have incorporated gain modulation, their abstract forms do not permit an understanding of the contribution of inhibitory gain modulation. Thus, the effects of co-modulating both excitatory and inhibitory neuronal gains on decision-making dynamics and behavioural performance remain unclear. In this work, we incorporate time-dependent co-modulation of the gains of both excitatory and inhibitory neurons into our previous biologically based decision circuit model. We base our computational study in the context of two classic motion-discrimination tasks performed in animals. Our model shows that by simultaneously increasing the gains of both excitatory and inhibitory neurons, a variety of the observed dynamic neuronal firing activities can be replicated. In particular, the model can exhibit winner-take-all decision-making behaviour with higher firing rates and within a significantly more robust model parameter range. It also exhibits short-tailed reaction time distributions even when operating near a dynamical bifurcation point. The model further shows that neuronal gain modulation can compensate for weaker recurrent excitation in a decision neural circuit, and support decision formation and storage. Higher neuronal gain is also suggested in the more cognitively demanding reaction time than in the fixed delay version of the task. Using the exact temporal delays from the animal experiments, fast recruitment of gain co-modulation is shown to maximize reward rate, with a timescale that is surprisingly near the experimentally fitted value. Our work provides insights into the simultaneous and rapid modulation of excitatory and inhibitory neuronal gains, which enables flexible, robust, and optimal decision-making.     

Representing where along with what information in a model of a cortical patch

Behaving in the real world requires flexibly combining and maintaining information about both continuous and discrete variables. In the visual domain, several lines of evidence show that neurons in some cortical networks can simultaneously represent information about the position and identity of objects, and maintain this combined representation when the object is no longer present. The underlying network mechanism for this combined representation is, however, unknown. In this paper, we approach this issue through a theoretical analysis of recurrent networks. We present a model of a cortical network that can retrieve information about the identity of objects from incomplete transient cues, while simultaneously representing their spatial position. Our results show that two factors are important in making this possible: A) a metric organisation of the recurrent connections, and B) a spatially localised change in the linear gain of neurons. Metric connectivity enables a localised retrieval of information about object identity, while gain modulation ensures localisation in the correct position. Importantly, we find that the amount of information that the network can retrieve and retain about identity is strongly affected by the amount of information it maintains about position. This balance can be controlled by global signals that change the neuronal gain. These results show that anatomical and physiological properties, which have long been known to characterise cortical networks, naturally endow them with the ability to maintain a conjunctive representation of the identity and location of objects.     

Computational modeling and exploration of contour integration for visual saliency

We propose a computational model of contour integration for visual saliency. The model uses biologically plausible devices to simulate how the representations of elements aligned collinearly along a contour in an image are enhanced. Our model adds such devices as a dopamine-like fast plasticity, local GABAergic inhibition and multi-scale processing of images. The fast plasticity addresses the problem of how neurons in visual cortex seem to be able to influence neurons they are not directly connected to, for instance, as observed in contour closure effect. Local GABAergic inhibition is used to control gain in the system without using global mechanisms which may be non-plausible given the limited reach of axonal arbors in visual cortex. The model is then used to explore not only its validity in real and artificial images, but to discover some of the mechanisms involved in processing of complex visual features such as junctions and end-stops as well as contours. We present evidence for the validity of our model in several phases, starting with local enhancement of only a few collinear elements. We then test our model on more complex contour integration images with a large number of Gabor elements. Sections of the model are also extracted and used to discover how the model might relate contour integration neurons to neurons that process end-stops and junctions. Finally, we present results from real world images. Results from the model suggest that it is a good current approximation of contour integration in human vision. As well, it suggests that contour integration mechanisms may be strongly related to mechanisms for detecting end-stops and junction points. Additionally, a contour integration mechanism may be involved in finding features for objects such as faces. This suggests that visual cortex may be more information efficient and that neural regions may have multiple roles.     

Attentional modulation strength in cortical area MT depends on stimulus contrast

The attentional modulation of sensory information processing in the visual system is the result of top-down influences, which can cause a multiplicative modulation of the firing rate of sensory neurons in extrastriate visual cortex, an effect reminiscent of the bottom-up effect of changes in stimulus contrast. This similarity could simply reflect the multiplicity of both effects. But, here we show that in direction-selective neurons in monkey visual cortical area MT, stimulus and attentional effects share a nonlinearity. These neurons show higher response gain for both contrast and attentional changes for intermediate contrast stimuli and smaller gain for low- and high-contrast stimuli. This finding suggests a close relationship between the neural encoding of stimulus contrast and the modulating effect of the behavioral relevance of stimuli.     

View-based encoding of actions in mirror neurons of area f5 in macaque premotor cortex

Converging experimental evidence indicates that mirror neurons in the monkey premotor area F5 encode the goals of observed motor acts [1-3]. However, it is unknown whether they also contribute to encoding the perspective from which the motor acts of others are seen. In order to address this issue, we recorded the visual responses of mirror neurons of monkey area F5 by using a novel experimental paradigm based on the presentation of movies showing grasping motor acts from different visual perspectives. We found that the majority of the tested mirror neurons?(74%) exhibited view-dependent activity with responses tuned to specific points of view. A minority of the tested mirror neurons (26%) exhibited view-independent responses. We conclude that view-independent mirror neurons encode action goals irrespective of the details of the observed motor acts, whereas the view-dependent ones might either form an intermediate step in the formation of view independence or contribute to a modulation of view-dependent representations in higher-level visual areas, potentially linking the goals of observed motor acts with their pictorial aspects.     

Contrast sensitivity and the detection of moving patterns and features

Theories based on optimal sampling by the retina have been widely applied to visual ecology at the level of the optics of the eye, supported by visual behaviour. This leads to speculation about the additional processing that must lie in between—in the brain itself. But fewer studies have adopted a quantitative approach to evaluating the detectability of specific features in these neural pathways. We briefly review this approach with a focus on contrast sensitivity of two parallel pathways for motion processing in insects, one used for analysis of wide-field optic flow, the other for detection of small features. We further use a combination of optical modelling of image blur and physiological recording from both photoreceptors and higher-order small target motion detector neurons sensitive to small targets to show that such neurons operate right at the limits imposed by the optics of the eye and the noise level of single photoreceptors. Despite this, and the limitation of only being able to use information from adjacent receptors to detect target motion, they achieve a contrast sensitivity that rivals that of wide-field motion sensitive pathways in either insects or vertebrates—among the highest in absolute terms seen in any animal.     

Mapping of contextual modulation in the population response of primary visual cortex

We review the evidence of long-range contextual modulation in V1. Populations of neurons in V1 are activated by a wide variety of stimuli outside of their classical receptive fields (RF), well beyond their surround region. These effects generally involve extra-RF features with an orientation component. The population mapping of orientation preferences to the upper layers of V1 is well understood, as far as the classical RF properties are concerned, and involves organization into pinwheel-like structures. We introduce a novel hypothesis regarding the organization of V1’s contextual response. We show that RF and extra-RF orientation preferences are mapped in related ways. Orientation pinwheels are the foci of both types of features. The mapping of contextual features onto the orientation pinwheel has a form that recapitulates the organization of the visual field: an iso-orientation patch within the pinwheel also responds to extra-RF stimuli of the same orientation. We hypothesize that the same form of mapping applies to other stimulus properties that are mapped out in V1, such as colour and contrast selectivity. A specific consequence is that fovea-like properties will be mapped in a systematic way to orientation pinwheels. We review the evidence that cytochrome oxidase blobs comprise the foci of this contextual remapping for colour and low contrasts. Neurodynamics and motion in the visual field are argued to play an important role in the shaping and maintenance of this type of mapping in V1.     

Dopaminergic input to GABAergic neurons in the rostral agranular insular cortex of the rat

Increasing evidence shows that the rostral agranular insular cortex (RAIC) is important in the modulation of nociception in humans and rats and that dopamine and GABA appear to be key neurotransmitters in the function of this cortical region. Here we use immunocytochemistry and path tracing to examine the relationship between dopamine and GABA related elements in the RAIC of the rat. We found that the RAIC has a high density of dopamine fibers that arise principally from the ipsilateral ventral tegmental area/substantia nigra (VTA/SN) and from a different set of neurons than those that project to the medial prefrontal cortex. Within the RAIC, there are close appositions between dopamine fibers and GABAergic interneurons. One target of cortical GABA appears to be a dense band of GABA_B receptor-bearing neurons located in lamina 5 of the RAIC. The GABA_B receptor-bearing neurons project principally to the amygdala and nucleus accumbens with few or no projections to the medial prefrontal cortex, cingulate gyrus, the mediodorsal thalamic nucleus or contralateral RAIC. The current anatomical data, together with previous behavioral results, suggest that part of the dopaminergic modulation of the RAIC occurs through GABAergic interneurons. GABA is able to exert specific effects through its action on GABA_B receptor-bearing projection neurons that target a few subcortical limbic structures. Through these connections, dopamine innervation of the RAIC is likely to affect the motivational and affective dimensions of pain.     

Goal-related activity in V4 during free viewing visual search. Evidence for a ventral stream visual salience map

Natural exploration of complex visual scenes depends on saccadic eye movements toward important locations. Saccade targeting is thought to be mediated by a retinotopic map that represents the locations of salient features. In this report, we demonstrate that extrastriate ventral area V4 contains a retinotopic salience map that guides exploratory eye movements during a naturalistic free viewing visual search task. In more than half of recorded cells, visually driven activity is enhanced prior to saccades that move the fovea toward the location previously occupied by a neuron's spatial receptive field. This correlation suggests that bottom-up processing in V4 influences the oculomotor planning process. Half of the neurons also exhibit top-down modulation of visual responses that depends on search target identity but not visual stimulation. Convergence of bottom-up and top-down processing streams in area V4 results in an adaptive, dynamic map of salience that guides oculomotor planning during natural vision.     

Cellular mechanisms underlying stimulus-dependent gain modulation in primary visual cortex neurons in vivo

Gain modulation is a widespread neuronal phenomenon that modifies response amplitude without changing selectivity. Computational and in vitro studies have proposed cellular mechanisms of gain modulation based on the postsynaptic effects of background synaptic activation, but these mechanisms have not been studied in vivo. Here, we used intracellular recordings from cat primary visual cortex to measure neuronal gain while changing background synaptic activity with visual stimulation. We found that increases in the membrane fluctuations associated with increases in synaptic input do not obligatorily result in gain modulation in vivo. However, visual stimuli that evoked sustained changes in resting membrane potential, input resistance, and membrane fluctuations robustly modulated neuronal gain. The magnitude of gain modulation depended critically on the spatiotemporal properties of the visual stimulus. Gain modulation in vivo may thus be determined on a moment-to-moment basis by sensory context and the consequent dynamics of synaptic activation.