Knowing with which eye we see: utrocular discrimination and eye-specific signals in human visual cortex

Neurophysiological and behavioral reports converge to suggest that monocular neurons in the primary visual cortex are biased toward low spatial frequencies, while binocular neurons favor high spatial frequencies. Here we tested this hypothesis with functional magnetic resonance imaging (fMRI). Human participants viewed flickering gratings at one of two spatial frequencies presented to either the left or the right eye, and judged which of the two eyes was being stimulated (utrocular discrimination). Using multivoxel pattern analysis we found that local spatial patterns of signals in primary visual cortex (V1) allowed successful decoding of the eye-of-origin. Decoding was above chance for low but not high spatial frequencies, confirming the presence of a bias reported by animal studies in human visual cortex. Behaviorally, we found that reliable judgment of the eye-of-origin did not depend on spatial frequency. We further analyzed the mean response in visual cortex to our stimuli and revealed a weak difference between left and right eye stimulation. Our results are thus consistent with the interpretation that participants use overall levels of neural activity in visual cortex, perhaps arising due to local luminance differences, to judge the eye-of-origin. Taken together, we show that it is possible to decode eye-specific voxel pattern information in visual cortex but, at least in healthy participants with normal binocular vision, these patterns are unrelated to awareness of which eye is being stimulated.     

The dynamic model of binocular rivalry

Binocular rivalry is an interesting phenomenon observed in the human vision. It occurs when the right and left eyes are given different stimuli (pictures). This paper describes a mathematical model which explains the mechanism of binocular rivalry. Our basic assumption is that binocular rivalry is elicited by the mutual inhibition between the right and left visual neuron systems. The mutual inhibition between two neurons is first discussed in detail, where a special emphasis is put on a fatigue effect of neurons, and then its results are applied to a simulation model of binocular rivalry.     

Spatial stereoresolution for depth corrugations may be set in primary visual cortex

Stereo "3D" depth perception requires the visual system to extract binocular disparities between the two eyes' images. Several current models of this process, based on the known physiology of primary visual cortex (V1), do this by computing a piecewise-frontoparallel local cross-correlation between the left and right eye's images. The size of the "window" within which detectors examine the local cross-correlation corresponds to the receptive field size of V1 neurons. This basic model has successfully captured many aspects of human depth perception. In particular, it accounts for the low human stereoresolution for sinusoidal depth corrugations, suggesting that the limit on stereoresolution may be set in primary visual cortex. An important feature of the model, reflecting a key property of V1 neurons, is that the initial disparity encoding is performed by detectors tuned to locally uniform patches of disparity. Such detectors respond better to square-wave depth corrugations, since these are locally flat, than to sinusoidal corrugations which are slanted almost everywhere. Consequently, for any given window size, current models predict better performance for square-wave disparity corrugations than for sine-wave corrugations at high amplitudes. We have recently shown that this prediction is not borne out: humans perform no better with square-wave than with sine-wave corrugations, even at high amplitudes. The failure of this prediction raised the question of whether stereoresolution may actually be set at later stages of cortical processing, perhaps involving neurons tuned to disparity slant or curvature. Here we extend the local cross-correlation model to include existing physiological and psychophysical evidence indicating that larger disparities are detected by neurons with larger receptive fields (a size/disparity correlation). We show that this simple modification succeeds in reconciling the model with human results, confirming that stereoresolution for disparity gratings may indeed be limited by the size of receptive fields in primary visual cortex.     

Stereoscopic vision: what's the first step?

Neurons in primary visual cortex respond to binocular disparity, the raw material of stereoscopic depth perception. Although these neurons are probably essential to depth perception, a recent study has shown that they are unable to compute depth itself.     

How birds use their eyes: Opposite left-right specialization for the lateral and frontal visual hemifield in the domestic chick

Recent evidence has demonstrated that, in animals with laterally placed eyes, functional cerebral asymmetry is revealed by preferential use of either the left or right eye in a range of behaviors (birds: [1, 2, 3]; fish: [4, 5]; reptiles: [6, 7]). These findings pose a theoretical problem. It seems that there would be disadvantages in having a substantial degree of asymmetry in the use of the two eyes; a deficit on one side would leave the organism vulnerable to attack on that side or unable to exploit resources appearing on one side. We here report a possible solution to the problem. We have found that domestic chicks show selective use of the lateral visual field of the left eye and of the right hemifield in the binocular, frontal visual field when they peck at strangers but not at cagemates. Thus, during social recognition, there seems to be opposite and complementary left-right specialization for the lateral and frontal visual fields of the two eyes. These findings can reconcile the computational advantages associated with asymmetry of the left and right sides of the brain with the ecological demands for an animal to perceive and respond equally well to the left and right sides of its midline.     

Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation

Primary visual cortex is often viewed as a “cyclopean retina”, performing the initial encoding of binocular disparities between left and right images. Because the eyes are set apart horizontally in the head, binocular disparities are predominantly horizontal. Yet, especially in the visual periphery, a range of non-zero vertical disparities do occur and can influence perception. It has therefore been assumed that primary visual cortex must contain neurons tuned to a range of vertical disparities. Here, I show that this is not necessarily the case. Many disparity-selective neurons are most sensitive to changes in disparity orthogonal to their preferred orientation. That is, the disparity tuning surfaces, mapping their response to different two-dimensional (2D) disparities, are elongated along the cell''s preferred orientation. Because of this, even if a neuron''s optimal 2D disparity has zero vertical component, the neuron will still respond best to a non-zero vertical disparity when probed with a sub-optimal horizontal disparity. This property can be used to decode 2D disparity, even allowing for realistic levels of neuronal noise. Even if all V1 neurons at a particular retinotopic location are tuned to the expected vertical disparity there (for example, zero at the fovea), the brain could still decode the magnitude and sign of departures from that expected value. This provides an intriguing counter-example to the common wisdom that, in order for a neuronal population to encode a quantity, its members must be tuned to a range of values of that quantity. It demonstrates that populations of disparity-selective neurons encode much richer information than previously appreciated. It suggests a possible strategy for the brain to extract rarely-occurring stimulus values, while concentrating neuronal resources on the most commonly-occurring situations.     

Stereoscopic vision: solving the correspondence problem

Neurons in early visual areas respond to horizontal disparity in images that do not give rise to stereopsis. False binocular matches, however, are discarded at the apex of the visual pathway: the activity of neurons in the primate inferior temporal cortex correlates directly with conscious depth perception.     

Receptive fields of disparity-tuned simple cells in macaque V1

Binocular simple cells in primary visual cortex (V1) are the first cells along the mammalian visual pathway to receive input from both eyes. Two models of how binocular simple cells could extract disparity information have been put forward. The phase-shift model proposes that the receptive fields in the two eyes have different subunit organizations, while the position-shift model proposes that they have different overall locations. In five fixating macaque monkeys, we recorded from 30 disparity-tuned simple cells that showed selectivity to the disparity in a random dot stereogram. High-resolution maps of the left and right eye receptive fields indicated that both phase and position shifts were common. Single cells usually showed a combination of the two, and the optimum disparity was best correlated with the sum of receptive field phase and position shift.     

Stereopsis and the random element in the organization of the striate cortex.

Receptive field position and orientation disparities are both properties of binocularly discharged striate neurons. Receptive field position desparities have been used as a key element in the neural theory for binocular depth discrimination. Since most striate cells in the cat are binocular, these position disparities require that cells immediately adjacent to one another in the cortex should show a random scatter in their monocular receptive field positions. Superimposed on the progressive topographical representation of the visual field on the striate cortex there is experimental evidence for a localized monocular receptive field position scatter. The suggestion is examined that the binocular position disparities are built up out of the two monocular position scatters. An examination of receptive field orientation disparities and their relation to the random variation in the monocular preferred orientations of immediately adjacent striate neurons also leads to the conclusion that binocular orientation disparities are a consequence of the two monocular scatters. As for receptive field position, the local scatter in preferred orientation is superimposed on a progressive representation of orientation over larger areas of the cortex. The representation in the striate cortex of visual field position and of stimulus orientation is examined in relation to the correlation between the disparities in receptive field position and preferred orientation. The role of orientation disparities in binocular vision is reviewed.     

Binocular depth perception mechanisms in tongue-projecting salamanders

Tongue-projecting salamanders (Bolitoglossini) combine extreme speed and high precision in prey capture. They possess all requirements for stereoscopic depth perception: frontally oriented eyes, a substantial amount of direct ipsilateral projection in addition to the contralateral one, and binocularly driven neurons. Extracellular recordings were made from retinal afferents in the tectum as well as from the somata of tectal neurons. RF-sizes of afferents and tectal neurons were determined, and the response properties of tectal neurons were tested under monocular and binocular conditions with stimuli of different size and velocity. While RF-sizes and response properties of binocular neurons during binocular and contralateral stimulation were similar, ipsilaterally stimulated neurons exhibited much smaller RFs, lower spike rates and different size preferences.Furthermore, the contralateral retinotectal projection from one eye and the ipsilateral from the other are in register. While retinal afferents are distributed linearly over the tectal surface, most tectal neurons are activated by a retinal area corresponding to the frontal visual field; this results in a magnification of this region. The two monocular receptive fields of binocular neurons exhibit zero disparities (horopter) at distances that coincide with the maximum reach of the tongue. We hypothesize that bolitoglossine salamanders (as well as amphibians in general) make use of two kinds of disparities: (1) between the maps in the left and right tectal hemisphere, coding for the lateral eccentricity of an object, and (2) between the ipsilateral and contralateral retinotectal map, coding for the distance. The presence of substantial direct ipsilateral afferents in bolitoglossine salamanders appears to be the basis for a fast computation of object distance, which is characteristic of these animals.Abbreviations Ax/Ay coordinates of a recorded afference - Nx/Ny coordinates of a recorded neuron - RF receptive field - RFc contralateral receptive field - RFi ipsilateral receptive field - RFx/RFy coordinates of a receptive field center - RGC retinal ganglion cell     

An inference upon the neural network finding binocular correspondence

Previously, the authors proposed a model of neural network extracting binocular parallax (Hirai and Fukushima, 1975). It is a multilayered network whose final layers consist of neural elements corresponding to binocular depth neurons found in monkey's visual cortex. The binocular depth neuron is selectively sensitive to a binocular stimulus with a specific amount of binocular parallax and does not respond to a monocular one. As described in the last chapter of the previous article (Hirai and Fukushima, 1975), when a binocular pair of input patterns consist of, for example, many vertical bars placed very closely to each other, the binocular depth neurons might respond not only to correct binocular pairs, but also to incorrect ones. Our present study is concentrated upon how the visual system finds correct binocular pairs or binocular correspondence. It is assumed that some neural network is cascaded after the binocular depth neurons and finds out correct binocular correspondence by eliminating the incorrect binocular pairs. In this article a model of such neural network is proposed. The performance of the model has been simulated on a digital computer. The results of the computer simulation show that this model finds binocular correspondence satisfactorily. It has been demonstrated by the computer simulation that this model also explains the mechanism of the hysteresis in the binocular depth perception reported by Fender and Julesz (1967)This work has been done in the NHK Broadcasting Science Research Laboratories     

Bistable percepts in the brain: FMRI contrasts monocular pattern rivalry and binocular rivalry

The neural correlates of binocular rivalry have been actively debated in recent years, and are of considerable interest as they may shed light on mechanisms of conscious awareness. In a related phenomenon, monocular rivalry, a composite image is shown to both eyes. The subject experiences perceptual alternations in which the two stimulus components alternate in clarity or salience. The experience is similar to perceptual alternations in binocular rivalry, although the reduction in visibility of the suppressed component is greater for binocular rivalry, especially at higher stimulus contrasts. We used fMRI at 3T to image activity in visual cortex while subjects perceived either monocular or binocular rivalry, or a matched non-rivalrous control condition. The stimulus patterns were left/right oblique gratings with the luminance contrast set at 9%, 18% or 36%. Compared to a blank screen, both binocular and monocular rivalry showed a U-shaped function of activation as a function of stimulus contrast, i.e. higher activity for most areas at 9% and 36%. The sites of cortical activation for monocular rivalry included occipital pole (V1, V2, V3), ventral temporal, and superior parietal cortex. The additional areas for binocular rivalry included lateral occipital regions, as well as inferior parietal cortex close to the temporoparietal junction (TPJ). In particular, higher-tier areas MT+ and V3A were more active for binocular than monocular rivalry for all contrasts. In comparison, activation in V2 and V3 was reduced for binocular compared to monocular rivalry at the higher contrasts that evoked stronger binocular perceptual suppression, indicating that the effects of suppression are not limited to interocular suppression in V1.     

Eye-specific segregation of optic afferents in mammals,fish, and frogs: The role of activity

Eye-specific patches or stripes normally develop in the visual cortex and superior colliculus of many (but not all) mammals and are also formed, after surgically produced binocular innervation, in the optic tectum of fish and frogs. The segregation of ocular dominance patches or columns has been studied using a variety of anatomical pathway-tracing techniques, by electrophysiological recording of postsynaptic units or field potentials, and by the 2-deoxyglucose method following visual stimulation of only one eye. In the tectum of both fish and frogs and in the cortex and colliculus of mammals, eye-specific patches develop from initially diffuse, overlapping projections. Of the various mechanisms that might cause such segregation, the evidence favors an activity-dependent process that stabilizes synapses from the same eye because of their correlated activity. First, several environmental manipulations affect the segregation of afferents in visual cortex: strabismus and alternate monocular exposure apparently enhance segregation, whereas dark rearing slows the segregation process, and monocular deprivation causes the experienced eye to form larger patches at the expense of those of the deprived eye. Second, blocking activity in both eyes is effective in preventing the segregation both in the tectum of fish and frog and in the visual cortex of cat. With the eyes blocked, alternate stimulation of the optic nerves permits the segregation of ocular dominance, at least onto single cells in the cat visual cortex. These findings are discussed in terms of an activity-dependent stabilization of those synapses having correlated activity (those from neighboring ganglion cells within one eye) but not of those lacking correlated activity (those from left and right eyes). We suggest that the eye-specific patches represent a compromise between total segregation of the projections from the two eyes and the formation of a single continuous retinotopic map across the surface of the cortex or tectum.     

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.This article is part of the themed issue ‘Vision in our three-dimensional world''.     

Frontal cortical control of smooth-pursuit

To maintain optimal clarity of objects moving slowly in three dimensional space, frontal eyed-primates use both smooth-pursuit and vergence (depth) eye movements to track precisely those objects and maintain their images on the foveae of left and right eyes. The caudal parts of the frontal eye fields contain neurons that discharge during smooth-pursuit. Recent results have provided a new understanding of the roles of the frontal eye field pursuit area and suggest that it may control the gain of pursuit eye movements, code predictive visual signals that drive pursuit, and code commands for smooth eye movements in a three dimensional coordinate frame.     

Stereoscopic subsystems for position in depth and for motion in depth.

We describe psychophysical evidence that the human visual system contains information-processing channels for motion in depth in addition to those for position in depth. These motion-in-depth channels include some that are selectively sensitive to the relative velocities of the left and right retinal images. We propose that the visual pathway contains stereoscopic (cyclopean) motion filters that respond to only a narrow range of the directions of motion in depth. Turning to the single-neuron level we report that, in addition to neurons turned to position to depth, cat visual cortex contains neurons that emphasize information about the direction of motion at the expense of positional information. We describe psychophysical evidence for the existence of channels that are sensitive to change size, and are separate from the channels both for motion and for flicker. These changing-size channels respond independently of whether the stimulus is a bright square on a dark ground or a dark square on a bright ground. At the physiological level we report single neurons in cat visual cortex that respond selectively to increasing or to decreasing size independently of the sign of stimulus contrast. Adaptation to a changing-size stimulus produces two separable after-effects: an illusion of changing size, and an illusion of motion in depth. These after-effects have different decay time constants. We propose a psychophysical model in which changing-size filters feed a motion-in-depth stage, and suppose that the motion-in-depth after-effect is due to activity at the motion-in-depth stage, while the changing-size after-effect is due to to activity at the changing-size and more peripheral stages. The motion-in-depth after-effect can be cancelled either by a changing-size test stimulus or by relative motion of the left and right retinal images. Opposition of these two cues can also cancel the impression of motion in depth produced by the adapting stimulus. These findings link the stereoscopic (cyclopean) motion filters and the changing-size filters: both feed the same motion-in-depth stage.     

Effects of cortical damage on binocular depth perception

Stereoscopic depth perception requires considerable neural computation, including the initial correspondence of the two retinal images, comparison across the local regions of the visual field and integration with other cues to depth. The most common cause for loss of stereoscopic vision is amblyopia, in which one eye has failed to form an adequate input to the visual cortex, usually due to strabismus (deviating eye) or anisometropia. However, the significant cortical processing required to produce the percept of depth means that, even when the retinal input is intact from both eyes, brain damage or dysfunction can interfere with stereoscopic vision. In this review, I examine the evidence for impairment of binocular vision and depth perception that can result from insults to the brain, including both discrete damage, temporal lobectomy and more systemic diseases such as posterior cortical atrophy.This article is part of the themed issue ‘Vision in our three-dimensional world’.     

A model of neural network extracting binocular parallax

A model of neural network extracting binocular parallax is proposed. It is a multilayered network composed of analog threshold elements. Three types of binocular neurons are included in this model. They are binocular simple neurons, binocular gate neurons and binocular depth neurons. The final layers of this model consist of elements which correspond to the binocular depth neurons. The performance of the model has been simulated on a digital computer. The results of the computer simulation show that every element of this model acts like neurons found in cat's and monkey's visual system and this model extracts binocular parallax caused by simple line components satisfactorily.     

The Role of Binocular Disparity in Stereoscopic Images of Objects in the Macaque Anterior Intraparietal Area

Neurons in the macaque Anterior Intraparietal area (AIP) encode depth structure in random-dot stimuli defined by gradients of binocular disparity, but the importance of binocular disparity in real-world objects for AIP neurons is unknown. We investigated the effect of binocular disparity on the responses of AIP neurons to images of real-world objects during passive fixation. We presented stereoscopic images of natural and man-made objects in which the disparity information was congruent or incongruent with disparity gradients present in the real-world objects, and images of the same objects where such gradients were absent. Although more than half of the AIP neurons were significantly affected by binocular disparity, the great majority of AIP neurons remained image selective even in the absence of binocular disparity. AIP neurons tended to prefer stimuli in which the depth information derived from binocular disparity was congruent with the depth information signaled by monocular depth cues, indicating that these monocular depth cues have an influence upon AIP neurons. Finally, in contrast to neurons in the inferior temporal cortex, AIP neurons do not represent images of objects in terms of categories such as animate-inanimate, but utilize representations based upon simple shape features including aspect ratio.