Touch Interacts with Vision during Binocular Rivalry with a Tight Orientation Tuning

Multisensory integration is a common feature of the mammalian brain that allows it to deal more efficiently with the ambiguity of sensory input by combining complementary signals from several sensory sources. Growing evidence suggests that multisensory interactions can occur as early as primary sensory cortices. Here we present incompatible visual signals (orthogonal gratings) to each eye to create visual competition between monocular inputs in primary visual cortex where binocular combination would normally take place. The incompatibility prevents binocular fusion and triggers an ambiguous perceptual response in which the two images are perceived one at a time in an irregular alternation. One key function of multisensory integration is to minimize perceptual ambiguity by exploiting cross-sensory congruence. We show that a haptic signal matching one of the visual alternatives helps disambiguate visual perception during binocular rivalry by both prolonging the dominance period of the congruent visual stimulus and by shortening its suppression period. Importantly, this interaction is strictly tuned for orientation, with a mismatch as small as 7.5° between visual and haptic orientations sufficient to annul the interaction. These results indicate important conclusions: first, that vision and touch interact at early levels of visual processing where interocular conflicts are first detected and orientation tunings are narrow, and second, that haptic input can influence visual signals outside of visual awareness, bringing a stimulus made invisible by binocular rivalry suppression back to awareness sooner than would occur without congruent haptic input.     

A Model of Binocular Rivalry and Cross-orientation Suppression

Binocular rivalry and cross-orientation suppression are well-studied forms of competition in visual cortex, but models of these two types of competition are in tension with one another. Binocular rivalry occurs during the presentation of dichoptic grating stimuli, where two orthogonal gratings presented separately to the two eyes evoke strong alternations in perceptual dominance. Cross-orientation suppression occurs during the presentation of plaid stimuli, where the responses to a component grating presented to both eyes is weakened by the presence of a superimposed orthogonal grating. Conventional models of rivalry that rely on strong competition between orientation-selective neurons incorrectly predict rivalry between the components of plaids. Lowering the inhibitory weights in such models reduces rivalry for plaids, but also reduces it for dichoptic gratings. Using an exhaustive grid search, we show that this problem cannot be solved simply by adjusting the parameters of the model. Instead, we propose a robust class of models that rely on ocular opponency neurons, previously proposed as a mechanism for efficient stereo coding, to yield rivalry only for dichoptic gratings, not for plaids. This class of models reconciles models of binocular rivalry with the divisive normalization framework that has been used to explain cross-orientation. Our model makes novel predictions that we confirmed with psychophysical tests.     

A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields

Computational models of primary visual cortex have demonstrated that principles of efficient coding and neuronal sparseness can explain the emergence of neurones with localised oriented receptive fields. Yet, existing models have failed to predict the diverse shapes of receptive fields that occur in nature. The existing models used a particular "soft" form of sparseness that limits average neuronal activity. Here we study models of efficient coding in a broader context by comparing soft and "bard" forms of neuronal sparseness. As a result of our analyses, we propose a novel network model for visual cortex. The model forms efficient visual representations in which the number of active neurones, rather than mean neuronal activity, is limited. This form of hard sparseness also economises cortical resources like synaptic memory and metabolic energy. Furthermore, our model accurately predicts the distribution of receptive field shapes found in the primary visual cortex of cat and monkey.     

Ten_m3 Regulates Eye-Specific Patterning in the Mammalian Visual Pathway and Is Required for Binocular Vision

Binocular vision requires an exquisite matching of projections from each eye to form a cohesive representation of the visual world. Eye-specific inputs are anatomically segregated, but in register in the visual thalamus, and overlap within the binocular region of primary visual cortex. Here, we show that the transmembrane protein Ten_m3 regulates the alignment of ipsilateral and contralateral projections. It is expressed in a gradient in the developing visual pathway, which is consistently highest in regions that represent dorsal visual field. Mice that lack Ten_m3 show profound abnormalities in mapping of ipsilateral, but not contralateral, projections, and exhibit pronounced deficits when performing visually mediated behavioural tasks. It is likely that the functional deficits arise from the interocular mismatch, because they are reversed by acute monocular inactivation. We conclude that Ten_m3 plays a key regulatory role in the development of aligned binocular maps, which are required for normal vision.     

Slow Feature Analysis on Retinal Waves Leads to V1 Complex Cells

The developing visual system of many mammalian species is partially structured and organized even before the onset of vision. Spontaneous neural activity, which spreads in waves across the retina, has been suggested to play a major role in these prenatal structuring processes. Recently, it has been shown that when employing an efficient coding strategy, such as sparse coding, these retinal activity patterns lead to basis functions that resemble optimal stimuli of simple cells in primary visual cortex (V1). Here we present the results of applying a coding strategy that optimizes for temporal slowness, namely Slow Feature Analysis (SFA), to a biologically plausible model of retinal waves. Previously, SFA has been successfully applied to model parts of the visual system, most notably in reproducing a rich set of complex-cell features by training SFA with quasi-natural image sequences. In the present work, we obtain SFA units that share a number of properties with cortical complex-cells by training on simulated retinal waves. The emergence of two distinct properties of the SFA units (phase invariance and orientation tuning) is thoroughly investigated via control experiments and mathematical analysis of the input-output functions found by SFA. The results support the idea that retinal waves share relevant temporal and spatial properties with natural visual input. Hence, retinal waves seem suitable training stimuli to learn invariances and thereby shape the developing early visual system such that it is best prepared for coding input from the natural world.     

Binocular fusion in Panum''''s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the "correspondence problem". While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum's limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum's limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum's limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum's limiting case and that the uniqueness constraint is still valid.     

Binocular depth perception and the cerebral cortex

Our ability to coordinate the use of our left and right eyes and to make use of subtle differences between the images received by each eye allows us to perceive stereoscopic depth, which is important for the visual perception of three-dimensional space. Binocular neurons in the visual cortex combine signals from the left and right eyes. Probing the roles of binocular neurons in different perceptual tasks has advanced our understanding of the stages within the visual cortex that lead to binocular depth perception.     

Binocular single vision and perceptual processing.

Stimuli with small binocular disparities are seen as single, despite their differing visual directions for the two eyes. Such stimuli also yield stereopsis, but stereopsis and single vision can be dissociated. The occurrence of binocular single vision depends not only on the disparities of individual stimulus elements, but also on the geometrical relation of different parts of the pattern presented to each eye. A pair of vertical bars with opposite binocular disparities is seen as single if the pair is moderately widely spaced but not if it is narrow. Vertical alignment and identity in length of such bars also increase the occurrence of double vision. It is argued that these effects reflect the extraction of features of the monocular patterns, with these detected monocular features determining the binocular percept. Single and double vision of bars differing in orientation can be similarly analysed. The occurrence of relatively elaborate processing of monocular signals does not exclude the possibility that binocular interaction can occur between signals that have not been so processed. Multiple sites or types of binocular interaction are likely.     

Neural mechanisms of recovery following early visual deprivation

Natural patterned early visual input is essential for the normal development of the central visual pathways and the visual capacities they sustain. Without visual input, the functional development of the visual system stalls not far from the state at birth, and if input is distorted or biased the visual system develops in an abnormal fashion resulting in specific visual deficits. Monocular deprivation, an extreme form of biased exposure, results in large anatomical and physiological changes in terms of territory innervated by the two eyes in primary visual cortex (V1) and to a loss of vision in the deprived eye reminiscent of that in human deprivation amblyopia. We review work that points to a special role for binocular visual input in the development of V1 and vision. Our unique approach has been to provide animals with mixed visual input each day, which consists of episodes of normal and biased (monocular) exposures. Short periods of concordant binocular input, if continuous, can offset much longer episodes of monocular deprivation to allow normal development of V1 and prevent amblyopia. Studies of animal models of patching therapy for amblyopia reveal that the benefits are both heightened and prolonged by daily episodes of binocular exposure.     

Binocular fusion in Panum’s limiting case of stereopsis obeys the uniqueness constraint

In the information processing procedure of stereo vision, the uniqueness constraint has been used as one of the constraints to solve the “correspondence problem”. While the uniqueness constraint is valid in most cases, whether it is still valid in some particular stimulus configuration (such as Panum’s limiting case) has been a problem of widespread debate for a long time. To investigate the problem, we adopted the Panum’s limiting case as its basic stimulus configuration, and delved into the phenomenon of binocular fusion from two distinct aspects: visual direction and orientation disparity. The results show that in Panum’s limiting case binocular fusion does not comply with the rules governing regular binocular fusion as far as visual direction and orientation disparity are concerned. This indicates that double fusion does not happen in Panum’s limiting case and that the uniqueness constraint is still valid.     

Contextual modulation outside of awareness

Contextual effects are ubiquitous in vision and reveal fundamental principles of sensory coding. Here, we demonstrate that an oriented surround grating can affect the perceived orientation of a central test grating even when backward masking of the surround prevents its orientation from being consciously perceived. The effect survives introduction of a gap between test and surround of over a degree even under masking, suggesting either that contextual information can effectively propagate across early visual cortex in the absence of awareness of the signaled context or that it can proceed undetected to higher processing levels at which such horizontal propagation may not be necessary. The effect under masking also shows partial interocular transfer, demonstrating processing of orientation by binocular neurons in visual cortex in the absence of conscious orientation perception. This pattern of results is consistent with the suggestion that simultaneous orientation contrast is mediated at multiple levels of the visual processing hierarchy, and it supports the view that propagation of signals to and, possibly, back from higher visual areas is necessary for conscious perception.     

Innate visual learning through spontaneous activity patterns

Patterns of spontaneous activity in the developing retina, LGN, and cortex are necessary for the proper development of visual cortex. With these patterns intact, the primary visual cortices of many newborn animals develop properties similar to those of the adult cortex but without the training benefit of visual experience. Previous models have demonstrated how V1 responses can be initialized through mechanisms specific to development and prior to visual experience, such as using axonal guidance cues or relying on simple, pairwise correlations on spontaneous activity with additional developmental constraints. We argue that these spontaneous patterns may be better understood as part of an "innate learning" strategy, which learns similarly on activity both before and during visual experience. With an abstraction of spontaneous activity models, we show how the visual system may be able to bootstrap an efficient code for its natural environment prior to external visual experience, and we continue the same refinement strategy upon natural experience. The patterns are generated through simple, local interactions and contain the same relevant statistical properties of retinal waves and hypothesized waves in the LGN and V1. An efficient encoding of these patterns resembles a sparse coding of natural images by producing neurons with localized, oriented, bandpass structure-the same code found in early visual cortical cells. We address the relevance of higher-order statistical properties of spontaneous activity, how this relates to a system that may adapt similarly on activity prior to and during natural experience, and how these concepts ultimately relate to an efficient coding of our natural world.     

Knowing with which eye we see: utrocular discrimination and eye-specific signals in human visual cortex

Neurophysiological and behavioral reports converge to suggest that monocular neurons in the primary visual cortex are biased toward low spatial frequencies, while binocular neurons favor high spatial frequencies. Here we tested this hypothesis with functional magnetic resonance imaging (fMRI). Human participants viewed flickering gratings at one of two spatial frequencies presented to either the left or the right eye, and judged which of the two eyes was being stimulated (utrocular discrimination). Using multivoxel pattern analysis we found that local spatial patterns of signals in primary visual cortex (V1) allowed successful decoding of the eye-of-origin. Decoding was above chance for low but not high spatial frequencies, confirming the presence of a bias reported by animal studies in human visual cortex. Behaviorally, we found that reliable judgment of the eye-of-origin did not depend on spatial frequency. We further analyzed the mean response in visual cortex to our stimuli and revealed a weak difference between left and right eye stimulation. Our results are thus consistent with the interpretation that participants use overall levels of neural activity in visual cortex, perhaps arising due to local luminance differences, to judge the eye-of-origin. Taken together, we show that it is possible to decode eye-specific voxel pattern information in visual cortex but, at least in healthy participants with normal binocular vision, these patterns are unrelated to awareness of which eye is being stimulated.     

Binocular Combination of Second-Order Stimuli

Phase information is a fundamental aspect of visual stimuli. However, the nature of the binocular combination of stimuli defined by modulations in contrast, so-called second-order stimuli, is presently not clear. To address this issue, we measured binocular combination for first- (luminance modulated) and second-order (contrast modulated) stimuli using a binocular phase combination paradigm in seven normal adults. We found that the binocular perceived phase of second-order gratings depends on the interocular signal ratio as has been previously shown for their first order counterparts; the interocular signal ratios when the two eyes were balanced was close to 1 in both first- and second-order phase combinations. However, second-order combination is more linear than previously found for first-order combination. Furthermore, binocular combination of second-order stimuli was similar regardless of whether the carriers in the two eyes were correlated, anti-correlated, or uncorrelated. This suggests that, in normal adults, the binocular phase combination of second-order stimuli occurs after the monocular extracting of the second-order modulations. The sensory balance associated with this second-order combination can be obtained from binocular phase combination measurements.     

Binocular information in the control of prehensile movements in multiple-object scenes

Recent evidence suggests that the visual control of prehension may be less dependent on binocular information than has previously been thought. Studies investigating this question, however, have generally only examined reaches to single objects presented in isolation, even though natural prehensile movements are typically directed at objects in cluttered scenes which contain many objects. The present study was designed, therefore, to assess the contribution of binocular information to the control of prehensile movements in multiple-object scenes. Subjects reached for and grasped objects presented either in isolation or in the presence of one, two or four additional 'flanking' objects, under binocular and monocular viewing conditions. So that the role of binocular information could be clearly determined, subjects made reaches both in the absence of a visible scene around the target objects (self-illuminated objects presented in the dark) and under normal ambient lighting conditions. Analysis of kinematic parameters indicated that the removal of binocular information did not significantly affect many of the major indices of the transport component, including peak wrist velocity. However, peak grip apertures increased and subjects spent more time in the final slow phase of movement, prior to grasping the object, during monocularly guided reaches. The dissociation between effects of binocular versus monocular viewing on transport and grasp parameters was observed irrespective of the presence of flanking objects. These results therefore further question the view that binocular vision is pre-eminent in the control of natural prehensile movements.     

Elimination of inhibitory synapses is a major component of adult ocular dominance plasticity

During development, cortical plasticity is associated with the rearrangement of excitatory connections. While these connections become more stable with age, plasticity can still be induced in the adult cortex. Here we provide evidence that structural plasticity of?inhibitory synapses onto pyramidal neurons is?a major component of plasticity in the adult neocortex. In?vivo two-photon imaging was used to monitor the formation and elimination of fluorescently labeled inhibitory structures on pyramidal neurons. We find that ocular dominance plasticity in the adult visual cortex is associated with rapid inhibitory synapse loss, especially of those present on dendritic spines. This occurs not only with monocular deprivation but also with subsequent restoration of binocular vision. We propose that in the adult visual cortex the experience-induced loss of inhibition may effectively strengthen specific visual inputs with limited need for rearranging the excitatory circuitry.     

Perceived direction of motion determined by adaptation to static binocular images

In Li and Atick's [1, 2] theory of efficient stereo coding, the two eyes' signals are transformed into uncorrelated binocular summation and difference signals, and gain control is applied to the summation and differencing channels to optimize their sensitivities. In natural vision, the optimal channel sensitivities vary from moment to moment, depending on the strengths of the summation and difference signals; these channels should therefore be separately adaptable, whereby a channel's sensitivity is reduced following overexposure to adaptation stimuli that selectively stimulate that channel. This predicts a remarkable effect of binocular adaptation on perceived direction of a dichoptic motion stimulus [3]. For this stimulus, the summation and difference signals move in opposite directions, so perceived motion direction (upward or downward) should depend on which of the two binocular channels is most strongly adapted, even if the adaptation stimuli are completely static. We confirmed this prediction: a single static dichoptic adaptation stimulus presented for less than 1 s can control perceived direction of a subsequently presented dichoptic motion stimulus. This is not predicted by any current model of motion perception and suggests that the visual cortex quickly adapts to the prevailing binocular image statistics to maximize information-coding efficiency.     

Binocular visual processing in the owl's telencephalon.

Single neurons recorded from the owl's visual Wulst are surprisingly similar to those found in mammalian striate cortex. The receptive fields of Wulst neurons are elaborated, in an apparently hierarchical fashion, from those of their monocular, concentrically organized inputs to produce binocular interneurons with increasingly sophisticated requirements for stimulus orientation, movement and binocular disparity. Output neurons located in the superficial laminae of the Wulst are the most sophisticated of all, with absolute requirements for a combination of stimuli, which include binocular presentation at a particular horizontal binocular disparity, and with no response unless all of the stimulus conditions are satisfied simultaneously. Such neurons have the properties required for 'global stereopsis', including a receptive field size many times larger than their optimal stimulus, which is more closely matched to the receptive fields of the simpler, disparity-selective interneurons. These marked similarities in functional organization between the avian and mammalian systems exist in spite of a number of structural differences which reflect their separate evoluntionary origins. Discussion therefore includes the possibility that there may exist for nervous systems only a very small number of possible solutions, perhaps a unique one, to the problem of stereopsis.     

Environmental enrichment promotes plasticity and visual acuity recovery in adult monocular amblyopic rats

Loss of visual acuity caused by abnormal visual experience during development (amblyopia) is an untreatable pathology in adults. In some occasions, amblyopic patients loose vision in their better eye owing to accidents or illnesses. While this condition is relevant both for its clinical importance and because it represents a case in which binocular interactions in the visual cortex are suppressed, it has scarcely been studied in animal models. We investigated whether exposure to environmental enrichment (EE) is effective in triggering recovery of vision in adult amblyopic rats rendered monocular by optic nerve dissection in their normal eye. By employing both electrophysiological and behavioral assessments, we found a full recovery of visual acuity in enriched rats compared to controls reared in standard conditions. Moreover, we report that EE modulates the expression of GAD67 and BDNF. The non invasive nature of EE renders this paradigm promising for amblyopia therapy in adult monocular people.     

Efficient sparse coding in early sensory processing: lessons from signal recovery

Sensory representations are not only sparse, but often overcomplete: coding units significantly outnumber the input units. For models of neural coding this overcompleteness poses a computational challenge for shaping the signal processing channels as well as for using the large and sparse representations in an efficient way. We argue that higher level overcompleteness becomes computationally tractable by imposing sparsity on synaptic activity and we also show that such structural sparsity can be facilitated by statistics based decomposition of the stimuli into typical and atypical parts prior to sparse coding. Typical parts represent large-scale correlations, thus they can be significantly compressed. Atypical parts, on the other hand, represent local features and are the subjects of actual sparse coding. When applied on natural images, our decomposition based sparse coding model can efficiently form overcomplete codes and both center-surround and oriented filters are obtained similar to those observed in the retina and the primary visual cortex, respectively. Therefore we hypothesize that the proposed computational architecture can be seen as a coherent functional model of the first stages of sensory coding in early vision.