An inference upon the neural network finding binocular correspondence

Previously, the authors proposed a model of neural network extracting binocular parallax (Hirai and Fukushima, 1975). It is a multilayered network whose final layers consist of neural elements corresponding to binocular depth neurons found in monkey's visual cortex. The binocular depth neuron is selectively sensitive to a binocular stimulus with a specific amount of binocular parallax and does not respond to a monocular one. As described in the last chapter of the previous article (Hirai and Fukushima, 1975), when a binocular pair of input patterns consist of, for example, many vertical bars placed very closely to each other, the binocular depth neurons might respond not only to correct binocular pairs, but also to incorrect ones. Our present study is concentrated upon how the visual system finds correct binocular pairs or binocular correspondence. It is assumed that some neural network is cascaded after the binocular depth neurons and finds out correct binocular correspondence by eliminating the incorrect binocular pairs. In this article a model of such neural network is proposed. The performance of the model has been simulated on a digital computer. The results of the computer simulation show that this model finds binocular correspondence satisfactorily. It has been demonstrated by the computer simulation that this model also explains the mechanism of the hysteresis in the binocular depth perception reported by Fender and Julesz (1967)This work has been done in the NHK Broadcasting Science Research Laboratories     

The Role of Binocular Disparity in Stereoscopic Images of Objects in the Macaque Anterior Intraparietal Area

Neurons in the macaque Anterior Intraparietal area (AIP) encode depth structure in random-dot stimuli defined by gradients of binocular disparity, but the importance of binocular disparity in real-world objects for AIP neurons is unknown. We investigated the effect of binocular disparity on the responses of AIP neurons to images of real-world objects during passive fixation. We presented stereoscopic images of natural and man-made objects in which the disparity information was congruent or incongruent with disparity gradients present in the real-world objects, and images of the same objects where such gradients were absent. Although more than half of the AIP neurons were significantly affected by binocular disparity, the great majority of AIP neurons remained image selective even in the absence of binocular disparity. AIP neurons tended to prefer stimuli in which the depth information derived from binocular disparity was congruent with the depth information signaled by monocular depth cues, indicating that these monocular depth cues have an influence upon AIP neurons. Finally, in contrast to neurons in the inferior temporal cortex, AIP neurons do not represent images of objects in terms of categories such as animate-inanimate, but utilize representations based upon simple shape features including aspect ratio.     

A system model for investigating passive electrical properties of neurons.

Passive membrane properties of neurons, characterized by a linear voltage response to constant current stimulation, were investigated by busing a system model approach. This approach utilizes the derived expression for the input impedance of a network, which simulates the passive properties of neurons, to correlate measured intracellular recordings with the response of network models. In this study, the input impedances of different network configurations and of dentate granule neurons, were derived as a function of the network elements and were validated with computer simulations. The parameters of the system model, which are the values of the network elements, were estimated using an optimization strategy. The system model provides for better estimation of the network elements than the previously described signal model, due to its explicit nature. In contrast, the signal model is an implicit function of the network elements which requires intermediate steps to estimate some of the passive properties.     

A model for repetitive firing in neurons

This paper describes a model for the generation of repetitive firing patterns in single neurons to be used as a module in large-scale network simulation studies. The model is based on the combination of extended versions of Hill's model for accomodation and of Kernell's model for adaptation. Both digital computer and electronic circuit realizations of the model are presented. The model is shown to produce strength-duration curves for accomodation which are compatible with available data from real neurons. Both “high ceiling” and “low ceiling” cell types can be matched by adjusting parameters in the model. An equation relating steady-state firing rate to amplitude of applied steady current is presented which includes the accumulation of potassium conductance changes with repetitive firing. The occurence of phasic and tonic responses to step stimulation is mapped in the parameter space of the model. Several representative response patterns to irregular inputs are presented.     

Adaptation to natural binocular disparities in primate V1 explained by a generalized energy model

Sensory processing in the brain is thought to have evolved to encode naturally occurring stimuli efficiently. We report an adaptation in binocular cortical neurons that reflects the tight constraints imposed by the geometry of 3D vision. We show that the widely used binocular energy model predicts that neurons dedicate part of their dynamic range to impossible combinations of left and right images. Approximately 42% of the neurons we record from V1 of awake monkeys behave in this way (a powerful confirmation of the model), while about 58% deviate from the model in a manner that concentrates more of their dynamic range on stimuli that obey the constraints of binocular geometry. We propose a simple extension of the energy model, using multiple subunits, that explains the adaptation we observe, as well as other properties of binocular neurons that have been hard to account for, such as the response to anti-correlated stereograms.     

Performance data indicate summation for pictorial depth-cues in slanted surfaces

Over recent years much has been learned about the way in which depth cues are combined (e.g. Landy et al.. 1995). The majority of this work has used subjective measures, a rating scale or a point of subjective equality, to deduce the relative contributions of different cues to perception. We have adopted a very different approach by using two interval forced-choice (21FC) performance measures and a signal processing framework. We performed summation experiments for depth cue increment thresholds between pairs of pictorial depth cues in displays depicting slanted planar surfaces made from arrays of circular 'contrast' elements. Summation was found to be ideal when size-gradient was paired with contrast-gradient for a wide range of depth-gradient magnitudes in the null stimulus. For a pairing of size-gradient and linear perspective, substantial summation (> 1.5 dB) was found only when the null stimulus had intermediate depth gradients; when flat or steeply inclined surfaces were depicted, summation was diminished or abolished. Summation was also abolished when one of the target cues was (i) not a depth cue, or (ii) added in conflict. We conclude that vision has a depth mechanism for the constructive combination of pictorial depth cues and suggest two generic models of summation to describe the results. Using similar psychophysical methods. Bradshaw and Rogers (1996) revealed a mechanism for the depth cues of motion parallax and binocular disparity. Whether this is the same or a different mechanism from the one reported here awaits elaboration.     

Spatial stereoresolution for depth corrugations may be set in primary visual cortex

Stereo "3D" depth perception requires the visual system to extract binocular disparities between the two eyes' images. Several current models of this process, based on the known physiology of primary visual cortex (V1), do this by computing a piecewise-frontoparallel local cross-correlation between the left and right eye's images. The size of the "window" within which detectors examine the local cross-correlation corresponds to the receptive field size of V1 neurons. This basic model has successfully captured many aspects of human depth perception. In particular, it accounts for the low human stereoresolution for sinusoidal depth corrugations, suggesting that the limit on stereoresolution may be set in primary visual cortex. An important feature of the model, reflecting a key property of V1 neurons, is that the initial disparity encoding is performed by detectors tuned to locally uniform patches of disparity. Such detectors respond better to square-wave depth corrugations, since these are locally flat, than to sinusoidal corrugations which are slanted almost everywhere. Consequently, for any given window size, current models predict better performance for square-wave disparity corrugations than for sine-wave corrugations at high amplitudes. We have recently shown that this prediction is not borne out: humans perform no better with square-wave than with sine-wave corrugations, even at high amplitudes. The failure of this prediction raised the question of whether stereoresolution may actually be set at later stages of cortical processing, perhaps involving neurons tuned to disparity slant or curvature. Here we extend the local cross-correlation model to include existing physiological and psychophysical evidence indicating that larger disparities are detected by neurons with larger receptive fields (a size/disparity correlation). We show that this simple modification succeeds in reconciling the model with human results, confirming that stereoresolution for disparity gratings may indeed be limited by the size of receptive fields in primary visual cortex.     

Reliability of pattern separation by the cerebellar mossy fiber — granule cell system

As a simple example of a neuronal network in which synaptic connectivity among neurons is probabilistic, Marr's model for the granular layer of cat cerebellar cortex is examined. The mean and variance are computed for the fraction of granule cells activated, and for the extent of pattern separation by granule cells, for various mossy fiber inputs and various values of connectivity and electrical parameters of the network structure. Results suggest different functions for the network, and different optimal ranges for its parameters, depending on whether Golgi cells are present or absent. The model network does not perform the functions originally prescribed for it with high reliability.     

Electrophysiological Correlates of Binocular Stereo Depth without Binocular Disparities

A small region of background presented to only one eye in an otherwise binocular display may, under certain conditions, be resolved in the visual system by interpreting the region as a small gap between two similar objects placed at different depths, with the gap hidden in one eye by parallax. This has been called monocular gap stereopsis. We investigated the electrophysiological correlate of this type of stereopsis by means of sum potential recordings in 12 observers, comparing VEP's for this stimulus ("Gillam Stereo", Author BG has strong reservations about this term) with those for similar stimuli containing disparity based depth and with no depth (flat). In addition we included several control stimuli. The results show a pronounced early negative potential at a latency of around 170 ms (N170) for all stimuli containing non- identical elements, be they a difference caused by binocular disparity or by completely unmatched monocular contours. A second negative potential with latency around 270 ms (N270), on the other hand, is present only with stimuli leading to fusion and the perception of depth. This second component is similar for disparity-based stereopsis and monocular gap, or "Gillam Stereo" although slightly more pronounced for the former. We conjecture that the first component is related to the detection of differences between the images of the two eyes that may then either be fused, leading to stereopsis and the corresponding second potential, or else to inhibition and rivalry without a later trace in the VEP. The finding that that "Gillam Stereo" leads to cortical responses at the same short latencies as disparity based stereopsis indicates that it may partly rely on quite early cortical mechanisms.     

Defining the V5/MT neuronal pool for perceptual decisions in a visual stereo-motion task

In the primate visual cortex, neurons signal differences in the appearance of objects with high precision. However, not all activated neurons contribute directly to perception. We defined the perceptual pool in extrastriate visual area V5/MT for a stereo-motion task, based on trial-by-trial co-variation between perceptual decisions and neuronal firing (choice probability (CP)). Macaque monkeys were trained to discriminate the direction of rotation of a cylinder, using the binocular depth between the moving dots that form its front and rear surfaces. We manipulated the activity of single neurons trial-to-trial by introducing task-irrelevant stimulus changes: dot motion in cylinders was aligned with neuronal preference on only half the trials, so that neurons were strongly activated with high firing rates on some trials and considerably less activated on others. We show that single neurons maintain high neurometric sensitivity for binocular depth in the face of substantial changes in firing rate. CP was correlated with neurometric sensitivity, not level of activation. In contrast, for individual neurons, the correlation between perceptual choice and neuronal activity may be fundamentally different when responding to different stimulus versions. Therefore, neuronal pools supporting sensory discrimination must be structured flexibly and independently for each stimulus configuration to be discriminated.This article is part of the themed issue ‘Vision in our three-dimensional world''.     

Oscillatory binocular system and temporal segmentation of stereoscopic depth surfaces

A dynamical neural network model of binocular stereopsis is proposed to solve the problem of segmentation which remains ambiguous even when the problem of binocular correspondence is solved. Being compatible with the recent neurophysiological findings (Engel et al. 1991), the model assumes that neural cells show oscillatory activities and that segmentation into a coherent depth surface is coded by synchronization of activities. Employing appropriate constraints for segmentation, the present model shows proper segmentation of depth surfaces and also solves segmentational ambiguity caused by a gap. It is newly shown that binocularly-unmatched monocular cells are discriminated in temporal segmentation of monocular cells caused by recurrent interactions between monocular and binocular cells. Integrative interactions with the other visual components through temporal segmentation are also discussed.     

A binocular model for the simple cell

We authors propose a mathematical model for simple cell binocular response. It comprises two Gabor-type receptive fields (RF) having the same RF center, preferred spatial frequency, and preferred orientation. The model integrates the equally weighted signals from both eyes and performs a threshold operation. Poggio and Fischer (1977) classified binocular disparity cells in the striate cortex into four groups: tuned excitatory (TE), tuned inhibitory (TI), near, and far cells. They also found that most of the TE cells are ocularly balanced and that the other three types are usually unbalanced. This model can imitate these four types of disparity sensitivities and their ocular dominance tendency. We perform model fittings to Poggio's data using the “simulated annealing” method and discuss parameter dependence of the model's response. The model can also respond with exceptional disparity sensitivity: i.e., flat type, alternating type, and intermediate type.     

A model for nonlinear stochastic behavior of the pupil

In binocular fusion, pairs of left and right stimuli yielding the same brightness perception constitute an equibrightness curve in a coordinate system whose ordinate and abscissa correspond to the left and right stimulus strengths. A neural network model is presented to elucidate the characteristics of the curve. According to the model, Fechner's paradox is due to the threshold characteristics of the neuron. If the shapes or movements are radically different between the left and right stimuli, the retinal rivalry is caused. That is, only the left stimulus is perceived at one moment and the right stimulus at another moment. The period of left or right eye dominance alternates randomly from time to time. The distribution of the period is approximate to the gamma distribution. In order to account for this fact, a neural network model is proposed, which consists of a pair of neurons receiving inputs with stochastic fluctuations. The computer simulation was carried out with satisfactory results. The model of retinal rivalry is integrated with that of brightness perception.     

Neural field model of binocular rivalry waves

We present a neural field model of binocular rivalry waves in visual cortex. For each eye we consider a one-dimensional network of neurons that respond maximally to a particular feature of the corresponding image such as the orientation of a grating stimulus. Recurrent connections within each one-dimensional network are assumed to be excitatory, whereas connections between the two networks are inhibitory (cross-inhibition). Slow adaptation is incorporated into the model by taking the network connections to exhibit synaptic depression. We derive an analytical expression for the speed of a binocular rivalry wave as a function of various neurophysiological parameters, and show how properties of the wave are consistent with the wave-like propagation of perceptual dominance observed in recent psychophysical experiments. In addition to providing an analytical framework for studying binocular rivalry waves, we show how neural field methods provide insights into the mechanisms underlying the generation of the waves. In particular, we highlight the important role of slow adaptation in providing a “symmetry breaking mechanism” that allows waves to propagate.     

Vertical Binocular Disparity is Encoded Implicitly within a Model Neuronal Population Tuned to Horizontal Disparity and Orientation

Primary visual cortex is often viewed as a “cyclopean retina”, performing the initial encoding of binocular disparities between left and right images. Because the eyes are set apart horizontally in the head, binocular disparities are predominantly horizontal. Yet, especially in the visual periphery, a range of non-zero vertical disparities do occur and can influence perception. It has therefore been assumed that primary visual cortex must contain neurons tuned to a range of vertical disparities. Here, I show that this is not necessarily the case. Many disparity-selective neurons are most sensitive to changes in disparity orthogonal to their preferred orientation. That is, the disparity tuning surfaces, mapping their response to different two-dimensional (2D) disparities, are elongated along the cell''s preferred orientation. Because of this, even if a neuron''s optimal 2D disparity has zero vertical component, the neuron will still respond best to a non-zero vertical disparity when probed with a sub-optimal horizontal disparity. This property can be used to decode 2D disparity, even allowing for realistic levels of neuronal noise. Even if all V1 neurons at a particular retinotopic location are tuned to the expected vertical disparity there (for example, zero at the fovea), the brain could still decode the magnitude and sign of departures from that expected value. This provides an intriguing counter-example to the common wisdom that, in order for a neuronal population to encode a quantity, its members must be tuned to a range of values of that quantity. It demonstrates that populations of disparity-selective neurons encode much richer information than previously appreciated. It suggests a possible strategy for the brain to extract rarely-occurring stimulus values, while concentrating neuronal resources on the most commonly-occurring situations.     

Stereoscopic Vision in the Absence of the Lateral Occipital Cortex

Both dorsal and ventral cortical visual streams contain neurons sensitive to binocular disparities, but the two streams may underlie different aspects of stereoscopic vision. Here we investigate stereopsis in the neurological patient D.F., whose ventral stream, specifically lateral occipital cortex, has been damaged bilaterally, causing profound visual form agnosia. Despite her severe damage to cortical visual areas, we report that DF''s stereo vision is strikingly unimpaired. She is better than many control observers at using binocular disparity to judge whether an isolated object appears near or far, and to resolve ambiguous structure-from-motion. DF is, however, poor at using relative disparity between features at different locations across the visual field. This may stem from a difficulty in identifying the surface boundaries where relative disparity is available. We suggest that the ventral processing stream may play a critical role in enabling healthy observers to extract fine depth information from relative disparities within one surface or between surfaces located in different parts of the visual field.     

Analysis and simulation of networks of mutually inhibiting neurons

Networks of mutually inhibiting neurons are analyzed and simulated on a digital computer. In the analysis and simulation a continuous-variable model of the neuron is used as the basic element. It consists of a many-input adder, a first-order low-pass filter and a diode-type nonlinearity. A mutually inhibiting network is formed by connecting the output of every element to inputs of the other elements through weight-coefficient setting units. Each element of the network is assumed to receive a certain number of constant inputs from elements of other networks.An autonomous system of nonlinear differential equations is introduced to describe the network dynamics, and the steady-state solutions of the system are investigated in detail. The network has a unique equilibrium solution, multiple equilibrium solutions or a periodic solution depending on the weight-coefficients and the inputs. It is shown that these three cases correspond to three types of information processing functions: the sharpening of input patterns, the temporary storage of information and the generation of periodic signals.     

Calibration of the parallax fluorescence quenching method for determination of membrane penetration depth: refinement and comparison of quenching by spin-labeled and brominated lipids.

We previously introduced the "parallax" method, which uses fluorescence quenching by spin-labeled lipids in order to measure the depth of molecules within a membrane [Chattopadhyay, A., & London, E. (1987) Biochemistry 26, 39-45]. In this report the accuracy of this method is established by comparison of spin-label quenching to that obtained using brominated lipids. To accomplish this, the fluorescent molecules used were a fatty acid labeled with a carbazole buried deeply within the acyl chain region of the membrane, an acyl-Trp with the Trp residue residing near the polar membrane region, and cytochrome b5, which has Trp residues in its membrane-inserted region. The depths calculated from the amount of bromine quenching agreed with those determined using parallax analysis. This indicates that the depth reported by parallax analysis is accurate and that the spin labels residue very close to their predicted locations in the membrane. Furthermore, there was good agreement when parallax analysis was applied both to quenching by brominated and spin-labeled molecules, suggesting that the analysis is valid in both cases. The effect that different distributions and motions of fluorophores and quenchers would have on parallax analysis was also examined. For uniform distributions of quenchers or fluorophores over a range of depths, it was found that the analysis reports the average fluorophore depth. In addition, experimental data suggest that motional effects do not significantly alter the measured depths. This is consistent with the motions during the short excited state lifetime of the fluorophores being relatively small and/or relatively isotropic.     

A Simple Rule for Dendritic Spine and Axonal Bouton Formation Can Account for Cortical Reorganization after Focal Retinal Lesions

Lasting alterations in sensory input trigger massive structural and functional adaptations in cortical networks. The principles governing these experience-dependent changes are, however, poorly understood. Here, we examine whether a simple rule based on the neurons'' need for homeostasis in electrical activity may serve as driving force for cortical reorganization. According to this rule, a neuron creates new spines and boutons when its level of electrical activity is below a homeostatic set-point and decreases the number of spines and boutons when its activity exceeds this set-point. In addition, neurons need a minimum level of activity to form spines and boutons. Spine and bouton formation depends solely on the neuron''s own activity level, and synapses are formed by merging spines and boutons independently of activity. Using a novel computational model, we show that this simple growth rule produces neuron and network changes as observed in the visual cortex after focal retinal lesions. In the model, as in the cortex, the turnover of dendritic spines was increased strongest in the center of the lesion projection zone, while axonal boutons displayed a marked overshoot followed by pruning. Moreover, the decrease in external input was compensated for by the formation of new horizontal connections, which caused a retinotopic remapping. Homeostatic regulation may provide a unifying framework for understanding cortical reorganization, including network repair in degenerative diseases or following focal stroke.