The representation of egomotion in the human brain

An essential function of visual processing is to establish the position of the body in space and, in concert with the other sense systems, to monitor movement of the whole body, or "egomotion." A key cue to egomotion is optic flow. For example, forward motion through the environment generates an expanding pattern of flow on the retina, and (with eyes fixed centrally) the direction of heading corresponds to the center of expansion [1]. In macaques, visual cortical area MST is sensitive to optic-flow structure [2, 3], and it has been suggested that MST has a central role in the computation of heading [4]. However, here we identify two areas of the human brain that represent visual cues to egomotion more directly than does MST. These areas respond strongly to a single optic-flow stimulus but become relatively unresponsive when the stimulus is surrounded with further flow patches and thereby made inconsistent with egomotion. One is putative area VIP in the anterior portion of the intraparietal sulcus. The other is a new visual area, which we refer to as cingulate sulcus visual area (CSv). Areas V1-V4 and MT respond about equally to both types of flow stimulus. MST has intermediate properties, responding well to multiple patches but with a modest preference for a single, egomotion-compatible patch. We suggest that MST is merely an intermediate processing stage for visual cues to egomotion and that such cues are more comprehensively encoded by VIP and CSv.     

Visual computation of egomotion using an image interpolation technique

A novel technique is presented for the computation of the parameters of egomotion of a mobile device, such as a robot or a mechanical arm, equipped with two visual sensors. Each sensor captures a panoramic view of the environment. We show that the parameters of egomotion can be computed by interpolating the position of the image captured by one of the sensors at the robot's present location, with respect to the images captured by the two sensors at the robot's previous location. The algorithm delivers the distance travelled and angle rotated, without the explicit measurement or integration of velocity fields. The result is obtained in a single step, without any iteration or successive approximation. Tests of the algorithm on real and synthetic images reveal an accuracy to within 5% of the actual motion. Implementation of the algorithm on a mobile robot reveals that stepwise rotation and translation can be measured to within 10% accuracy in a three-dimensional world of unknown structure. The position and orientation of the robot at the end of a 30-step trajectory can be estimated with accuracies of 5% and 5°, respectively.     

Egomotion and relative depth map from optical flow

When an observer moves in a 3D world, optical flow fields are generated on his retina. We argue that such an observer can in principle compute the parameters of his egomotion, and following this, the relative depth map of the stationary environment solely from the instantaneous positional velocity fields (IPVF). Moreover, we argue that in the stationary world, this analysis can be done locally, and is not dependent on global properties of the optical flow under the imposed constraints (smoothness of the egomotion path, rigidity of objects, temporal continuity of perception). To investigate the method, and to analyze its performance, a computer model has been constructed which simulates an observer moving through a 3D world of stationary rectangular planes at different depths and orientations. The results suggest that the method offers a reasonable and computationally feasible means of extracting information about egomotion and surface layout from optical flows, under certain circumstances. We discuss some issues related to extending the analysis to the case of a rigid world of moving objects, and some issues related to the status of information extractable from optical flows with respect to other sources of information.     

Egomotion estimation with optic flow and air velocity sensors

We develop a method that allows a flyer to estimate its own motion (egomotion), the wind velocity, ground slope, and flight height using only inputs from onboard optic flow and air velocity sensors. Our artificial algorithm demonstrates how it could be possible for flying insects to determine their absolute egomotion using their available sensors, namely their eyes and wind sensitive hairs and antennae. Although many behaviors can be performed by only knowing the direction of travel, behavioral experiments indicate that odor tracking insects are able to estimate the wind direction and control their absolute egomotion (i.e., groundspeed). The egomotion estimation method that we have developed, which we call the opto-aeronautic algorithm, is tested in a variety of wind and ground slope conditions using a video recorded flight of a moth tracking a pheromone plume. Over all test cases that we examined, the algorithm achieved a mean absolute error in height of 7% or less. Furthermore, our algorithm is suitable for the navigation of aerial vehicles in environments where signals from the Global Positioning System are unavailable.     

Perception matches selectivity in the human anterior color center

Human ventral cortex contains at least two visual areas selective for color [1]: a posterior center in the lingual gyrus labeled V4 [2-4], V8 [5], or VO-1 [6] and an anterior center in the medial fusiform that has been labeled V4alpha[3, 4]. We examined the properties of the anterior color center using electrical recording and electrical stimulation in a subject with an electrode implanted over the anterior color center, as determined with BOLD fMRI in the same subject. Presentation of visual stimuli evoked local field potentials from the electrode. Consistent with fMRI, the potentials were larger for chromatic than achromatic stimuli. The potentials differed depending on stimulus color, with blue-purple colors evoking the largest response. The spatial receptive field of the electrode was central/parafoveal with a contralateral bias. In the absence of a visual stimulus, electrical stimulation of the electrode produced an artificial visual percept of a blue-purple color near the center of gaze. These results provide direct evidence of a tight link between selectivity and perception in ventral temporal cortex. Electrical stimulation of the anterior color center is sufficient to produce the conscious percept of a color whose identity is determined by the selectivity of the stimulated neurons.     

Selectivity to Translational Egomotion in Human Brain Motion Areas

The optic flow generated when a person moves through the environment can be locally decomposed into several basic components, including radial, circular, translational and spiral motion. Since their analysis plays an important part in the visual perception and control of locomotion and posture it is likely that some brain regions in the primate dorsal visual pathway are specialized to distinguish among them. The aim of this study is to explore the sensitivity to different types of egomotion-compatible visual stimulations in the human motion-sensitive regions of the brain. Event-related fMRI experiments, 3D motion and wide-field stimulation, functional localizers and brain mapping methods were used to study the sensitivity of six distinct motion areas (V6, MT, MST+, V3A, CSv and an Intra-Parietal Sulcus motion [IPSmot] region) to different types of optic flow stimuli. Results show that only areas V6, MST+ and IPSmot are specialized in distinguishing among the various types of flow patterns, with a high response for the translational flow which was maximum in V6 and IPSmot and less marked in MST+. Given that during egomotion the translational optic flow conveys differential information about the near and far external objects, areas V6 and IPSmot likely process visual egomotion signals to extract information about the relative distance of objects with respect to the observer. Since area V6 is also involved in distinguishing object-motion from self-motion, it could provide information about location in space of moving and static objects during self-motion, particularly in a dynamically unstable environment.     

Visual space distortion

We are surrounded by surfaces that we perceive by visual means. Understanding the basic principles behind this perceptual process is a central theme in visual psychology, psychophysics, and computational vision. In many of the computational models employed in the past, it has been assumed that a metric representation of physical space can be derived by visual means. Psychophysical experiments, as well as computational considerations, can convince us that the perception of space and shape has a much more complicated nature, and that only a distorted version of actual, physical space can be computed. This paper develops a computational geometric model that explains why such distortion might take place. The basic idea is that, both in stereo and motion, we perceive the world from multiple views. Given the rigid transformation between the views and the properties of the image correspondence, the depth of the scene can be obtained. Even a slight error in the rigid transformation parameters causes distortion of the computed depth of the scene. The unified framework introduced here describes this distortion in computational terms. We characterize the space of distortions by its level sets, that is, we characterize the systematic distortion via a family of iso-distortion surfaces which describes the locus over which depths are distorted by some multiplicative factor. Given that humans' estimation of egomotion or estimation of the extrinsic parameters of the stereo apparatus is likely to be imprecise, the framework is used to explain a number of psychophysical experiments on the perception of depth from motion or stereo. Received: 9 January 1997 / Accepted in revised form: 8 July 1997     

Spatial mapping in the primate sensory projection: Analytic structure and relevance to perception

The retinotopic mapping of the visual field to the surface of the striate cortex is characterized as a longarithmic conformal mapping. This summarizes in a concise way the observed curve of cortical magnification, the linear scaling of receptive field size with eccentricity, and the mapping of global visual field landmarks. It is shown that if this global structure is reiterated at the local level, then the sequence regularity of the simple cells of area 17 may be accounted for as well. Recently published data on the secondary visual area, the medial visual area, and the inferior pulvinar of the owl monkey suggests that same global logarithmic structure holds for these areas as well. The available data on the structure of the somatotopic mapping (areaS-1) supports a similar analysis. The possible relevance of the analytical form of the cortical receptotopic maps to perception is examined and a brief discussion of the developmental implications of these findings is presented.This work was supported by Grant No. 1 F32MH05367-01 from the USPHS, ADAMHA     

Inhomogeneous retino-cortical mapping is supported and stabilized with correlation-learning during self-motion

In primates, the area of primary visual cortex representing a fixed area of visual space decreases with increasing eccentricity. We identify visual situations to which this inhomogeneous retino-cortical mapping is well adapted and study their relevance during natural vision and development. We assume that cortical activations caused by stationary objects during self-motion along the direction of gaze travel on average with constant speed across the cortical surface, independent of retinal eccentricity. This is the case if the distribution of objects corresponds to an ellipsoid with the observer in its center. We apply the resulting flow field to train a simple network of pulse coding neurons with Hebbian learning and demonstrate that the density of learned receptive field centers is in close agreement with primate retino-cortical magnification. In addition, the model reproduces the increase of receptive field size and the decrease of its peak sensitivity with increasing eccentricity. Our results suggest that self-motion may have played an important role in the evolution of the visual system and that cortical magnification can be refined and stabilized by Hebbian learning mechanisms in ontogenesis under natural viewing conditions.     

Fovea-Periphery Axis Symmetry of Surround Modulation in the Human Visual System

A visual stimulus activates different sized cortical area depending on eccentricity of the stimulus. Here, our aim is to understand whether the visual field size of a stimulus or cortical size of the corresponding representation determines how strongly it interacts with other stimuli. We measured surround modulation of blood-oxygenation-level-dependent signal and perceived contrast with surrounds that extended either towards the periphery or the fovea from a center stimulus, centered at 6° eccentricity. This design compares the effects of two surrounds which are identical in visual field size, but differ in the sizes of their cortical representations. The surrounds produced equally strong suppression, which suggests that visual field size of the surround determines suppression strength. A modeled population of neuronal responses, in which all the parameters were experimentally fixed, captured the pattern of results both in psychophysics and functional magnetic resonance imaging. Although the fovea-periphery anisotropy affects nearly all aspects of spatial vision, our results suggest that in surround modulation the visual system compensates for it.     

An image-interpolation technique for the computation of optic flow and egomotion

 A technique for measuring the motion of a rigid, textured plane in the frontoparallel plane is developed and tested on synthetic and real image sequences. The parameters of motion – translation in two dimensions, and rotation about a previously unspecified axis perpendicular to the plane – are computed by a single-stage, non-iterative process which interpolates the position of the moving image with respect to a set of reference images. The method can be extended to measure additional parameters of motion, such as expansion or shear. Advantages of the technique are that it does not require tracking of features, measurement of local image velocities or computation of high-order spatial or temporal derivatives of the image. The technique is robust to noise, and it offers a simple, novel way of tackling the ‘aperture’ problem. An application to the computation of robot egomotion is also described. Received: 3 September 1993/Accepted in revised form: 16 April 1994     

Perceived surface slant is systematically biased in the actively-generated optic flow

Humans make systematic errors in the 3D interpretation of the optic flow in both passive and active vision. These systematic distortions can be predicted by a biologically-inspired model which disregards self-motion information resulting from head movements (Caudek, Fantoni, & Domini 2011). Here, we tested two predictions of this model: (1) A plane that is stationary in an earth-fixed reference frame will be perceived as changing its slant if the movement of the observer's head causes a variation of the optic flow; (2) a surface that rotates in an earth-fixed reference frame will be perceived to be stationary, if the surface rotation is appropriately yoked to the head movement so as to generate a variation of the surface slant but not of the optic flow. Both predictions were corroborated by two experiments in which observers judged the perceived slant of a random-dot planar surface during egomotion. We found qualitatively similar biases for monocular and binocular viewing of the simulated surfaces, although, in principle, the simultaneous presence of disparity and motion cues allows for a veridical recovery of surface slant.     

Saccadic facilitation in natural backgrounds

In visual systems with a fovea, only a small portion of the visual field can be analyzed with high accuracy. Saccadic eye movements shift that center of gaze around several times a second. Saccades have been characterized in great detail and depend critically on a number of visual properties of the stimuli. However, typical experiments have used bright spots on dark backgrounds, while our natural environment has a highly characteristic rich spatial structure. Here we show that the saccadic system, unlike the perceptual system, is able to compensate for the masking caused by structured backgrounds. Consequently, saccadic latencies in the context of natural backgrounds are much faster than unstructured backgrounds at equal levels of visibility. The results suggest that whenever a structured background acts to mask the visibility of the saccade target, it simultaneously preactivates saccadic circuitry and thus ensures a fast reaction to potentially critical stimuli that are difficult to detect in our environment.     

Coherence in nervous system design: the visual system of Pantodon buchholzi

One of the more unusual visual systems of the Actinopterygii is that of Pantodon buchholzi (Osteoglossomorpha: Osteoglossidae). Its adaptations associate neuroanatomy at different levels of the visual system with ecological and behavioural correlates and demonstrate that the visual system of this fish has adapted for simultaneous vision in air and water. The visual field is divided into three distinct areas: for viewing into the water column, into air, and for viewing the aquatic reflection from the underside of the water surface. Cone diameters in different retinal areas correlate with the differing physical constraints in the respective visual field. Retinal differentiation between the aquatic and aerial views is paralleled at different levels of the central nervous system. A diencephalic nucleus receives both direct and indirect (tectal) afferent input from only the aerial visual system and a specific type of cell in the optic tectum is preferentially distributed in the tectum processing aerial inputs. Distinctions within a single sensory system suggest that some behaviours may be organized according to visual field. For Pantodon, feeding is initiated by stimuli seen by the ventral hemiretina so the anatomical specializations may well play an important role as elements in a feeding circuit.     

宽周边视视网膜皮层映射的核磁共振研究

目的:人类视觉皮层的组织方式是视网膜皮层映射组织,先前研究已经证实视觉皮层在中心视采用这种组织方式,本文主要研究宽周边视的视觉皮层组织方式.方法:本文采用一种可以在核磁共振室中使用的光纤设备,设计了30度、40度、50度、60度的类圆环block刺激,使用1.5T的功能性核磁共振仪器,T1高分辨率图像分辨率为1*1*5.5mm,T2加权图像分辨率为4*4*5.5mm,TR反应时间为60,矩阵大小为64*64.核磁共振数据分析使用了SPM2和Brain voyager软件.结果:通过对试验者的数据处理分析,周边视的刺激的反应区域在枕叶上,主要分布在枕叶的前部,刺激反应区域随着偏心率的增大而沿着距状沟从距状沟的后部向前部移动.结论:周边视的视网膜皮层映射组织特性和中心视的特性非常相似.     

A 'first stage' central performance drop in a Gabor luminance-modulation detection task

In an experiment, 20 participants had to detect a backward masked Gabor luminance-modulation target imposed on a field of uniform luminance at varying eccentricities along the horizontal meridian. Different spatial frequencies were used as target modulations. Results for a 7.0 c/deg target patch showed peak detection performance at the center of the visual field and a steady decrease toward the periphery. For 1.0 c/deg, 0.75 c/deg, and 0.5 c/deg target patches, in contrast, the peak was several degrees off retinal center and decreased steadily toward the center. Findings not only confirmed the familiar sensitivity loss toward peripheral areas for high spatial frequencies, but also indicated a sensitivity loss toward central areas for low spatial frequencies. It is concluded that they further support Gurnsey et al.'s (1996) 'mismatch hypothesis' extending its scope to also include 'first-stage' stimuli.     

Facts on optic flow

We employ an optimal solution to both the shape from motion problem and the related problem of the estimation of self-movement on a purely optical basis to deduce practical rules of thumb for the limits of the optic flow information content in the presence of perturbation of the motion parallax field. The results are illustrated and verified by means of a computer simulation.The results allow estimates of the accuracy of depth and egomotion estimates as a function of the accuracy of data sampling and the width of field of view, as well as estimates of the interaction between rotational and translational components of the movement.     

Stimulus size selectivity and receptive field organization of ectostriatal neurons in the pigeon

The avian ectostriatum is the telencephalic recipient zone of the tectofugal pathway, which may be homologous to the colliculo-pulvinar-cortical pathway in mammals. The present paper studies the visual response properties and receptive field organization of ectostriatal cells in pigeons with extracellular recording and computer mapping techniques. The results indicate that 90% of ectostriatal cells prefer forward, downward and upward motion of visual stimuli at velocities in the range of 16-128 degrees s(-1). They respond optimally to small stimuli (1-4 degrees visual angle), mostly preferring a target of 2 degrees. Most cells (78.8%) have one excitatory receptive field that usually possesses one or two hotspots, some cells (13.5%) have two excitatory receptive fields each having one or two hotspots, and a few cells (7.7%) have one excitatory receptive field with an unresponsive region in the center. An inhibitory receptive field is not found surrounding the excitatory receptive field in the ectostriatal cells examined. These response properties and receptive field organization may reflect the possible roles of the ectostriatum in stimulus discrimination and visual cognition.     

Diffusion model of tumor vascularization and growth

A diffusion model of tumor growth, vascularization and necrosis is used to analyze experimental data describing the temporal changes in tumor cell and blood vessel radial distributions in a host-tissue field transplanted with a fibrosarcoma. The experimental results showed a peak density of vessels occurring at the advancing migration front of the tumor and a decline in the vessel surface area at the tumor center with time. The peak density of tumor cells shifts away from the tumor center with time. These dynamic changes can be explained by a mathematical model which views the process as one of diffusion and proliferation in time and space. Coupled diffusion equations with nonlinear source and sink terms describe the proliferation, death, and migration of tumor cells and vascular surface area. The concept of an angiogenic factor elaborated by tumor cells is incorporated.     

Mechanism of directional sensitivity of receptive fields in the cat visual cortex

Responses of directional-sensitive neurons in area 17 of the cat's cortex were studied to presentation of two flashing bars of light, one located in the center of the field, the other in the inhibitory zone relative to the center. The order of activation of the stimuli and the time interval between them could be varied; presentation of two bars was thus the analog of a moving stimulus. The inhibitory off-zone located at the entrance to the field from the side of the optimal direction of movement was found to have an initial inhibitory phase, followed by a phase of disinhibition, and again by a second inhibitory phase. Presentation of the bars with different time intervals in cases when stimulation of the center of the field coincided in time with one of the inhibitory phases, led to inhibition of the response, but if stimulation coincided with the phase of disinhibition, it led to facilitation. Phases of disinhibition were not found in the inhibitory zone located at the entrance to the field along the course of a nonoptimal direction of movement. The importance of the temporal characteristics of inhibitory zones for the appearance of directional sensitivity of visual courtical neurons is discussed.