A statistical correlation technique and a neural network for the motion correspondence problem

A statistical correlation technique (SCT) and two variants of a neural network are presented to solve the motion correspondence problem. Solutions of the motion correspondence problem aim to maintain the identities of individuated elements as they move. In a pre-processing stage, two snapshots of a moving scene are convoluted with two-dimensional Gabor functions, which yields orientations and spatial frequencies of the snapshots at every position. In this paper these properties are used to extract, respectively, the attributes orientation, size and position of line segments. The SCT uses cross-correlations to find the correct translation components, angle of rotation and scaling factor. These parameters are then used in combination with the positions of the line segments to calculate the centre of motion. When all of these parameters are known, the new positions of the line segments from the first snapshot can be calculated and compared to the features in the second snapshot. This yields the solution of the motion correspondence problem. Since the SCT is an indirect way of solving the problem, the principles of the technique are implemented in interactive activation and competition neural networks. With boundary problems and noise these networks perform better than the SCT. They also have the advantage that at every stage of the calculations the best candidates for corresponding pairs of line segments are known.     

Stereoscopic vision: solving the correspondence problem

Neurons in early visual areas respond to horizontal disparity in images that do not give rise to stereopsis. False binocular matches, however, are discarded at the apex of the visual pathway: the activity of neurons in the primate inferior temporal cortex correlates directly with conscious depth perception.     

Characteristics of the flux dimension cue in apparent motion correspondence.

It has been shown that element flux and size (but not luminance) serve as correspondence cues in the apparent motion visual system. Results are now presented of a study of the characteristics of the flux cue. It was found that flux rather than luminance is used by the system even when the size of the elements is greater than the size limit of Ricco's law. There were interactions between the apparent motion processing of the size and flux dimensions, beyond the obvious dependence of flux on size: positively correlated size and flux differences between elements have a greater effect on correspondence than do negatively correlated differences. Finally, when comparing the fluxes of different elements, the apparent motion system uses relative flux (above or below background) rather than absolute flux (relative to zero).     

Cue combination and the effect of horizontal disparity and perspective on stereoacuity

Relative depth judgments of vertical lines based on horizontal disparity deteriorate enormously when the lines form part of closed configurations (Westheimer, 1979). In studies showing this effect, perspective was not manipulated and thus produced inconsistency between horizontal disparity and perspective. We show that stereoacuity improves dramatically when perspective and horizontal disparity are made consistent. Observers appear to use unhelpful perspective cues in judging the relative depth of the vertical sides of rectangles in a way not incompatible with a form of cue weighting. However, 95% confidence intervals for the weights derived for cues usually exceed the a-priori [0-1] range.     

Visual motion,binocular correspondence and binocular rivalry

The human pupillary control system has been the subject of interest to biologists and engineers as an example of a sensorimotor reflex which can be embedded in a control system paradigm. We present a nonlinear feedback model whose compact structure allows us to hypothesize possible physiological mechanisms which generate the proper behavior of the pupil system. The important pupil responses, including pupil size effect, asymmetry, and response to high-frequency stimuli, are defined. This model was simulated on a digital computer and comparisons to the paradigm experimental responses were performed, demonstrating a fit to each of the observed conditions. Improvements on previous models are discussed.     

The computational measurement of apparent motion: A recurrent pattern recognition strategy as an approach to solve the correspondence problem

In short, the model consists of a two-dimensional set of edge detecting units, modelled according to the zero-crossing detectors introduced first by Marr and Ullman (1981). These detectors are located peripherally in our synthetic vision system and are the input elements for an intelligent recurrent network. The purpose of that network is to recognize and categorize the previously detected contrast changes in a multi-resolution representation of the original image in such a manner that the original information will be decomposed into a relatively small numberN of well-defined edge primitives. The advantage of such a construction is that time-consuming pattern recognition has no longer to be done on the originally complex motion-blurred images of moving objects, but on a limited number of categorized forms. Based on a numberM of elementary feature attributes for each individual edge primitive, the model is then able to decompose each edge pattern into certain features. In this way anM-dimensional vector can be constructed for each edge. For each sequence of two successive frames a tensor can be calculated containing the distances (measured inM-dimensional feature space) between all features in both images. This procedure yields a set ofK—1 tensors for a sequence ofK images. After cross-correlation of allN ×M feature attributes from image (i) with those from image (i+1), wherei = 1, ...,K - 1, probability distributions can be computed. The final step is to search for maxima in these probability functions and then to construct from these extremes an optimal motion field. A number of simulation examples will be presented.     

Interocular motion combination for dichoptic moving stimuli

When gratings moving in different directions are presented separately to the two eyes, we typically perceive periods of the combination of motion in the two eyes as well as periods of one or the other monocular motions. To investigate whether such interocular motion combination is determined by the intersection-of-constraints (IOC) or vector average mechanism, we recorded both optokinetic nystagmus eye movements (OKN) and perception during dichoptic presentation of moving gratings and random-dot patterns with various differences of interocular motion direction. For moving gratings, OKN alternately tracks not only the direction of the two monocular motions but also the direction of their combined motion. The OKN in the combined motion direction is highly correlated with the perceived direction of combined motion; its velocity complies with the IOC rule rather than the vector average of the dichoptic motion stimuli. For moving random-dot patterns, both OKN and perceived motion alternate only between the directions of the two monocular motions. These results suggest that interocular motion combination in dichoptic gratings is determined by the IOC and depends on their form.     

Sensory convergence solves a motion ambiguity problem

Our inner ear is equipped with a set of linear accelerometers, the otolith organs, that sense the inertial accelerations experienced during self-motion. However, as Einstein pointed out nearly a century ago, this signal would by itself be insufficient to detect our real movement, because gravity, another form of linear acceleration, and self-motion are sensed identically by otolith afferents. To deal with this ambiguity, it was proposed that neural populations in the pons and midline cerebellum compute an independent, internal estimate of gravity using signals arising from the vestibular rotation sensors, the semicircular canals. This hypothesis, regarding a causal relationship between firing rates and postulated sensory contributions to inertial motion estimation, has been directly tested here by recording neural activities before and after inactivation of the semicircular canals. We show that, unlike cells in normal animals, the gravity component of neural responses was nearly absent in canal-inactivated animals. We conclude that, through integration of temporally matched, multimodal information, neurons derive the mathematical signals predicted by the equations describing the physics of the outside world.     

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.     

At least at the level of inferior temporal cortex,the stereo correspondence problem is solved

Stereoscopic vision requires the correspondence problem to be solved, i.e., discarding "false" matches between images of the two eyes, while keeping correct ones. To advance our understanding of the underlying neuronal mechanisms, we compared single neuron responses to correlated and anticorrelated random dot stereograms (RDSs). Inferior temporal neurons, which respond selectively to disparity-defined three-dimensional shapes, showed robust selectivity for correlated RDSs portraying concave or convex surfaces, but unlike neurons in areas V1, MT/V5, and MST, were not selective for anticorrelated RDSs. These results show that the correspondence problem is solved at least in far extrastriate cortex, as it is in the monkey's perception.     

End-stopping and the aperture problem: two-dimensional motion signals in macaque V1

Our perception of fine visual detail relies on small receptive fields at early stages of visual processing. However, small receptive fields tend to confound the orientation and velocity of moving edges, leading to ambiguous or inaccurate motion measurements (the aperture problem). Thus, it is often assumed that neurons in primary visual cortex (V1) carry only ambiguous motion information. Here we show that a subpopulation of V1 neurons is capable of signaling motion direction in a manner that is independent of contour orientation. Specifically, end-stopped V1 neurons obtain accurate motion measurements by responding only to the endpoints of long contours, a strategy which renders them largely immune to the aperture problem. Furthermore, the time course of end-stopping is similar to the time course of motion integration by MT neurons. These results suggest that cortical neurons might represent object motion by responding selectively to two-dimensional discontinuities in the visual scene.     

Homology and the origin of correspondence

Homology is a natural kind term and a precise account of what homologyis has to come out of theories about the role of homologues in evolution anddevelopment. Definitions of homology are discussed with respect to the questionas to whether they are able to give a non-circular account of thecorrespondenceor sameness referred to by homology. It is argued that standard accounts tiehomology to operational criteria or specific research projects, but are not yetable to offer a concept of homology that does not presuppose a version ofhomology or a comparable notion of sameness. This is the case for phylogeneticdefinitions that trace structures back to the common ancestor as well as fordevelopmental approaches such as Wagner's biological homology concept. Incontrast, molecular homology is able to offer a definition of homology in genesand proteins that explicates homology by reference to more basic notions.Molecular correspondence originates by means of specific features of causalprocesses. It is speculated that further understanding of morphogenesis mightenable biologists to give a theoretically deeper definition of homology alongsimilar lines: an account which makes reference to the concrete mechanisms thatoperate in organisms.     

Quality of Visual Cue Affects Visual Reweighting in Quiet Standing

Sensory reweighting is a characteristic of postural control functioning adopted to accommodate environmental changes. The use of mono or binocular cues induces visual reduction/increment of moving room influences on postural sway, suggesting a visual reweighting due to the quality of available sensory cues. Because in our previous study visual conditions were set before each trial, participants could adjust the weight of the different sensory systems in an anticipatory manner based upon the reduction in quality of the visual information. Nevertheless, in daily situations this adjustment is a dynamical process and occurs during ongoing movement. The purpose of this study was to examine the effect of visual transitions in the coupling between visual information and body sway in two different distances from the front wall of a moving room. Eleven young adults stood upright inside of a moving room in two distances (75 and 150 cm) wearing a liquid crystal lenses goggles, which allow individual lenses transition from opaque to transparent and vice-versa. Participants stood still during five minutes for each trial and the lenses status changed every one minute (no vision to binocular vision, no vision to monocular vision, binocular vision to monocular vision, and vice-versa). Results showed that farther distance and monocular vision reduced the effect of visual manipulation on postural sway. The effect of visual transition was condition dependent, with a stronger effect when transitions involved binocular vision than monocular vision. Based upon these results, we conclude that the increased distance from the front wall of the room reduced the effect of visual manipulation on postural sway and that sensory reweighting is stimulus quality dependent, with binocular vision producing a much stronger down/up-weighting than monocular vision.     

Pre-attentive segmentation and correspondence in stereo

Traditional stereo grouping models have focused on the problem of stereo correspondence between monocular inputs. Recent physiological data revealed that the disparity selective V2 cells increase their responses when (random-dot stereograms) stimuli within their receptive fields are at or near the boundary of a depth surface. Such highlights to depth (non-luminance) edges are seemingly not computationally required for the correspondence problem. Computationally, these highlights make the boundaries of a depth surface more salient, serving pre-attentive segmentation (between depth planes) and attracting visual attention. In special cases, they enable the psychophysically observed perceptual pop-out of a target from a background of visually identical distractors at a different depth. To achieve the highlights, mutual inhibition between disparity selective cells that are tuned to the same or similar depths is required. However, such mutual inhibition would impede the computation for the correspondence problem, which requires mutual excitation between the same cells. In this work, I introduce a computational model that, I believe, is the first to address both stereo correspondence and pre-attentive stereo segmentation. The computational mechanisms in the model are based on intracortical interactions in V2. I will demonstrate that the model captures the following physiological and psychophysical phenomena: (i) depth-edge highlighting; (ii) disparity capture; (iii) pop-out; and (iv) transparency.     

A combination of fuzzy set ordination with detrended correspondence analysis: One way to combine multi-environmental variables with vegetation data

This paper examines one possible way of Fuzzy Set Ordination by using multi-environmental variables. FSO's function is improved through combination with Detrended Correspondence Analysis which is used to summarize environmental information. It can be used to analyse the relationships between vegetation and environment no matter how many environmental variables are involved. An example with vegetation and environmental data collected from upland grasslands in Northern Snowdonia, Wales, is presented. Its results are consistent with that of CCA and DCCA.Abbreviations FSO Fuzzy set ordination - DCA Detrended correspondence analysis - CCA Canonical correspondence analysis - DCCA Detrended canonical correspondence analysis     

Notions of heredity in the correspondence of Edwin Grant Conklin

This article examines five letters from the correspondence of American zoologist Edwin Grant Conklin that highlight his theories of genetic and social inheritance, in order to suggest that Conklin's eugenic beliefs--like those of many American authorities during this time--were complex and sometimes contradictory. The letters reveal the international prestige of American science after the two world wars and illuminate key moments in the emergence of the concepts of heredity and inheritance, within both the science of genetics and the social movement of eugenics.     

Cue integration in categorical tasks: insights from audio-visual speech perception

Previous cue integration studies have examined continuous perceptual dimensions (e.g., size) and have shown that human cue integration is well described by a normative model in which cues are weighted in proportion to their sensory reliability, as estimated from single-cue performance. However, this normative model may not be applicable to categorical perceptual dimensions (e.g., phonemes). In tasks defined over categorical perceptual dimensions, optimal cue weights should depend not only on the sensory variance affecting the perception of each cue but also on the environmental variance inherent in each task-relevant category. Here, we present a computational and experimental investigation of cue integration in a categorical audio-visual (articulatory) speech perception task. Our results show that human performance during audio-visual phonemic labeling is qualitatively consistent with the behavior of a Bayes-optimal observer. Specifically, we show that the participants in our task are sensitive, on a trial-by-trial basis, to the sensory uncertainty associated with the auditory and visual cues, during phonemic categorization. In addition, we show that while sensory uncertainty is a significant factor in determining cue weights, it is not the only one and participants' performance is consistent with an optimal model in which environmental, within category variability also plays a role in determining cue weights. Furthermore, we show that in our task, the sensory variability affecting the visual modality during cue-combination is not well estimated from single-cue performance, but can be estimated from multi-cue performance. The findings and computational principles described here represent a principled first step towards characterizing the mechanisms underlying human cue integration in categorical tasks.