Massively parallel implementations of theories for apparent motion

Two solutions for the correspondence problem for long-range motion are investigated. The first is a modification of the Minimal Mapping Theory (S. Ullman: The Interpretation of Visual Motion, MIT Press, Cambridge, 1979) that is implemented by a massively parallel network. In this network, every two units are interconnected, and thus, its convergence is fast and relatively independent of the number of image features. Computer simulations show that our method accounts as well as the Minimal Mapping Theory for apparent-motion phenomena, although some differences exist. Mathematical proofs provide conditions for the convergence of the network. The second 'solution' for the correspondence problem is called the Structural Theory. This theory assumes that the three-dimensional structure of viewed objects does not change fast in time. Then, the theory looks for the correspondence and three-dimensional structure that best fulfill this assumption. A massively parallel network implementation of this theory is also possible. However, its performance is poor due to the high complexity of its solution space. This supports Ullman's (1979) suggestion that the visual system separates the structure-from-motion process into two stages. First, a stage for motion measurement, and then a stage for structure recovery.     

Biological computation of image motion from flows over boundaries.

A theory of early motion processing in the human and primate visual system is presented which is based on the idea that spatio-temporal retinal image data is represented in primary visual cortex by a truncated 3D Taylor expansion that we refer to as a jet vector. This representation allows all the concepts of differential geometry to be applied to the analysis of visual information processing. We show in particular how the generalised Stokes theorem can be used to move from the calculation of derivatives of image brightness at a point to the calculation of image brightness differences on the boundary of a volume in space-time and how this can be generalised to apply to integrals of products of derivatives. We also provide novel interpretations of the roles of direction selective, bi-directional and pan-directional cells and of type I and type II cells in V5/MT.     

Stereo disparity computation using Gabor filters

A solution to the correspondence problem for stereopsis is proposed using the differences in the complex phase of local spatial frequency components. One-dimensional spatial Gabor filters (Gabor 1946; Marcelja 1980), at different positions and spatial frequencies are convolved with each member of a stereo pair. The difference between the complex phase at corresponding points in the two images is used to find the stereo disparity. Disparity values are combined across spatial frequencies for each image location. Three-dimensional depth maps have been computed from real images under standard lighting conditions, as well as from random-dot stereograms (Julesz 1971). The algorithm can discriminate disparities significantly smaller than the width of a pixel. It is possible that a similar mechanism might be used in the human visual system.     

A two dimensional field theory for motion computation

The local extraction of motion information from brightness patterns by individual movement detectors of the correlation-type is considered in the first part of the paper. A two-dimensional field theory of movement detection is developed by treating the distance between two adjacent photoreceptors as a differential. In the first approximation of the theory we only consider linear terms of the time interval between the reception of a contrast element and its delayed representation by the detector and linear terms of the spatial distances between adjacent photoreceptors. As a result we may neglect terms of higher order than quadratic in a Taylor series development of the brightness pattern. The responses of pairs of individual movement detectors are combined to a local response vector. In the first approximation of the detector field theory the response vector is proportional to the instantaneous pattern velocity vector and linearly dependent on local properties of the moving pattern. The linear dependence on pattern properties is represented by a two by two tensor consisting of elements which are nonlinear, local functional of the moving pattern. Some of the properties of the tensor elements are treated in detail. So, for instance, it is shown that the off-diagonal elements of the tensor disappear when the moving pattern consists of x- and y-dependent separable components. In the second part of the paper the tensor relation leading to the output of a movement detector pair is spatially integrated. The result of the integration is an approximation to a summation of the outputs of an array of detector pairs. The spatially integrated detector tensor relates the translatory motion vector to the resultant output vector. It is shown that the angle between the motion vector and the resultant output vector is always smaller than ±90° whereas the angle between the motion vector and local response vectors, elicited by detector pairs, may cover the entire angular range. In the discussion of the paper the limits of the field theory for motion computation as well as its higher approximations are pointed out in some detail. In a special chapter the dependence of the detector response on the pattern properties is treated and in another chapter questions connected with the so called aperture problem are discussed. Furthermore, properties for compensation of the pattern dependent deviation angle by spatial physiological integration are mentioned in the discussion.     

Efficient computation of Hessian-based enhancement filters for tubular structures in 3D images

This work presents guidelines for a computationally efficient implementation of multiscale image filters based on eigenanalysis of the Hessian matrix, for the enhancement of tubular structures. Our focus is the application to 3D medical images of blood vessels. The method uses matrix trace, determinant and sign to discard voxels unlikely to belong to vessels, prior to the calculation of the Hessian eigenvalues. As example of time savings, we provide results obtained in four computed tomography datasets (300 × 300 × 300 voxels) containing coronary and pulmonary arteries. The test based on the Hessian trace avoided the computation of the eigenvalues in half of the voxels on average, while the test combining the Hessian determinant and sign eliminated up to 10% additional voxels. The actual time savings depend on the algorithm used to compute the eigenvalues for the remaining voxels. With a very fast algorithm using a closed-form solution, the computational time was reduced from 20.5 to 12.5 seconds per scale, but the time gained thanks to the more complex of the two tests was negligible. However, this fast algorithm is prone to numerical instabilities. Accurate computation of the eigenvalues requires the use of iterative or hybrid algorithms. In this case, both tests produce time savings and the computational time can be reduced by several minutes per scale.     

Fly vision: neural mechanisms of motion computation

A wide range of novel approaches are being used to dissect the visual system of the fly, both the neural networks of motion detection and the performance of these networks under complex natural stimulus conditions.     

The temporal and spatial distribution of nematodes in percolating filters

The spatial and temporal distribution of nematodes in two low rate percolating filter bed sewage treatment plants at Carnforth and Caton, Lancashire, were investigated. Nematodes were abundant reaching densities up to 900 cm⁻² of media surface. Numbers decreased down the bed and species composition displayed spatial differences with bed depth. Eight orders of nematode were represented, members of the sub-family Diplogasterinae and Rhabditinae were particularly common. A seasonal pattern of abundance was clearly apparent; two peaks occurred, one in May and a second during the winter months. Lowest numbers corresponded with periods of highest temperature. The demise of the nematodes in the spring was related to the sloughing of the surface film on the media.     

Generalized gradient schemes for the measurement of two-dimensional image motion

This paper describes a procedure for recovering the global velocity of an image by incorporating spatial filtering, and, optionally, temporal filtering, into a scheme that employs a generalized version of the gradient algorithm of motion detection. Motion within a patch is analysed by six parallel channels, each incorporating a different spatiotemporal filter. Advantageous features of this scheme are: (a) global velocity is derived directly, without first computing local velocity at a number of image locations; (b) the filters compute first derivatives rather than second derivatives, making the scheme potentially more resistant to noise than certain other schemes; (c) two of the six filters can be chosen almost completely arbitrarily, and can therefore be tailored to maximize signal reliability, and (d) the measurement of velocity can be made as local or as global as desired by altering the size of the patch that is viewed by the filters. An analogous scheme is derived for the measurement of rotation, as well as expansion or contraction of the image.     

The computation of structure from fixed-axis motion: rigid structures

We show that three distinct orthographic views of three points in a rigid configuration are compatibel with at most 64 interpretations of the three-dimensional structure and motion of the points. If, in addition, one assumes that the three points spin about a fixed axis over the three views, then with probability one there is a unique three-dimensional interpretation (plus a reflection). Moreover the probability of false targets is zero. In the special case that the axis of rotation is parallel to the image plane three views of the three points are sufficient to obtain at most two interpretations (plus reflections)-unless one assumes the angular velocity about the axis is constant, in which case three views of two points are sufficient to determine a unique interpretation. Closed form solutions are obtained for each of these cases. The systems of equations studied here are in each case overconstraining (i.e. there are more independent equations than unknowns) and are amenable to solution by nonlinear programming. These two properties make possible the construction of noise insensitive algorithms for computer vision systems. Our uniqueness proofs employ the Principle of upper semicontinuity, a principle which underlies a general mathematical framework for the analysis of solutions to overconstraining systems of equations.     

Visual computation of egomotion using an image interpolation technique

A novel technique is presented for the computation of the parameters of egomotion of a mobile device, such as a robot or a mechanical arm, equipped with two visual sensors. Each sensor captures a panoramic view of the environment. We show that the parameters of egomotion can be computed by interpolating the position of the image captured by one of the sensors at the robot's present location, with respect to the images captured by the two sensors at the robot's previous location. The algorithm delivers the distance travelled and angle rotated, without the explicit measurement or integration of velocity fields. The result is obtained in a single step, without any iteration or successive approximation. Tests of the algorithm on real and synthetic images reveal an accuracy to within 5% of the actual motion. Implementation of the algorithm on a mobile robot reveals that stepwise rotation and translation can be measured to within 10% accuracy in a three-dimensional world of unknown structure. The position and orientation of the robot at the end of a 30-step trajectory can be estimated with accuracies of 5% and 5°, respectively.     

The computation of structure from fixed-axis motion: Nonrigid structures

We show that an assumption of rigidity or quasi-rigidity is not necessary, in principle, for the computation of three-dimensional structure and motion from changing retinal images. In particular, we show that the three-dimensional structure of certain nonrigid objects, namely objects whose texture elements rotate about a common axis but at varying angular velocities, can in principle be computed from three successive retinal images of four texture elements, or from four successive images of two texture elements. We then show that in both cases the computed structure matches the actual structure of the object with probability one.     

Role of ACh-GABA cotransmission in detecting image motion and motion direction

Starburst amacrine cells (SACs) process complex visual signals in the retina using both acetylcholine (ACh) and gamma-aminobutyric acid (GABA), but the synaptic organization and function of ACh-GABA corelease remain unclear. Here, we show that SACs make cholinergic synapses onto On-Off direction-selective ganglion cells (DSGCs) from all directions but make GABAergic synapses onto DSGCs only from the null direction. ACh and GABA were released differentially in a Ca(2+) level-specific manner, suggesting the two transmitters were released from different vesicle populations. Despite the symmetric cholinergic connection, the light-evoked cholinergic input to a DSGC, detected at both light onset and offset, was motion- and direction-sensitive. This input was facilitated by two-spot apparent motion in the preferred direction but supressed in the null direction, presumably by a GABAergic mechanism. The results revealed a high level of synaptic intricacy in the starburst circuit and suggested differential, yet synergistic, roles of ACh-GABA cotransmission in motion sensitivity and direction selectivity.     

Auditory temporal computation: interval selectivity based on post-inhibitory rebound

The measurement of time is fundamental to the perception of complex, temporally structured acoustic signals such as speech and music, yet the mechanisms of temporal sensitivity in the auditory system remain largely unknown. Recently, temporal feature detectors have been discovered in several vertebrate auditory systems. For example, midbrain neurons in the fish Pollimyrus are activated by specific rhythms contained in the simple sounds they use for communication. This poses the significant challenge of uncovering the neuro-computational mechanisms that underlie temporal feature detection. Here we describe a model network that responds selectively to temporal features of communication sounds, yielding temporal selectivity in output neurons that matches the selectivity functions found in the auditory system of Pollimyrus. The output of the network depends upon the timing of excitatory and inhibitory input and post-inhibitory rebound excitation. Interval tuning is achieved in a behaviorally relevant range (10 to 40 ms) using a biologically constrained model, providing a simple mechanism that is suitable for the neural extraction of the relatively long duration temporal cues (i.e. tens to hundreds of ms) that are important in animal communication and human speech.     

Robust velocity computation from a biologically motivated model of motion perception

Current computational models of motion processing in the primate motion pathway do not cope well with image sequences in which a moving pattern is superimposed upon a static texture. The use of non-linear operations and the need for contrast normalization in motion models mean that the separation of the influences of moving and static patterns on the motion computation is not trivial. Therefore, the response to the superposition of static and moving patterns provides an important means of testing various computational strategies. Here we describe a computational model of motion processing in the visual cortex, one of the advantages of which is that it is highly resistant to interference from static patterns.     

The inferior temporal cortex: Architecture, computation, and representation

Neurons in the inferior temporal cortex (IT), an area crucially involved in visual object recognition in monkeys, show the visual response properties and anatomical/chemical nature which are distinct from those in the cortical areas that feed visual inputs to the IT. Earlier physiological studies showed that IT neurons have large receptive fields covering the center and contralateral (often bilateral) visual fields, stimulus selectivity for images of complex objects or shapes, and translation invariance of the stimulus selectivity. Recent studies have revealed new aspects of their properties such as invariant selectivity for shapes despite drastic changes in various physical attributes of stimuli, latent excitatory inputs masked by stimulus-specific GABAergic inhibition, selectivity for binocular disparity and 3-dimensional surface structures, profound effects of learning on the stimulus selectivity, and columnar clustering of neurons with similarstimulus selectivity for shapes and other object features. Another line of research using histological techniques have revealed that pyramidal neurons in the IT are larger in the size of dendritic arbors, in the number of dendritic branches and spines, and in the size and distribution of horizontal axonal arbors than those in the earlier areas, allowing them to integrate a larger population of afferents and process more diverse inputs. The concentration of several neurochemicals including those related to synaptic transmission or plasticity changes systematically towards the IT along the occipitotemporal pathway. Many of the characteristics of IT neurons parallel or explain certain aspects of visual object perception, although the behavioral relevance has yet to be addressed experimentally.     

Ultrasonic motion analysis system--measurement of temporal and spatial gait parameters

The duration of stance and swing phase and step and stride length are important parameters in human gait. In this technical note a low-cost ultrasonic motion analysis system is described that is capable of measuring these temporal and spatial parameters while subjects walk on the floor. By using the propagation delay of sound when transmitted in air, this system is able to record the position of the subjects' feet. A small ultrasonic receiver is attached to both shoes of the subject while a transmitter is placed stationary on the floor. Four healthy subjects were used to test the device. Subtracting positions of the foot with zero velocity yielded step and stride length. The duration of stance and swing phase was calculated from heel-strike and toe-off. Comparison with data obtained from foot contact switches showed that applying two relative thresholds to the speed graph of the foot could reliably generate heel-strike and toe-off. Although the device is tested on healthy subjects in this study, it promises to be extremely valuable in examining pathological gait. When gait is asymmetrical, walking speed is not constant or when patients do not completely lift their feet, most existing devices will fail to correctly assess the proper gait parameters. Our device does not have this shortcoming and it will accurately demonstrate asymmetries and variations in the patient's gait. As an example, the recording of a left hemiplegic patient is presented in the discussion.     

Flexible filtering by neural inputs supports motion computation across states and stimuli

1. Download : Download high-res image (188KB)
2. Download : Download full-size image