Visual space distortion

We are surrounded by surfaces that we perceive by visual means. Understanding the basic principles behind this perceptual process is a central theme in visual psychology, psychophysics, and computational vision. In many of the computational models employed in the past, it has been assumed that a metric representation of physical space can be derived by visual means. Psychophysical experiments, as well as computational considerations, can convince us that the perception of space and shape has a much more complicated nature, and that only a distorted version of actual, physical space can be computed. This paper develops a computational geometric model that explains why such distortion might take place. The basic idea is that, both in stereo and motion, we perceive the world from multiple views. Given the rigid transformation between the views and the properties of the image correspondence, the depth of the scene can be obtained. Even a slight error in the rigid transformation parameters causes distortion of the computed depth of the scene. The unified framework introduced here describes this distortion in computational terms. We characterize the space of distortions by its level sets, that is, we characterize the systematic distortion via a family of iso-distortion surfaces which describes the locus over which depths are distorted by some multiplicative factor. Given that humans' estimation of egomotion or estimation of the extrinsic parameters of the stereo apparatus is likely to be imprecise, the framework is used to explain a number of psychophysical experiments on the perception of depth from motion or stereo. Received: 9 January 1997 / Accepted in revised form: 8 July 1997     

Two-dimensional substructure of stereo and motion interactions in macaque visual cortex

The analysis of object motion and stereoscopic depth are important tasks that are begun at early stages of the primate visual system. Using sparse white noise, we mapped the receptive field substructure of motion and disparity interactions in neurons in V1 and MT of alert monkeys. Interactions in both regions revealed subunits similar in structure to V1 simple cells. For both motion and stereo, the scale and shape of the receptive field substructure could be predicted from conventional tuning for bars or dot-field stimuli, indicating that the small-scale interactions were repeated across the receptive fields. We also found neurons in V1 and in MT that were tuned to combinations of spatial and temporal binocular disparities, suggesting a possible neural substrate for the perceptual Pulfrich phenomenon. Our observations constrain computational and developmental models of motion-stereo integration.     

Object Recognition in Flight: How Do Bees Distinguish between 3D Shapes?

Honeybees (Apis mellifera) discriminate multiple object features such as colour, pattern and 2D shape, but it remains unknown whether and how bees recover three-dimensional shape. Here we show that bees can recognize objects by their three-dimensional form, whereby they employ an active strategy to uncover the depth profiles. We trained individual, free flying honeybees to collect sugar water from small three-dimensional objects made of styrofoam (sphere, cylinder, cuboids) or folded paper (convex, concave, planar) and found that bees can easily discriminate between these stimuli. We also tested possible strategies employed by the bees to uncover the depth profiles. For the card stimuli, we excluded overall shape and pictorial features (shading, texture gradients) as cues for discrimination. Lacking sufficient stereo vision, bees are known to use speed gradients in optic flow to detect edges; could the bees apply this strategy also to recover the fine details of a surface depth profile? Analysing the bees’ flight tracks in front of the stimuli revealed specific combinations of flight maneuvers (lateral translations in combination with yaw rotations), which are particularly suitable to extract depth cues from motion parallax. We modelled the generated optic flow and found characteristic patterns of angular displacement corresponding to the depth profiles of our stimuli: optic flow patterns from pure translations successfully recovered depth relations from the magnitude of angular displacements, additional rotation provided robust depth information based on the direction of the displacements; thus, the bees flight maneuvers may reflect an optimized visuo-motor strategy to extract depth structure from motion signals. The robustness and simplicity of this strategy offers an efficient solution for 3D-object-recognition without stereo vision, and could be employed by other flying insects, or mobile robots.     

Cortical connections and early visual function: intra- and inter-columnar processing.

While it is widely assumed that the long-range horizontal connections in V1 are present to support contour integration, there has been only limited consideration of other possible relationships between anatomy and physiology (the horizontal connections) and visual function beyond contour integration. We introduce the possibility of other relationships directly from the perspective of computation and differential geometry by identifying orientation columns in visual physiology with the (unit) tangent bundle in differential geometry. This suggests abstracting early vision in a space that incorporates both position and orientation, from which we show that the physiology is capable of supporting a number of functional computations beyond contour integration, including texture-flow and shading-flow integration, as well as certain relationships between them. The geometric abstraction emphasizes the role of curvature, which necessitates a coupled investigation into how it might be estimated. The result is an elaboration of layer-to-layer interactions within an orientation column, with non-linearities possibly implemented by shunting inhibition. Finally, we show how the same computational framework naturally lends itself to solving stereo correspondence, with binocular tangents abstracting curves in space.     

Illuminator: increasing synergies between medicinal and computational chemists

We present Illuminator, a user-friendly web front end to computational models such as docking and 3D shape similarity calculations. Illuminator was specifically created to allow non-experts to design and submit molecules to computational chemistry programs. As such it provides a simple user interface allowing users to submit jobs starting from a 2D structure. The models provided are pre-optimized by computational chemists for each specific target. We provide an example of how Illuminator was used to prioritize the design of molecular substituents in the Anadys HCV Polymerase (NS5B) project. With 7500 submitted jobs in 1.5 years, Illuminator has allowed project teams at Anadys to accelerate the optimization of novel leads. It has also improved communication between project members and increased demand for computational drug discovery tools.     

Stereo vision and isoluminance.

Recent experiments have shown that stereo depth is given by fusion of illusory ('cognitive') contours. They occur across quite large gaps in figures, when these gaps are unlikely and form the shape of a probable (nearer) masking object or masking feature. Implications are that: (a) clearly defined contours and regions of brightness difference can be produced as postulates from sensory evidence, which may be surprising absence of stimulation; (b) each eye-system can derive its own postulates, or hypotheses, which (c) can be combined to give stereo vision. It has been shown that random-dot stereo depth does not occur when there is colour contrast but no brightness difference between the dots and their background. This we have confirmed by using a new technique for producing isoluminant pictures, of any complexity, with exact registration for any two colours. With this technique, we find large displacements of narrow borders bounding regions that are shifted across isoluminance. These displacements, which are clearly seen as movements, occur with or without colour differences. The direction of shift depends on whether the narrow border is light or dark. It is found that these dramatic shifts do not - when produced in opposite directions to the two eyes and fused - produce stereo depth. It is concluded, following Julesz's paradigm, that these contour displacements have their neural orgin not retinally, but after stereo fusion. Experiments combining the 'cognitive contours' stereo depth with isoluminance are described.     

Pattern-dependent response modulations in motion-sensitive visual interneurons--a model study

Even if a stimulus pattern moves at a constant velocity across the receptive field of motion-sensitive neurons, such as lobula plate tangential cells (LPTCs) of flies, the response amplitude modulates over time. The amplitude of these response modulations is related to local pattern properties of the moving retinal image. On the one hand, pattern-dependent response modulations have previously been interpreted as 'pattern-noise', because they deteriorate the neuron's ability to provide unambiguous velocity information. On the other hand, these modulations might also provide the system with valuable information about the textural properties of the environment. We analyzed the influence of the size and shape of receptive fields by simulations of four versions of LPTC models consisting of arrays of elementary motion detectors of the correlation type (EMDs). These models have previously been suggested to account for many aspects of LPTC response properties. Pattern-dependent response modulations decrease with an increasing number of EMDs included in the receptive field of the LPTC models, since spatial changes within the visual field are smoothed out by the summation of spatially displaced EMD responses. This effect depends on the shape of the receptive field, being the more pronounced--for a given total size--the more elongated the receptive field is along the direction of motion. Large elongated receptive fields improve the quality of velocity signals. However, if motion signals need to be localized the velocity coding is only poor but the signal provides--potentially useful--local pattern information. These modelling results suggest that motion vision by correlation type movement detectors is subject to uncertainty: you cannot obtain both an unambiguous and a localized velocity signal from the output of a single cell. Hence, the size and shape of receptive fields of motion sensitive neurons should be matched to their potential computational task.     

Isotropic integration of binocular disparity and relative motion in the perception of three-dimensional shape

Richards (1985) showed that veridical three-dimensional shape may be recovered from the integration of binocular disparity and retinal motion information, but proposed that this integration may only occur for horizontal retinal motion. Psychophysical evidence supporting the combination of stereo and motion information is limited to the case of horizontal motion (Johnston et al., 1994), and has been criticised on the grounds of potential object boundary cues to shape present in the stimuli. We investigated whether veridical shape can be recovered under more general conditions. Observers viewed cylinders that were defined by binocular disparity, two-frame motion or a combination of disparity and motion, presented at simulated distances of 30 cm, 90 cm or 150 cm. Horizontally and vertically oriented cylinders were rotated about vertical and horizontal axes. When rotation was about the cylinder's own axis, no boundary cues to shape were introduced. Settings were biased for the disparity and two-frame motion stimuli, while more veridical shape judgements were made under all conditions for combined cue stimuli. These results demonstrate that the improved perception of three-dimensional shape in these stimuli is not a consequence of the presence of object boundary cues, and that the combination of disparity and motion is not restricted to horizontal image motion.     

Mathematical analysis and modeling of motion direction selectivity in the retina

Motion detection is one of the most important and primitive computations performed by our visual system. Specifically in the retina, ganglion cells producing motion direction-selective responses have been addressed by different disciplines, such as mathematics, neurophysiology and computational modeling, since the beginnings of vision science. Although a number of studies have analyzed theoretical and mathematical considerations for such responses, a clear picture of the underlying cellular mechanisms is only recently emerging. In general, motion direction selectivity is based on a non-linear asymmetric computation inside a receptive field differentiating cell responses between preferred and null direction stimuli. To what extent can biological findings match these considerations? In this review, we outline theoretical and mathematical studies of motion direction selectivity, aiming to map the properties of the models onto the neural circuitry and synaptic connectivity found in the retina. Additionally, we review several compartmental models that have tried to fill this gap. Finally, we discuss the remaining challenges that computational models will have to tackle in order to fully understand the retinal motion direction-selective circuitry.     

Speed encoding in correlation motion detectors as a consequence of spatial structure

For animals to carry out a wide range of detection, recognition and navigation tasks, visual motion signals are crucial. The encoding of motion information has therefore, attracted much attention in the experimental and computational study of brain function. Two main alternative mechanisms have been proposed on the basis of behavioural and physiological experiments. On one hand, correlation-type and motion energy detectors are simple and efficient in the design of their basic mechanism but are tuned to temporal frequency rather than to speed. On other hand, gradient-type motion detectors directly represent an estimate of speed, but may require more demanding processing mechanisms. We demonstrate here how the temporal frequency dependence observed for sine-wave gratings can disappear for less constrained stimuli, to be replaced by responses reflecting speed for stimuli like square waves when a phase-sensitive detection mechanism is employed. We conclude from these observations that temporal frequency tuning is not necessarily a limitation for motion vision based on correlation detectors, and more generally demonstrate in view of the typical Fourier composition of natural scenes, that correlation detectors operating in such environments can encode image speed. In the context of our results, we discuss the implications of the loss of phase sensitivity inherent in using a linear system approach to describe neural processing.     

Algorithms in nature: the convergence of systems biology and computational thinking

Computer science and biology have enjoyed a long and fruitful relationship for decades. Biologists rely on computational methods to analyze and integrate large data sets, while several computational methods were inspired by the high‐level design principles of biological systems. Recently, these two directions have been converging. In this review, we argue that thinking computationally about biological processes may lead to more accurate models, which in turn can be used to improve the design of algorithms. We discuss the similar mechanisms and requirements shared by computational and biological processes and then present several recent studies that apply this joint analysis strategy to problems related to coordination, network analysis, and tracking and vision. We also discuss additional biological processes that can be studied in a similar manner and link them to potential computational problems. With the rapid accumulation of data detailing the inner workings of biological systems, we expect this direction of coupling biological and computational studies to greatly expand in the future.     

Structure-from-motion based on information at surface boundaries

Existing computational models of structurefrom-motion — the appearance of three-dimensional motion generated by moving two-dimensional patterns — are all based on variations of optical flow or feature point correspondence within the interior of single objects. Three separate phenomena provide strong evidence that in human vision, structure-from-motion is significantly affected by surface boundary cues. In the first, a rotating cylinder is seen, though no variation in optical flow exists across the apparent cylinder. In the second, the shape of the bounding contour of a moving pattern dominates the actual differential motion within the pattern. In the third, the appearance of independently moving objects changes significantly when the boundary between them becomes indistinct. We describe a simple computational model sufficient to account for these effects. The model is based on qualitative constraints relating possible object motions to patterns of flow, together with an understanding of the patterns of flow that can be discriminated in practice.     

On the role of medial geometry in human vision.

A key challenge underlying theories of vision is how the spatially restricted, retinotopically represented feature analysis can be integrated to form abstract, coordinate-free object models. A resolution likely depends on the use of intermediate-level representations which can on the one hand be populated by local features and on the other hand be used as atomic units underlying the formation of, and interaction with, object hypotheses. The precise structure of this intermediate representation derives from the varied requirements of a range of visual tasks which motivate a significant role for incorporating a geometry of visual form. The need to integrate input from features capturing surface properties such as texture, shading, motion, color, etc., as well as from features capturing surface discontinuities such as silhouettes, T-junctions, etc., implies a geometry which captures both regional and boundary aspects. Curves, as a geometric model of boundaries, have been extensively used as an intermediate representation in computational, perceptual, and physiological studies, while the use of the medial axis (MA) has been popular mainly in computer vision as a geometric region-based model of the interior of closed boundaries. We extend the traditional model of the MA to represent images, where each MA segment represents a region of the image which we call a visual fragment. We present a unified theory of perceptual grouping and object recognition where through various sequences of transformations of the MA representation, visual fragments are grouped in various configurations to form object hypotheses, and are related to stored models. The mechanisms underlying both the computation and the transformation of the MA is a lateral wave propagation model. Recent psychophysical experiments depicting contrast sensitivity map peaks at the medial axes of stimuli, and experiments on perceptual filling-in, and brightness induction and modulation, are consistent with both the use of an MA representation and a propagation-based scheme. Also, recent neurophysiological recordings in V1 correlate with the MA hypothesis and a horizontal propagation scheme. This evidence supports a geometric computational paradigm for processing sensory data where both dynamic in-plane propagation and feedforward-feedback connections play an integral role.     

Force Transduction by the Microtubule-Bound Dam1 Ring

The coupling between the depolymerization of microtubules (MTs) and the motion of the Dam1 ring complex is now thought to play an important role in the generation of forces during mitosis. Our current understanding of this motion is based on a number of detailed computational models. Although these models realize possible mechanisms for force transduction, they can be extended by variation of any of a large number of poorly measured parameters and there is no clear strategy for determining how they might be distinguished experimentally. Here we seek to identify and analyze two distinct mechanisms present in the computational models. In the first, the splayed protofilaments at the end of the depolymerizing MT physically prevent the Dam1 ring from falling off the end, and in the other, an attractive binding secures the ring to the microtubule. Based on this analysis, we discuss how to distinguish between competing models that seek to explain how the Dam1 ring stays on the MT. We propose novel experimental approaches that could resolve these models for the first time, either by changing the diffusion constant of the Dam1 ring (e.g., by tethering a long polymer to it) or by using a time-varying load.     

Colour spaces in ecology and evolutionary biology

The recognition that animals sense the world in a different way than we do has unlocked important lines of research in ecology and evolutionary biology. In practice, the subjective study of natural stimuli has been permitted by perceptual spaces, which are graphical models of how stimuli are perceived by a given animal. Because colour vision is arguably the best‐known sensory modality in most animals, a diversity of colour spaces are now available to visual ecologists, ranging from generalist and basic models allowing rough but robust predictions on colour perception, to species‐specific, more complex models giving accurate but context‐dependent predictions. Selecting among these models is most often influenced by historical contingencies that have associated models to specific questions and organisms; however, these associations are not always optimal. The aim of this review is to provide visual ecologists with a critical perspective on how models of colour space are built, how well they perform and where their main limitations are with regard to their most frequent uses in ecology and evolutionary biology. We propose a classification of models based on their complexity, defined as whether and how they model the mechanisms of chromatic adaptation and receptor opponency, the nonlinear association between the stimulus and its perception, and whether or not models have been fitted to experimental data. Then, we review the effect of modelling these mechanisms on predictions of colour detection and discrimination, colour conspicuousness, colour diversity and diversification, and for comparing the perception of colour traits between distinct perceivers. While a few rules emerge (e.g. opponent log–linear models should be preferred when analysing very distinct colours), in general model parameters still have poorly known effects. Colour spaces have nonetheless permitted significant advances in ecology and evolutionary biology, and more progress is expected if ecologists compare results between models and perform behavioural experiments more routinely. Such an approach would further contribute to a better understanding of colour vision and its links to the behavioural ecology of animals. While visual ecology is essentially a transfer of knowledge from visual sciences to evolutionary ecology, we hope that the discipline will benefit both fields more evenly in the future.     

Characterization of Cell Boundary and Confocal Effects Improves Quantitative FRAP Analysis

Fluorescence recovery after photobleaching (FRAP) is an important tool used by cell biologists to study the diffusion and binding kinetics of vesicles, proteins, and other molecules in the cytoplasm, nucleus, or cell membrane. Although many FRAP models have been developed over the past decades, the influence of the complex boundaries of 3D cellular geometries on the recovery curves, in conjunction with regions of interest and optical effects (imaging, photobleaching, photoswitching, and scanning), has not been well studied. Here, we developed a 3D computational model of the FRAP process that incorporates particle diffusion, cell boundary effects, and the optical properties of the scanning confocal microscope, and validated this model using the tip-growing cells of Physcomitrella patens. We then show how these cell boundary and optical effects confound the interpretation of FRAP recovery curves, including the number of dynamic states of a given fluorophore, in a wide range of cellular geometries—both in two and three dimensions—namely nuclei, filopodia, and lamellipodia of mammalian cells, and in cell types such as the budding yeast, Saccharomyces pombe, and tip-growing plant cells. We explored the performance of existing analytical and algorithmic FRAP models in these various cellular geometries, and determined that the VCell VirtualFRAP tool provides the best accuracy to measure diffusion coefficients. Our computational model is not limited only to these cells types, but can easily be extended to other cellular geometries via the graphical Java-based application we also provide. This particle-based simulation—called the Digital Confocal Microscopy Suite or DCMS—can also perform fluorescence dynamics assays, such as number and brightness, fluorescence correlation spectroscopy, and raster image correlation spectroscopy, and could help shape the way these techniques are interpreted.     

In Silico Reconstitution of Listeria Propulsion Exhibits Nano-Saltation

To understand how the actin-polymerization-mediated movements in cells emerge from myriad individual protein–protein interactions, we developed a computational model of Listeria monocytogenes propulsion that explicitly simulates a large number of monomer-scale biochemical and mechanical interactions. The literature on actin networks and L. monocytogenes motility provides the foundation for a realistic mathematical/computer simulation, because most of the key rate constants governing actin network dynamics have been measured. We use a cluster of 80 Linux processors and our own suite of simulation and analysis software to characterize salient features of bacterial motion. Our “in silico reconstitution” produces qualitatively realistic bacterial motion with regard to speed and persistence of motion and actin tail morphology. The model also produces smaller scale emergent behavior; we demonstrate how the observed nano-saltatory motion of L. monocytogenes, in which runs punctuate pauses, can emerge from a cooperative binding and breaking of attachments between actin filaments and the bacterium. We describe our modeling methodology in detail, as it is likely to be useful for understanding any subcellular system in which the dynamics of many simple interactions lead to complex emergent behavior, e.g., lamellipodia and filopodia extension, cellular organization, and cytokinesis.     

A multi-differential neuromorphic approach to motion detection

This paper presents a multi-differential neuromorphic approach to motion detection. The model is based evidence for a differential operators interpretation of the properties of the cortical motion pathway. We discuss how this strategy, which provides a robust measure of speed for a range of types of image motion using a single computational mechanism, forms a useful framework in which to develop future neuromorphic motion systems. We also discuss both our approaches to developing computational motion models, and constraints in the design strategy for transferring motion models to other domains of early visual processing.