Tracking facilitates 3-D motion estimation

The recently emerging paradigm of Active Vision advocates studying visual problems in form of modules that are directly related to a visual task for observers that are active. Along these lines, we are arguing that in many cases when an object is moving in an unrestricted manner (translation and rotation) in the 3D world, we are just interested in the motion's translational components. For a monocular observer, using only the normal flow — the spatio-temporal derivatives of the image intensity function — we solve the problem of computing the direction of translation and the time to collision. We do not use optical flow since its computation is an ill-posed problem, and in the general case it is not the same as the motion field — the projection of 3D motion on the image plane. The basic idea of our motion parameter estimation strategy lies in the employment of fixation and tracking. Fixation simplifies much of the computation by placing the object at the center of the visual field, and the main advantage of tracking is the accumulation of information over time. We show how tracking is accomplished using normal flow measurements and use it for two different tasks in the solution process. First it serves as a tool to compensate for the lack of existence of an optical flow field and thus to estimate the translation parallel to the image plane; and second it gathers information about the motion component perpendicular to the image plane.     

Computational aspects of motion perception in natural and artificial vision systems.

In this paper a computational scheme for motion perception in artificial and natural vision systems is described. The scheme is motivated by a mathematical analysis in which first-order spatial properties of optical flow, such as singular points and elementary components of optical flow, are shown to be salient features for the computation and analysis of visual motion. The fact that different methods for the computation of optical flow produce similar results is explained in terms of the simple spatial structure of the image motion of rigid bodies. Singular points and elementary flow components are used to compute motion parameters, such as time-to-collision and angular velocity, and also to segment the visual field into areas which correspond to different motions. Then a number of biological implications are discussed. Electrophysiological findings suggest that the brain perceives visual motion by detecting and analysing optical flow components. However, the cortical neurons, which seem to detect elementary flow components, are not able to extract these components from more complex flows. A simple model for the organization of the receptive field of these cells, which is consistent with anatomical and electrophysiological data, is described at the end of the paper.     

An image-interpolation technique for the computation of optic flow and egomotion

 A technique for measuring the motion of a rigid, textured plane in the frontoparallel plane is developed and tested on synthetic and real image sequences. The parameters of motion – translation in two dimensions, and rotation about a previously unspecified axis perpendicular to the plane – are computed by a single-stage, non-iterative process which interpolates the position of the moving image with respect to a set of reference images. The method can be extended to measure additional parameters of motion, such as expansion or shear. Advantages of the technique are that it does not require tracking of features, measurement of local image velocities or computation of high-order spatial or temporal derivatives of the image. The technique is robust to noise, and it offers a simple, novel way of tackling the ‘aperture’ problem. An application to the computation of robot egomotion is also described. Received: 3 September 1993/Accepted in revised form: 16 April 1994     

Active vision in insects: an analysis of object-directed zig-zag flights in wasps (Odynerus spinipes , Eumenidae)

Ground-nesting wasps (Odynerus spinipes, Eumenidae) perform characteristic zig-zag flight manoeuvres when they encounter a novel object in the vicinity of their nests. We analysed flight parameters and flight control mechanisms and reconstructed the optical flow fields which the wasps generate by these flight manoeuvres. During zig-zag flights, the wasps move sideways and turn to keep the object in their frontal visual field. Their turning speed is controlled by the relative motion between object and background. We find that the wasps adjust their rotational and translational speed in such a way as to produce a specific vortex field of image motion that is centred on the novel object. As a result, differential image motion and changes in the direction of motion vectors are maximal in the vicinity and at the edges of the object. Zig-zag flights thus seem to be a `depth from motion' procedure for the extraction of object-related depth information. Accepted: 31 August 1997     

Structure-from-motion based on information at surface boundaries

Existing computational models of structurefrom-motion — the appearance of three-dimensional motion generated by moving two-dimensional patterns — are all based on variations of optical flow or feature point correspondence within the interior of single objects. Three separate phenomena provide strong evidence that in human vision, structure-from-motion is significantly affected by surface boundary cues. In the first, a rotating cylinder is seen, though no variation in optical flow exists across the apparent cylinder. In the second, the shape of the bounding contour of a moving pattern dominates the actual differential motion within the pattern. In the third, the appearance of independently moving objects changes significantly when the boundary between them becomes indistinct. We describe a simple computational model sufficient to account for these effects. The model is based on qualitative constraints relating possible object motions to patterns of flow, together with an understanding of the patterns of flow that can be discriminated in practice.     

The computational measurement of apparent motion: A recurrent pattern recognition strategy as an approach to solve the correspondence problem

In short, the model consists of a two-dimensional set of edge detecting units, modelled according to the zero-crossing detectors introduced first by Marr and Ullman (1981). These detectors are located peripherally in our synthetic vision system and are the input elements for an intelligent recurrent network. The purpose of that network is to recognize and categorize the previously detected contrast changes in a multi-resolution representation of the original image in such a manner that the original information will be decomposed into a relatively small numberN of well-defined edge primitives. The advantage of such a construction is that time-consuming pattern recognition has no longer to be done on the originally complex motion-blurred images of moving objects, but on a limited number of categorized forms. Based on a numberM of elementary feature attributes for each individual edge primitive, the model is then able to decompose each edge pattern into certain features. In this way anM-dimensional vector can be constructed for each edge. For each sequence of two successive frames a tensor can be calculated containing the distances (measured inM-dimensional feature space) between all features in both images. This procedure yields a set ofK—1 tensors for a sequence ofK images. After cross-correlation of allN ×M feature attributes from image (i) with those from image (i+1), wherei = 1, ...,K - 1, probability distributions can be computed. The final step is to search for maxima in these probability functions and then to construct from these extremes an optimal motion field. A number of simulation examples will be presented.     

Measurement of instantaneous velocity vectors of organelle transport: mitochondrial transport and bioenergetics in hippocampal neurons

Impaired transport of mitochondria, in dendrites and axons of neurons, and bioenergetic deficit are increasingly recognized to be of pathological importance in neurodegenerative diseases. To study the relationship between transport and bioenergetics, we have developed what to our knowledge is a novel technique to quantify organelle velocity in cultured cells. The aim was to combine measurement of motion and bioenergetic parameters while minimizing photodynamic oxidative artifacts evoked by fluorescence excitation. Velocity determination from sequential fluorescence images is not trivial, and here we describe an application of “optical flow”, the flow of gray values in grayscale images, to this problem. Based on the principles of photon shot noise occurring in low light level fluorescence microscopy, we describe and validate here an optical flow-based, robust method to measure velocity vectors for organelles expressing fluorescent proteins. This method features instantaneous velocity determination from a pair of images by detecting motion of edges, with no assumptions about the separation or shapes of the objects in the image. Optical flow was used in combination with single mitochondrion assay of mitochondrial thiol redox status by mitochondrially targeted redox-sensitive green fluorescent protein and measurement of mitochondrial membrane potential by tetramethylrhodamine methyl ester. Mitochondrial populations of resting cultured hippocampal neurons were analyzed. It was found that mitochondria with more oxidized thiol redox status have lower membrane potentials and are smaller in size. These mitochondria are more motile than the average; however, mitochondrial motility is only slightly dependent on the observed bioenergetic parameters and is correlated the best to the size of the mitochondria.     

Dynamic Lung Modeling and Tumor Tracking Using Deformable Image Registration and Geometric Smoothing

A greyscale-based fully automatic deformable image registration algorithm, based on an optical flow method together with geometric smoothing, is developed for dynamic lung modeling and tumor tracking. In our computational processing pipeline, the input data is a set of 4D CT images with 10 phases. The triangle mesh of the lung model is directly extracted from the more stable exhale phase (Phase 5). In addition, we represent the lung surface model in 3D volumetric format by applying a signed distance function and then generate tetrahedral meshes. Our registration algorithm works for both triangle and tetrahedral meshes. In CT images, the intensity value reflects the local tissue density. For each grid point, we calculate the displacement from the static image (Phase 5) to match with the moving image (other phases) by using merely intensity values of the CT images. The optical flow computation is followed by a regularization of the deformation field using geometric smoothing. Lung volume change and the maximum lung tissue movement are used to evaluate the accuracy of the application. Our testing results suggest that the application of deformable registration algorithm is an effective way for delineating and tracking tumor motion in image-guided radiotherapy.     

Finding motion parameters from spherical motion fields (or the advantages of having eyes in the back of your head)

A theory is developed for determining the motion of an observer given the motion field over a full 360 degree image sphere. The method is based on the fact that for an observer translating without rotation, the projected circular motion field about any equator can be divided into disjoint semicircles of clockwise and counterclockwise flow, and on the observation that the effects of rotation decouple around the three equators defining the three principal axes of rotation. Since the effect of rotation is geometrical, the three rotational parameters can be determined independently by searching, in each case, for a rotational value for which the derotated equatorial motion field can be partitioned into 180 degree arcs of clockwise and counterclockwise flow. The direction of translation is also obtained from this analysis. This search is two dimensional in the motion parameters, and can be performed relatively efficiently. Because information is correlated over large distances, the method can be considered a pattern recognition rather than a numerical algorithm. The algorithm is shown to be robust and relatively insensitive to noise and to missing data. Both theoretical and empirical studies of the error sensitivity are presented. The theoretical analysis shows that for white noise of bounded magnitude M, the expected errors is at worst linearly proportional to M. Empirical tests demonstrate negligible error for perturbations of up to 20% in the input, and errors of less than 20% for perturbations of up to 200%.     

Clustering with neural networks

Partitioning a set ofN patterns in ad-dimensional metric space intoK clusters — in a way that those in a given cluster are more similar to each other than the rest — is a problem of interest in many fields, such as, image analysis, taxonomy, astrophysics, etc. As there are approximatelyK ^N/K! possible ways of partitioning the patterns amongK clusters, finding the best solution is beyond exhaustive search whenN is large. We show that this problem, in spite of its exponential complexity, can be formulated as an optimization problem for which very good, but not necessarily optimal, solutions can be found by using a Hopfield model of neural networks. To obtain a very good solution, the network must start from many randomly selected initial states. The network is simulated on the MPP, a 128 × 128 SIMD array machine, where we use the massive parallelism not only in solving the differential equations that govern the evolution of the network, but also in starting the network from many initial states at once thus obtaining many solutions in one run. We achieve speedups of two to three orders of magnitude over serial implementations and the promise through Analog VLSI implementations of further speedups of three to six orders of magnitude.Supported by a National Research Council-NASA Research Associatship     

Measuring Hearing Organ Vibration Patterns with Confocal Microscopy and Optical Flow

A new method for visualizing vibrating structures is described. The system provides a means to capture very fast repeating events by relatively minor modifications to a standard confocal microscope. An acousto-optic modulator was inserted in the beam path, generating brief pulses of laser light. Images were formed by summing consecutive frames until every pixel of the resulting image had been exposed to a laser pulse. Images were analyzed using a new method for optical flow computation; it was validated through introducing artificial displacements in confocal images. Displacements in the range of 0.8 to 4 pixels were measured with 5% error or better. The lower limit for reliable motion detection was 20% of the pixel size. These methods were used for investigating the motion pattern of the vibrating hearing organ. In contrast to standard theory, we show that the organ of Corti possesses several degrees of freedom during sound-evoked vibration. Outer hair cells showed motion indicative of deformation. After acoustic overstimulation, supporting cells contracted. This slowly developing structural change was visualized during simultaneous intense sound stimulation and its speed measured with the optical flow technique.     

Double-Exposure Optical Sectioning Structured Illumination Microscopy Based on Hilbert Transform Reconstruction

Structured illumination microscopy (SIM) with axially optical sectioning capability has found widespread applications in three-dimensional live cell imaging in recent years, since it combines high sensitivity, short image acquisition time, and high spatial resolution. To obtain one sectioned slice, three raw images with a fixed phase-shift, normally 2π/3, are generally required. In this paper, we report a data processing algorithm based on the one-dimensional Hilbert transform, which needs only two raw images with arbitrary phase-shift for each single slice. The proposed algorithm is different from the previous two-dimensional Hilbert spiral transform algorithm in theory. The presented algorithm has the advantages of simpler data processing procedure, faster computation speed and better reconstructed image quality. The validity of the scheme is verified by imaging biological samples in our developed DMD-based LED-illumination SIM system.     

Solving the N-Queens problem with a binary Hopfield-type network

The application of a discrete Hopfield-type neural network to solving the NP-Hard optimization problem — the N-Queens Problem (NQP) — is presented. The applied network is binary, and at every moment each neuron potential is equal to either 0 or 1. The network can be implemented in the asynchronous mode as well as in the synchronous one with n parallel running processors. In both cases the convergence rate is up to 100%, and the experimental estimate of the average computational complexity is polynomial. Based on the computer simulation results and the theoretical analysis, the proper network parameters are established. The behaviour of the network is explained.     

Flow Measurements in a Blood-Perfused Collagen Vessel Using X-Ray Micro-Particle Image Velocimetry

Blood-perfused tissue models are joining the emerging field of tumor engineering because they provide new avenues for modulation of the tumor microenvironment and preclinical evaluation of the therapeutic potential of new treatments. The characterization of fluid flow parameters in such in-vitro perfused tissue models is a critical step towards better understanding and manipulating the tumor microenvironment. However, traditional optical flow measurement methods are inapplicable because of the opacity of blood and the thickness of the tissue sample. In order to overcome the limitations of optical method we demonstrate the feasibility of using phase-contrast x-ray imaging to perform microscale particle image velocimetry (PIV) measurements of flow in blood perfused hydrated tissue-representative microvessels. However, phase contrast x-ray images significantly depart from the traditional PIV image paradigm, as they have high intensity background, very low signal-to-noise ratio, and volume integration effects. Hence, in order to achieve accurate measurements special attention must be paid to the image processing and PIV cross-correlation methodologies. Therefore we develop and demonstrate a methodology that incorporates image preprocessing as well as advanced PIV cross-correlation methods to result in measured velocities within experimental uncertainty.     

Systématique des Saccharomyces: Osidases et besoins vitaminiques

A study of the genus Saccharomyces by new biocharacters demonstrates indirectly the genotype of each species; intracellular osidases (lactose — maltose — cellobiase — melibiase — invertase — trehalase) and vitamine requirements (biotin — folic acid — inositol — niacine — pantothénate —para-aminobenzoïc acid — pyridoxine — thiamine) reveal some phylogenetic links between the species. It appears that Van der Walt's group I is very homogeneous and must be restricted to only 7 species. The Torulospora group seems very similar to the genus Debaryomyces when vitamine needs and osidases are considered; the relations between the 2 groups are discussed. When other biocharacters are studied (nitrite reductase-immunological properties, etc. ...) the value in the GC content (4–5% lower in Debaryomyces) will be established.These results confirm that intracellular osidases and vitamine needs are very valuable tools for systematic criteria.

---

---

Collaboration technique: Mme Billon-Grand.     

Movement correction of digital sequence angiographies of the retina]

Retinal hemodynamics can be quantified from videoangiographic image sequences by digital image processing. Intensity changes of dye dilution curves provide dynamics parameters of the local retinal blood flow. The measuring points of dye dilution curves have to be fixed on identical image contents in each image of a complete image sequence. To obtain measurements for every pixel on the retinal surface a motion-compensated image sequence is required. A new method adapted to the compensation of eye motion and movement artifacts in Scanning Laser Ophthalmoscopy in long image sequences (300-500 images) is presented in this paper. To inhibit error propagation of time sequential motion estimation, the eye movement is divided into two dynamic movements components. The method presented permits compensation for eye motion in retinal fluorescein angiographic sequences. Owing to the short calculation times, this algorithm can be used in clinical routine.     

Energetic aspects of the gliding mitility of mycoplasmas

This report reviews the question of whether mycoplasma gliding is an active form of surface translocation or a biased Brownian motion. The theoretical calculations presented here, along with recently described experimental observations, unequivocally prove that the motility of some mycoplasmas—at least, however, ofMycoplasma mobile—requires a cellular energy source. In an accompanying report we present further experimental evidence supporting this conclusion.     

Quantitative Analysis of Respiration-Related Movement for Abdominal Artery in Multiphase Hepatic CT

Objectives

Respiration-induced motion in the liver causes potential errors on the measurement of contrast medium in abdominal artery from multiphase hepatic CT scans. In this study, we investigated the use of hepatic CT images to quantitatively estimate the abdominal artery motion due to respiration by optical flow method.

Materials and Methods

A total of 132 consecutive patients were included in our patient cohort. We apply the optical flow method to compute the motion of the abdominal artery due to respiration.

Results

The minimum and maximum displacements of the abdominal artery motion were 0.02 and 30.87 mm by manual delineation, 0.03 and 40.75 mm calculated by optical flow method, respectively. Both high consistency and correlation between the present method and the physicians’ manual delineations were acquired with the regression equation of movement, y = 0.81x+0.25, r = 0.95, p<0.001.

Conclusion

We estimated the motion of abdominal artery due to respiration using the optical flow method in multiphase hepatic CT scans and the motion estimations were validated with the visualization of physicians. The quantitative analysis of respiration-related movement of abdominal artery could be used for motion correction in the measurement of contrast medium passing though abdominal artery in multiphase CT liver scans.     

基于主成分分析的双对比光学投影断层成像

目的 本文提出了一种基于主成分分析（PCA）的双对比光学投影断层成像（DC-OPT）方法,以获得活体中血流网络和骨骼的三维可视化。方法 使用主成分分析方法来提取吸收图像和血流图像,原始图像序列的第一主成分用于获取吸收图像;通过计算每个像素的调制深度来获得流动图像。不同投影位置的流动和吸收对比图像被用于三维血流网络和骨骼的同步重建。结果 采用PCA和OPT相结合的方法,通过将动态血流信号和静态背景信号分离,实现了对微生物样本的血流网络和骨骼的三维成像。结论 本文研究的新颖之处在于通过同一光学系统获得了快速、同步、双对比的血流网络和骨骼三维图像。实验结果可用于活体生物的生理发育研究。