A Flexible Calibration Method Using the Planar Target with a Square Pattern for Line Structured Light Vision System

A flexible calibration approach for line structured light vision system is proposed in this paper. Firstly a camera model is established by transforming the points from the 2D image plane to the world coordinate frame, and the intrinsic parameters of camera can be obtained accurately. Then a novel calibration method for structured light projector is presented by moving a planar target with a square pattern randomly, and the method mainly involves three steps: first, a simple linear model is proposed, by which the plane equation of the target at any orientations can be determined based on the square’s geometry information; second, the pixel coordinates of the light stripe center on the target images are extracted as the control points; finally, the points are projected into the camera coordinate frame with the help of the intrinsic parameters and the plane equations of the target, and the structured light plane can be determined by fitting these three-dimensional points. The experimental data show that the method has good repeatability and accuracy.     

An algorithm to correct for camera vibrations in optical motion tracking systems

Recording and reconstruction of 3D motion capturing data relies on fixed, static camera positions with given inter-camera distances in a laboratory frame. To overcome this limitation, we present a correction algorithm that allows us to address camera movements in moving camera setups. Camera vibrations are identified by comparison of specialized target positions in dynamic measurements with their respective positions in static trials. This results in a 2D shift vector Δw with which the individual camera streams are corrected. The capabilities of this vibration reduction procedure are demonstrated in a test setup of four cameras that are (i) separately and (ii) simultaneously perturbed while capturing a static test object. In the former case, the correction algorithm is capable of reducing the reconstruction residuals to the order of the calibrations residual and enables reconstruction in the latter case, which is impossible without any correction. This approach extends the application of marker-based infrared motion tracking to moving and even accelerated camera setups.     

Animal Scanner: Software for classifying humans,animals, and empty frames in camera trap images

Camera traps are a popular tool to sample animal populations because they are noninvasive, detect a variety of species, and can record many thousands of animal detections per deployment. Cameras are typically set to take bursts of multiple photographs for each detection and are deployed in arrays of dozens or hundreds of sites, often resulting in millions of photographs per study. The task of converting photographs to animal detection records from such large image collections is daunting, and made worse by situations that generate copious empty pictures from false triggers (e.g., camera malfunction or moving vegetation) or pictures of humans. We developed computer vision algorithms to detect and classify moving objects to aid the first step of camera trap image filtering—separating the animal detections from the empty frames and pictures of humans. Our new work couples foreground object segmentation through background subtraction with deep learning classification to provide a fast and accurate scheme for human–animal detection. We provide these programs as both Matlab GUI and command prompt developed with C++. The software reads folders of camera trap images and outputs images annotated with bounding boxes around moving objects and a text file summary of results. This software maintains high accuracy while reducing the execution time by 14 times. It takes about 6 seconds to process a sequence of ten frames (on a 2.6 GHZ CPU computer). For those cameras with excessive empty frames due to camera malfunction or blowing vegetation automatically removes 54% of the false‐triggers sequences without influencing the human/animal sequences. We achieve 99.58% on image‐level empty versus object classification of Serengeti dataset. We offer the first computer vision tool for processing camera trap images providing substantial time savings for processing large image datasets, thus improving our ability to monitor wildlife across large scales with camera traps.     

Moment Feature Based Fast Feature Extraction Algorithm for Moving Object Detection Using Aerial Images

Fast and computationally less complex feature extraction for moving object detection using aerial images from unmanned aerial vehicles (UAVs) remains as an elusive goal in the field of computer vision research. The types of features used in current studies concerningmoving object detection are typically chosen based on improving detection rate rather than on providing fast and computationally less complex feature extraction methods. Because moving object detection using aerial images from UAVs involves motion as seen from a certain altitude, effective and fast feature extraction is a vital issue for optimum detection performance. This research proposes a two-layer bucket approach based on a new feature extraction algorithm referred to as the moment-based feature extraction algorithm (MFEA). Because a moment represents thecoherent intensity of pixels and motion estimation is a motion pixel intensity measurement, this research used this relation to develop the proposed algorithm. The experimental results reveal the successful performance of the proposed MFEA algorithm and the proposed methodology.     

Fast Fight Detection

Action recognition has become a hot topic within computer vision. However, the action recognition community has focused mainly on relatively simple actions like clapping, walking, jogging, etc. The detection of specific events with direct practical use such as fights or in general aggressive behavior has been comparatively less studied. Such capability may be extremely useful in some video surveillance scenarios like prisons, psychiatric centers or even embedded in camera phones. As a consequence, there is growing interest in developing violence detection algorithms. Recent work considered the well-known Bag-of-Words framework for the specific problem of fight detection. Under this framework, spatio-temporal features are extracted from the video sequences and used for classification. Despite encouraging results in which high accuracy rates were achieved, the computational cost of extracting such features is prohibitive for practical applications. This work proposes a novel method to detect violence sequences. Features extracted from motion blobs are used to discriminate fight and non-fight sequences. Although the method is outperformed in accuracy by state of the art, it has a significantly faster computation time thus making it amenable for real-time applications.     

A theory on the determination of 3D motion and 3D structure from features

The present paper proposes a mathematical theory and a method of recognition of both the 3D structure and the motion of a moving object from its monocular image. Initially, characteristic features are extracted from the 2D perspective image of the object. Because motion of the object induces a change in its 2D perspective image, it also induces a change in the features which depends on the 3D structure and the velocity of the object. This suggests the possibility of detecting the 3D structure and the motion directly from the features and their changing rate, without the need for calculating optical flows. An analysis is made of the relation between the 3D rigid motion of a surface element and the change in local linear features. From this relation, a method is proposed for calculating the velocity of and the normal to the surface element without considering any correspondence of points. An optical flow can also be calculated by this method. Two simple computer simulations are provided.     

Acoustic facilitation of object movement detection during self-motion

In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visual-only task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations.     

Motion Control in Saccade and Smooth Pursuit for Bionic Eye Based on Three-dimensional Coordinates

Saccade and smooth pursuit are two important functions of human eye.In order to enable bionic eye to imitate the two functions,a control method that implements saccade and smooth pursuit based on the three-dimensional coordinates of target is proposed.An optimal observation position is defined for bionic eye based on three-dimensional coordinates.A kind of motion planning method with high accuracy is developed.The motion parameters of stepper motor consisting of angle acceleration and turning time are computed according to the position deviation,the target's angular velocity and the stepper motor's current angular velocity in motion planning.The motors are controlled with the motion parameters moving to given position with desired angular velocity in schedule time.The experimental results show that the bionic eye can move to optimal observation positions in 0.6 s from initial location and the accuracy of 3D coordinates is improved.In addition,the bionic eye can track a target within the error of less than 20 pixels based on three-dimensional coordinates.It is verified that saccade and smooth pursuit of bionic eye based on three-dimensional coordinates are feasible.     

Motion vectors and deep neural networks for video camera traps

Commercial camera traps are usually triggered by a Passive Infra-Red (PIR) motion sensor necessitating a delay between triggering and the image being captured. This often seriously limits the ability to record images of small and fast moving animals. It also results in many “empty” images, e.g., owing to moving foliage against a background of different temperature. In this paper we detail a new triggering mechanism based solely on the camera sensor. This is intended for use by citizen scientists and for deployment on an affordable, compact, low-power Raspberry Pi computer (RPi). Our system introduces a video frame filtering pipeline consisting of movement and image-based processing. This makes use of Machine Learning (ML) feasible on a live camera stream on an RPi. We describe our free and open-source software implementation of the system; introduce a suitable ecology efficiency measure that mediates between specificity and recall; provide ground-truth for a video clip collection from camera traps; and evaluate the effectiveness of our system thoroughly. Overall, our video camera trap turns out to be robust and effective.     

High-speed multicolor microscopy of repeating dynamic processes

Images of multiply labeled fluorescent samples provide unique insights into the localization of molecules, cells, and tissues. The ability to image multiple channels simultaneously at high speed without cross talk is limited to a few colors and requires dedicated multichannel or multispectral detection procedures. Simpler microscopes, in which each color is imaged sequentially, produce a much lower frame rate. Here, we describe a technique to image, at high frame rate, multiply labeled samples that have a repeating motion. We capture images in a single channel at a time over one full occurrence of the motion then repeat acquisition for other channels over subsequent occurrences. We finally build a high-speed multichannel image sequence by combining the images after applying a normalized mutual information-based time registration procedure. We show that this technique is amenable to image the beating heart of a double-labeled embryonic quail in three channels (brightfield, yellow, and mCherry fluorescent proteins) using a fluorescence wide-field microscope equipped with a single monochrome camera and without fast channel switching optics. We experimentally evaluate the accuracy of our method on image series from a two-channel confocal microscope.     

3D Orbital Tracking in a Modified Two-photon Microscope: An Application to the Tracking of Intracellular Vesicles

The objective of this video protocol is to discuss how to perform and analyze a three-dimensional fluorescent orbital particle tracking experiment using a modified two-photon microscope¹. As opposed to conventional approaches (raster scan or wide field based on a stack of frames), the 3D orbital tracking allows to localize and follow with a high spatial (10 nm accuracy) and temporal resolution (50 Hz frequency response) the 3D displacement of a moving fluorescent particle on length-scales of hundreds of microns². The method is based on a feedback algorithm that controls the hardware of a two-photon laser scanning microscope in order to perform a circular orbit around the object to be tracked: the feedback mechanism will maintain the fluorescent object in the center by controlling the displacement of the scanning beam^3-5. To demonstrate the advantages of this technique, we followed a fast moving organelle, the lysosome, within a living cell^6,7. Cells were plated according to standard protocols, and stained using a commercially lysosome dye. We discuss briefly the hardware configuration and in more detail the control software, to perform a 3D orbital tracking experiment inside living cells. We discuss in detail the parameters required in order to control the scanning microscope and enable the motion of the beam in a closed orbit around the particle. We conclude by demonstrating how this method can be effectively used to track the fast motion of a labeled lysosome along microtubules in 3D within a live cell. Lysosomes can move with speeds in the range of 0.4-0.5 µm/sec, typically displaying a directed motion along the microtubule network⁸.     

基于帧间光流特征和改进RNN的草鱼摄食状态分类

针对鱼类连续摄食行为较难识别与量化的问题, 提出一种基于帧间光流特征和改进递归神经网络(Recurrent neural network, RNN)的草鱼摄食状态分类方法。首先利用偏振相机搭建户外池塘采样系统, 采集不同偏振角度水面图像, 并基于图像饱和度和亮度模型自动选择低反光角度图像, 构建图像样本库; 其次通过光流法提取帧间运动特征, 并基于投饲机开关状态构建时间序列帧间特征样本集, 然后利用样本集训练改进RNN分类网络。以上海市崇明区瑞钵水产养殖专业合作社的试验数据对该方法进行验证。结果表明, 研究方法综合准确率为91%, 召回率为92.2%, 均优于传统的鱼类摄食行为识别方法。研究结果可为鱼类精准投喂技术研究提供参考。     

Chaotic neural network applied to two-dimensional motion control

Chaotic dynamics generated in a chaotic neural network model are applied to 2-dimensional (2-D) motion control. The change of position of a moving object in each control time step is determined by a motion function which is calculated from the firing activity of the chaotic neural network. Prototype attractors which correspond to simple motions of the object toward four directions in 2-D space are embedded in the neural network model by designing synaptic connection strengths. Chaotic dynamics introduced by changing system parameters sample intermediate points in the high-dimensional state space between the embedded attractors, resulting in motion in various directions. By means of adaptive switching of the system parameters between a chaotic regime and an attractor regime, the object is able to reach a target in a 2-D maze. In computer experiments, the success rate of this method over many trials not only shows better performance than that of stochastic random pattern generators but also shows that chaotic dynamics can be useful for realizing robust, adaptive and complex control function with simple rules.     

Correcting out-of-plane errors in two-dimensional imaging using nonimage-related information

Two-dimensional imaging with a single camera assumes that the motion occurs in a calibrated plane perpendicular to the camera axis. It is well known that kinematic errors result if the object fails to remain in this plane and that if both the distance to the calibration plane from the camera and the distance out-of-plane are known, an analytical correction for the out-of-plane error can be made. Less well appreciated is that out-of-plane distance can frequently be acquired from other, nonimage-related information. In the two examples given, the mediolateral center of pressure coordinate of the foot measured from a force plate and the measured landing point of a shot put throw were used. In both cases, the resulting out-of-plane correction improved the accuracy of the 2-D kinematic data dramatically. These examples also demonstrate that the use of nonimage-related data can increase the accuracy of kinematic data without an increase in the complexity of the experiment.     

Tracking single particles: a user-friendly quantitative evaluation

As our knowledge of biological processes advances, we are increasingly aware that cells actively position sub-cellular organelles and other constituents to control a wide range of biological processes. Many studies quantify the position and motion of, for example, fluorescently labeled proteins, protein aggregates, mRNA particles or virus particles. Both differential interference contrast (DIC) and fluorescence microscopy can visualize vesicles, nuclei or other small organelles moving inside cells. While such studies are increasingly important, there has been no complete analysis of the different tracking methods in use, especially from the practical point of view. Here we investigate these methods and clarify how well different algorithms work and also which factors play a role in assessing how accurately the position of an object can be determined. Specifically, we consider how ultimate performance is affected by magnification, by camera type (analog versus digital), by recording medium (VHS and SVHS tape versus direct tracking from camera), by image compression, by type of imaging used (fluorescence versus DIC images) and by a variety of sources of noise. We show that most methods are capable of nanometer scale accuracy under realistic conditions; tracking accuracy decreases with increasing noise. Surprisingly, accuracy is found to be insensitive to the numerical aperture, but, as expected, it scales with magnification, with higher magnification yielding improved accuracy (within limits of signal-to-noise). When noise is present at reasonable levels, the effect of image compression is in most cases small. Finally, we provide a free, robust implementation of a tracking algorithm that is easily downloaded and installed.     

Mechanical quality assurance using light field for linear accelerators with camera calibration

Mechanical Quality Assurance (QA) is important to assure spatially precise delivery of external-beam radiation therapy. As an alternative to the conventional-film based method, we have developed a new tool for mechanical QA of LINACs which uses a light field rather than radiation. When light passes through the collimator, a shadow is projected onto a piece of translucent paper and the resulting image is captured by a digital camera via a mirror. With this method, we evaluated the position of the LINAC isocenter and the accuracy of the gantry, collimator, and couch rotation. We also evaluated the accuracy of the digital readouts of the gantry, collimator, and couch rotation. In addition, the treatment couch position indicator was tested. We performed camera calibration as an essential pre-requisite for quantitative measurements of the position of isocenter, the linear motion of the couch, and the rotation angles of the gantry and collimator. Camera calibration reduced the measurement error to submillimeter based on uncertainty in pixel size of the image, while, without calibration, the measurement error of up to 2 mm could occur for an object with a length of 5 cm.     

Quantitative underwater 3D motion analysis using submerged video cameras: accuracy analysis and trajectory reconstruction

In this study we aim at investigating the applicability of underwater 3D motion capture based on submerged video cameras in terms of 3D accuracy analysis and trajectory reconstruction. Static points with classical direct linear transform (DLT) solution, a moving wand with bundle adjustment and a moving 2D plate with Zhang's method were considered for camera calibration. As an example of the final application, we reconstructed the hand motion trajectories in different swimming styles and qualitatively compared this with Maglischo's model. Four highly trained male swimmers performed butterfly, breaststroke and freestyle tasks. The middle fingertip trajectories of both hands in the underwater phase were considered. The accuracy (mean absolute error) of the two calibration approaches (wand: 0.96 mm – 2D plate: 0.73 mm) was comparable to out of water results and highly superior to the classical DLT results (9.74 mm). Among all the swimmers, the hands' trajectories of the expert swimmer in the style were almost symmetric and in good agreement with Maglischo's model. The kinematic results highlight symmetry or asymmetry between the two hand sides, intra- and inter-subject variability in terms of the motion patterns and agreement or disagreement with the model. The two outcomes, calibration results and trajectory reconstruction, both move towards the quantitative 3D underwater motion analysis.     

Sampling frequencies and measurement error for linear and temporal gait parameters in primate locomotion

Quantitative analyses of animal motion are increasingly easy to conduct using simple video equipment and relatively inexpensive software packages. With careful use, such analytical tools have the potential to quantify differences in movement between individuals or species and to allow insights into the behavioral consequences of morphological differences between taxa. However, as with any other type of measurement, there are errors associated with kinematic measurements. Because normative kinematic data on human and nonhuman primate locomotion are used to model aspects of gait of fossil hominins, errors in the extant data influence the accuracy of fossil gait reconstructions. The principal goal of this paper is to illustrate the effect of camera speeds (frame rates) on kinematic measurement errors, and to demonstrate how these errors vary with subject size, movement velocity, and sample size. Kinematic data for human walking and running (240 Hz), as well as data for primate quadrupedal walking and running (180 Hz) were used as inputs for a simulation of the measurement errors associated with various linear and temporal kinematic variables. Measurement errors were shown to increase as camera speed, subject body size, and interval duration all decrease, and as movement velocity increases. These results have implications for the methods used to calculate subject velocity and suggest that using a moving marker to measure the linear displacements of the body is preferable to the use of a stationary marker. Finally, while slower camera speeds will always result in higher measurement errors than do faster camera speeds, this effect can be moderated to some extent by collecting sufficiently large samples of data.     

Object Segmentation from Motion Discontinuities and Temporal Occlusions–A Biologically Inspired Model

Background

Optic flow is an important cue for object detection. Humans are able to perceive objects in a scene using only kinetic boundaries, and can perform the task even when other shape cues are not provided. These kinetic boundaries are characterized by the presence of motion discontinuities in a local neighbourhood. In addition, temporal occlusions appear along the boundaries as the object in front covers the background and the objects that are spatially behind it.

Methodology/Principal Findings

From a technical point of view, the detection of motion boundaries for segmentation based on optic flow is a difficult task. This is due to the problem that flow detected along such boundaries is generally not reliable. We propose a model derived from mechanisms found in visual areas V1, MT, and MSTl of human and primate cortex that achieves robust detection along motion boundaries. It includes two separate mechanisms for both the detection of motion discontinuities and of occlusion regions based on how neurons respond to spatial and temporal contrast, respectively. The mechanisms are embedded in a biologically inspired architecture that integrates information of different model components of the visual processing due to feedback connections. In particular, mutual interactions between the detection of motion discontinuities and temporal occlusions allow a considerable improvement of the kinetic boundary detection.

Conclusions/Significance

A new model is proposed that uses optic flow cues to detect motion discontinuities and object occlusion. We suggest that by combining these results for motion discontinuities and object occlusion, object segmentation within the model can be improved. This idea could also be applied in other models for object segmentation. In addition, we discuss how this model is related to neurophysiological findings. The model was successfully tested both with artificial and real sequences including self and object motion.     

Automatically detecting the wild giant panda using deep learning with context and species distribution model

The giant panda is a flagship species in ecological conservation. The infrared camera trap is an effective tool for monitoring the giant panda. Images captured by infrared camera traps must be accurately recognized before further statistical analyses can be implemented. Previous research has demonstrated that spatiotemporal and positional contextual information and the species distribution model (SDM) can improve image detection accuracy, especially for difficult-to-see images. Difficult-to-see images include those in which individual animals are only partially observed and it is challenging for the model to detect those individuals. By utilizing the attention mechanism, we developed a unique method based on deep learning that incorporates object detection, contextual information, and the SDM to achieve better detection performance in difficult-to-see images. We obtained 1169 images of the wild giant panda and divided them into a training set and a test set in a 4:1 ratio. Model assessment metrics showed that our proposed model achieved an overall performance of 98.1% in mAP_0.5 and 82.9% in recall on difficult-to-see images. Our research demonstrated that the fine-grained multimodal-fusing method applied to monitoring giant pandas in the wild can better detect the difficult-to-see panda images to enhance the wildlife monitoring system.