A computer vision based method for 3D posture estimation of symmetrical lifting

Work-related musculoskeletal disorders (WMSD) are commonly observed among the workers involved in material handling tasks such as lifting. To improve work place safety, it is necessary to assess musculoskeletal and biomechanical risk exposures associated with these tasks. Such an assessment has been mainly conducted using surface marker-based methods, which is time consuming and tedious. During the past decade, computer vision based pose estimation techniques have gained an increasing interest and may be a viable alternative for surface marker-based human movement analysis. The aim of this study is to develop and validate a computer vision based marker-less motion capture method to assess 3D joint kinematics of lifting tasks. Twelve subjects performing three types of symmetrical lifting tasks were filmed from two views using optical cameras. The joints kinematics were calculated by the proposed computer vision based motion capture method as well as a surface marker-based motion capture method. The joint kinematics estimated from the computer vision based method were practically comparable to the joint kinematics obtained by the surface marker-based method. The mean and standard deviation of the difference between the joint angles estimated by the computer vision based method and these obtained by the surface marker-based method was 2.31 ± 4.00°. One potential application of the proposed computer vision based marker-less method is to noninvasively assess 3D joint kinematics of industrial tasks such as lifting.     

Towards Active Tracking of Beating Heart Motion in the Presence of Arrhythmia for Robotic Assisted Beating Heart Surgery

In robotic assisted beating heart surgery, the control architecture for heart motion tracking has stringent requirements in terms of bandwidth of the motion that needs to be tracked. In order to achieve sufficient tracking accuracy, feed-forward control algorithms, which rely on estimations of upcoming heart motion, have been proposed in the literature. However, performance of these feed-forward motion control algorithms under heart rhythm variations is an important concern. In their past work, the authors have demonstrated the effectiveness of a receding horizon model predictive control-based algorithm, which used generalized adaptive predictors, under constant and slowly varying heart rate conditions. This paper extends these studies to the case when the heart motion statistics change abruptly and significantly, such as during arrhythmias. A feasibility study is carried out to assess the motion tracking capabilities of the adaptive algorithms in the occurrence of arrhythmia during beating heart surgery. Specifically, the tracking performance of the algorithms is evaluated on prerecorded motion data, which is collected in vivo and includes heart rhythm irregularities. The algorithms are tested using both simulations and bench experiments on a three degree-of-freedom robotic test bed. They are also compared with a position-plus-derivative controller as well as a receding horizon model predictive controller that employs an extended Kalman filter algorithm for predicting future heart motion.     

A Deep Neural Network-based method for estimation of 3D lifting motions

The aim of this study is developing and validating a Deep Neural Network (DNN) based method for 3D pose estimation during lifting. The proposed DNN based method addresses problems associated with marker-based motion capture systems like excessive preparation time, movement obstruction, and controlled environment requirement. Twelve healthy adults participated in a protocol and performed nine lifting tasks with different vertical heights and asymmetry angles. They lifted a crate and placed it on a shelf while being filmed by two camcorders and a synchronized motion capture system, which directly measured their body movement. A DNN with two-stage cascaded structure was designed to estimate subjects’ 3D body pose from images captured by camcorders. Our DNN augmented Hourglass network for monocular 2D pose estimation with a novel 3D pose generator subnetwork, which synthesized information from all available views to predict accurate 3D pose. We validated the results against the marker-based motion capture system as a reference and examined the method performance under different lifting conditions. The average Euclidean distance between the estimated 3D pose and reference (3D pose error) on the whole dataset was 14.72 ± 2.96 mm. Repeated measures ANOVAs showed lifting conditions can affect the method performance e.g. 60° asymmetry angle and shoulder height lifting showed higher 3D pose error compare to other lifting conditions. The results demonstrated the capability of the proposed method for 3D pose estimation with high accuracy and without limitations of marker-based motion capture systems. The proposed method may be utilized as an on-site biomechanical analysis tool.     

Kinematical models to reduce the effect of skin artifacts on marker-based human motion estimation

The estimation of the skeletal motion obtained from marker-based motion capture systems is known to be affected by significant bias caused by skin movement artifacts, which affects joint center and rotation axis estimation. Among different techniques proposed in the literature, that based on rigid body model, still the most used by commercial motion capture systems, can smooth only part of the above effects without eliminating their main components. In order to sensibly improve the accuracy of the motion estimation, a novel technique, named local motion estimation (LME), is proposed. This rests on a recently described approach that, using virtual humans and extended Kalman filters, estimates the kinematical variables directly from 2D measurements without requiring the 3D marker reconstruction. In this paper, we show how such method can be extended to include the computation of the local marker displacement due to skin artifacts. The 3D marker coordinates, expressed in the corresponding local reference coordinate frames, are inserted into the state vector of the filter and their dynamics is automatically estimated, with adequate accuracy, without assuming any particular deformation function. Simulated experiments of lower limb motion, involving systematic mislocations (5, 10, 20 mm) and random errors of the marker coordinates and joint center locations (+/-5, +/-10, +/-15 mm), have shown that artifact motion can be substantially decoupled from the global skeletal motion with an effective increase of the accuracy wrt standard techniques. In particular, the comparison between the nominal kinematical variables and the one recovered from markers attached to the skin surface proved LME to be sensibly superior (50% in the worse condition) to the methods imposing marker-bone rigidity. In conclusion, while requiring further validation on real movement data, we argue that the proposed method can constitute an appropriate approach toward the improvement of the human motion estimation.     

A Markerless 3D Computerized Motion Capture System Incorporating a Skeleton Model for Monkeys

In this study, we propose a novel markerless motion capture system (MCS) for monkeys, in which 3D surface images of monkeys were reconstructed by integrating data from four depth cameras, and a skeleton model of the monkey was fitted onto 3D images of monkeys in each frame of the video. To validate the MCS, first, estimated 3D positions of body parts were compared between the 3D MCS-assisted estimation and manual estimation based on visual inspection when a monkey performed a shuttling behavior in which it had to avoid obstacles in various positions. The mean estimation error of the positions of body parts (3–14 cm) and of head rotation (35–43°) between the 3D MCS-assisted and manual estimation were comparable to the errors between two different experimenters performing manual estimation. Furthermore, the MCS could identify specific monkey actions, and there was no false positive nor false negative detection of actions compared with those in manual estimation. Second, to check the reproducibility of MCS-assisted estimation, the same analyses of the above experiments were repeated by a different user. The estimation errors of positions of most body parts between the two experimenters were significantly smaller in the MCS-assisted estimation than in the manual estimation. Third, effects of methamphetamine (MAP) administration on the spontaneous behaviors of four monkeys were analyzed using the MCS. MAP significantly increased head movements, tended to decrease locomotion speed, and had no significant effect on total path length. The results were comparable to previous human clinical data. Furthermore, estimated data following MAP injection (total path length, walking speed, and speed of head rotation) correlated significantly between the two experimenters in the MCS-assisted estimation (r = 0.863 to 0.999). The results suggest that the presented MCS in monkeys is useful in investigating neural mechanisms underlying various psychiatric disorders and developing pharmacological interventions.     

Sub-millimetre accurate human hand kinematics: from surface to skeleton

A highly accurate human hand kinematics model and identification are proposed. The model includes the five digits and the palm arc based on mapping function between surface landmarks and estimated joint centres of rotation. Model identification was experimentally performed using a motion tracking system. The evaluation of the marker position estimation error, which is on sub-millimetre level across all digits, underlines model quality and accuracy. Noticeably, with the development of this model, we were able to improve various modelling assumptions from literature and found a basic linear relationship between surface and skeleton rotational angles.     

Numerosity estimation in visual stimuli in the absence of luminance-based cues

Background

Numerosity estimation is a basic preverbal ability that humans share with many animal species and that is believed to be foundational of numeracy skills.It is notoriously difficult, however, to establish whether numerosity estimation is based on numerosity itself, or on one or more non-numerical cues like—in visual stimuli—spatial extent and density. Frequently, different non-numerical cues are held constant on different trials. This strategy, however, still allows numerosity estimation to be based on a combination of non-numerical cues rather than on any particular one by itself.

Methodology/Principal Findings

Here we introduce a novel method, based on second-order (contrast-based) visual motion, to create stimuli that exclude all first-order (luminance-based) cues to numerosity. We show that numerosities can be estimated almost as well in second-order motion as in first-order motion.

Conclusions/Significance

The results show that numerosity estimation need not be based on first-order spatial filtering, first-order density perception, or any other processing of luminance-based cues to numerosity. Our method can be used as an effective tool to control non-numerical variables in studies of numerosity estimation.     

A combined morphometric analysis of foot form and its association with sex,stature, and body mass

Objectives: Morphometric analysis of footprints is a classic means for orthopedic diagnosis. In forensics and physical anthropology, it is commonly used for the estimation of stature and body mass. We studied individual variation and sexual dimorphism of foot dimensions and footprint shape by a combination of classic foot measurements and geometric morphometric methods. Methods: Left and right feet of 134 healthy adult males and females were scanned twice with a 3D optical laser scanner, and stature as well as body mass were recorded. Foot length and width were measured on the 3D scans. The 2D footprints were extracted as the plantar‐most 2 mm of the 3D scans and measured with 85 landmarks and semilandmarks. Results: Both foot size and footprint shape are sexually dimorphic and relate to stature and body mass. While dimorphism in foot length largely results from dimorphism in stature, dimorphism in footprint shape partly owes to the dimorphism in BMI. Stature could be estimated well based on foot length (R² = 0.76), whereas body mass was more closely related to foot width (R² = 0.62). Sex could be estimated correctly for 95% of the individuals based on a combination of foot width and length. Discussion: Geometric morphometrics proved to be an effective tool for the detailed analysis of footprint shape. However, for the estimation of stature, body mass, and sex, shape variables did not considerably improve estimates based on foot length and width. Am J Phys Anthropol 157:582–591, 2015. © 2015 Wiley Periodicals, Inc.     

Continuous ambulatory hand force monitoring during manual materials handling using instrumented force shoes and an inertial motion capture suit

Hand forces (HFs) are commonly measured during biomechanical assessment of manual materials handling; however, it is often a challenge to directly measure HFs in field studies. Therefore, in a previous study we proposed a HF estimation method based on ground reaction forces (GRFs) and body segment accelerations and tested it with laboratory equipment: GFRs were measured with force plates (FPs) and segment accelerations were measured using optical motion capture (OMC). In the current study, we evaluated the HF estimation method based on an ambulatory measurement system, consisting of inertial motion capture (IMC) and instrumented force shoes (FSs).Sixteen participants lifted and carried a 10-kg crate from ground level while 3D full-body kinematics were measured using OMC and IMC, and 3D GRFs were measured using FPs and FSs. We estimated 3D hand force vectors based on: (1) FP+OMC, (2) FP+IMC and (3) FS+IMC. We calculated the root-mean-square differences (RMSDs) between the estimated HFs to reference HFs calculated based on crate kinematics and the GRFs of a FP that the crate was lifted from.Averaged over subjects and across 3D force directions, the HF RMSD ranged between 10-15N when using the laboratory equipment (FP + OMC), 11-18N when using the IMC instead of OMC data (FP+IMC), and 17-21N when using the FSs in combination with IMC (FS + IMC). This error is regarded acceptable for the assessment of spinal loading during manual lifting, as it would results in less than 5% error in peak moment estimates.     

Localized strain measurements of the intervertebral disc annulus during biaxial tensile testing

Both inter-lamellar and intra-lamellar failures of the annulus have been described as potential modes of disc herniation. Attempts to characterize initial lamellar failure of the annulus have involved tensile testing of small tissue samples. The purpose of this study was to evaluate a method of measuring local surface strains through image analysis of a tensile test conducted on an isolated sample of annular tissue in order to enhance future studies of intervertebral disc failure. An annulus tissue sample was biaxial strained to 10%. High-resolution images captured the tissue surface throughout testing. Three test conditions were evaluated: submerged, non-submerged and marker. Surface strains were calculated for the two non-marker conditions based on motion of virtual tracking points. Tracking algorithm parameters (grid resolution and template size) were varied to determine the effect on estimated strains. Accuracy of point tracking was assessed through a comparison of the non-marker conditions to a condition involving markers placed on tissue surface. Grid resolution had a larger effect on local strain than template size. Average local strain error ranged from 3% to 9.25% and 0.1% to 2.0%, for the non-submerged and submerged conditions, respectively. Local strain estimation has a relatively high potential for error. Submerging the tissue provided superior strain estimates.     

Tracking facilitates 3-D motion estimation

The recently emerging paradigm of Active Vision advocates studying visual problems in form of modules that are directly related to a visual task for observers that are active. Along these lines, we are arguing that in many cases when an object is moving in an unrestricted manner (translation and rotation) in the 3D world, we are just interested in the motion's translational components. For a monocular observer, using only the normal flow — the spatio-temporal derivatives of the image intensity function — we solve the problem of computing the direction of translation and the time to collision. We do not use optical flow since its computation is an ill-posed problem, and in the general case it is not the same as the motion field — the projection of 3D motion on the image plane. The basic idea of our motion parameter estimation strategy lies in the employment of fixation and tracking. Fixation simplifies much of the computation by placing the object at the center of the visual field, and the main advantage of tracking is the accumulation of information over time. We show how tracking is accomplished using normal flow measurements and use it for two different tasks in the solution process. First it serves as a tool to compensate for the lack of existence of an optical flow field and thus to estimate the translation parallel to the image plane; and second it gathers information about the motion component perpendicular to the image plane.     

Ultrasound tracking for intra-fractional motion compensation in radiation therapy

Modern techniques as ion beam therapy or 4D imaging require precise target position information. However, target motion particularly in the abdomen due to respiration or patient movement is still a challenge and demands methods that detect and compensate this motion. Ultrasound represents a non-invasive, dose-free and model-independent alternative to fluoroscopy, respiration belt or optical tracking of the patient surface. Thus, ultrasound based motion tracking was integrated into irradiation with actively scanned heavy ions. In a first in vitro experiment, the ultrasound tracking system was used to compensate diverse sinusoidal target motions in two dimensions. A time delay of ∼200 ms between target motion and reported position data was compensated by a prediction algorithm (artificial neural network). The irradiated films proved feasibility of the proposed method. Furthermore, a practicable and reliable calibration workflow was developed to enable the transformation of ultrasound tracking data to the coordinates of the treatment delivery or imaging system – even if the ultrasound probe moves due to respiration. A first proof of principle experiment was performed during time-resolved positron emission tomography (4DPET) to test the calibration workflow and to show the accuracy of an ultrasound based motion tracking in vitro. The results showed that optical ultrasound tracking can reach acceptable accuracies and encourage further research.     

3D Orbital Tracking in a Modified Two-photon Microscope: An Application to the Tracking of Intracellular Vesicles

The objective of this video protocol is to discuss how to perform and analyze a three-dimensional fluorescent orbital particle tracking experiment using a modified two-photon microscope¹. As opposed to conventional approaches (raster scan or wide field based on a stack of frames), the 3D orbital tracking allows to localize and follow with a high spatial (10 nm accuracy) and temporal resolution (50 Hz frequency response) the 3D displacement of a moving fluorescent particle on length-scales of hundreds of microns². The method is based on a feedback algorithm that controls the hardware of a two-photon laser scanning microscope in order to perform a circular orbit around the object to be tracked: the feedback mechanism will maintain the fluorescent object in the center by controlling the displacement of the scanning beam^3-5. To demonstrate the advantages of this technique, we followed a fast moving organelle, the lysosome, within a living cell^6,7. Cells were plated according to standard protocols, and stained using a commercially lysosome dye. We discuss briefly the hardware configuration and in more detail the control software, to perform a 3D orbital tracking experiment inside living cells. We discuss in detail the parameters required in order to control the scanning microscope and enable the motion of the beam in a closed orbit around the particle. We conclude by demonstrating how this method can be effectively used to track the fast motion of a labeled lysosome along microtubules in 3D within a live cell. Lysosomes can move with speeds in the range of 0.4-0.5 µm/sec, typically displaying a directed motion along the microtubule network⁸.     

On the inverse problem of binocular 3D motion perception

It is shown that existing processing schemes of 3D motion perception such as interocular velocity difference, changing disparity over time, as well as joint encoding of motion and disparity, do not offer a general solution to the inverse optics problem of local binocular 3D motion. Instead we suggest that local velocity constraints in combination with binocular disparity and other depth cues provide a more flexible framework for the solution of the inverse problem. In the context of the aperture problem we derive predictions from two plausible default strategies: (1) the vector normal prefers slow motion in 3D whereas (2) the cyclopean average is based on slow motion in 2D. Predicting perceived motion directions for ambiguous line motion provides an opportunity to distinguish between these strategies of 3D motion processing. Our theoretical results suggest that velocity constraints and disparity from feature tracking are needed to solve the inverse problem of 3D motion perception. It seems plausible that motion and disparity input is processed in parallel and integrated late in the visual processing hierarchy.     

Stabilized structure from motion without disparity induces disparity adaptation

3D structures can be perceived based on the patterns of 2D motion signals. With orthographic projection of a 3D stimulus onto a 2D plane, the kinetic information can give a vivid impression of depth, but the depth order is intrinsically ambiguous, resulting in bistable or even multistable interpretations. For example, an orthographic projection of dots on the surface of a rotating cylinder is perceived as a rotating cylinder with ambiguous direction of rotation. We show that the bistable rotation can be stabilized by adding information, not to the dots themselves, but to their spatial context. More interestingly, the stabilized bistable motion can generate consistent rotation aftereffects. The rotation aftereffect can only be observed when the adapting and test stimuli are presented at the same stereo depth and the same retinal location, and it is not due to attentional tracking. The observed rotation aftereffect is likely due to direction-contingent disparity adaptation, implying that stimuli with kinetic depth may have activated neurons sensitive to different disparities, even though the stimuli have zero relative disparity. Stereo depth and kinetic depth may be supported by a common neural mechanism at an early stage in the visual system.     

A new method for motion capture of the scapula using an optoelectronic tracking device: a feasibility study

Optoelectronic tracking systems are rarely used in 3D studies examining shoulder movements including the scapula. Among the reasons is the important slippage of skin markers with respect to scapula. Methods using electromagnetic tracking devices are validated and frequently applied. Thus, the aim of this study was to develop a new method for in vivo optoelectronic scapular capture dealing with the accepted accuracy issues of validated methods.

Eleven arm positions in three anatomical planes were examined using five subjects in static mode. The method was based on local optimisation, and recalculation procedures were made using a set of five scapular surface markers.

The scapular rotations derived from the recalculation-based method yielded RMS errors comparable with the frequently used electromagnetic scapular methods (RMS up to 12.6° for 150° arm elevation). The results indicate that the present method can be used under careful considerations for 3D kinematical studies examining different shoulder movements.     

Digital stereophotogrammetry based on circular markers and zooming cameras: evaluation of a method for 3D analysis of small motions in orthopaedic research

Background  

Orthopaedic research projects focusing on small displacements in a small measurement volume require a radiation free, three dimensional motion analysis system. A stereophotogrammetrical motion analysis system can track wireless, small, light-weight markers attached to the objects. Thereby the disturbance of the measured objects through the marker tracking can be kept at minimum. The purpose of this study was to develop and evaluate a non-position fixed compact motion analysis system configured for a small measurement volume and able to zoom while tracking small round flat markers in respect to a fiducial marker which was used for the camera pose estimation.     

Prediction methods for synchronization of scanned ion beam tracking

Beam tracking as a mitigation technique for treatment of intra-fractionally moving organs requires prediction to overcome latencies in the adaptation process. We implemented and experimentally tested a prediction method for scanned carbon beam tracking. Beam tracking parameters, i.e. the shift of the Bragg peak position in 3D, are determined prior to treatment in 4D treatment planning and applied during treatment delivery in dependence on the motion state of the target as well as on the scanning spot in the target. Hence, prediction is required for the organ motion trajectory as well as the scanning progress to achieve maximal performance. Prediction algorithms to determine beam displacements that overcome these latencies were implemented. Prediction times of 25 ms for target spot prediction were required for ～6 mm water-equivalent longitudinal beam shifts. The experimental tests proved feasibility of the implemented prediction algorithm.     

In-vivo measurement of dynamic joint motion using high speed biplane radiography and CT: application to canine ACL deficiency

Dynamic assessment of three-dimensional (3D) skeletal kinematics is essential for understanding normal joint function as well as the effects of injury or disease. This paper presents a novel technique for measuring in-vivo skeletal kinematics that combines data collected from high-speed biplane radiography and static computed tomography (CT). The goals of the present study were to demonstrate that highly precise measurements can be obtained during dynamic movement studies employing high frame-rate biplane video-radiography, to develop a method for expressing joint kinematics in an anatomically relevant coordinate system and to demonstrate the application of this technique by calculating canine tibio-femoral kinematics during dynamic motion. The method consists of four components: the generation and acquisition of high frame rate biplane radiographs, identification and 3D tracking of implanted bone markers, CT-based coordinate system determination, and kinematic analysis routines for determining joint motion in anatomically based coordinates. Results from dynamic tracking of markers inserted in a phantom object showed the system bias was insignificant (-0.02 mm). The average precision in tracking implanted markers in-vivo was 0.064 mm for the distance between markers and 0.31 degree for the angles between markers. Across-trial standard deviations for tibio-femoral translations were similar for all three motion directions, averaging 0.14 mm (range 0.08 to 0.20 mm). Variability in tibio-femoral rotations was more dependent on rotation axis, with across-trial standard deviations averaging 1.71 degrees for flexion/extension, 0.90 degree for internal/external rotation, and 0.40 degree for varus/valgus rotation. Advantages of this technique over traditional motion analysis methods include the elimination of skin motion artifacts, improved tracking precision and the ability to present results in a consistent anatomical reference frame.