Comparison of different camera calibration approaches for underwater applications

The purpose of this study was to compare three camera calibration approaches applied to underwater applications: (1) static control points with nonlinear DLT; (2) moving wand with nonlinear camera model and bundle adjustment; (3) moving plate with nonlinear camera model. The DVideo kinematic analysis system was used for underwater data acquisition. The system consisted of two gen-locked Basler cameras working at 100 Hz, with wide angle lenses that were enclosed in housings. The accuracy of the methods was compared in a dynamic rigid bar test (acquisition volume-4.5×1×1.5 m(3)). The mean absolute errors were 6.19 mm for the nonlinear DLT, 1.16 mm for the wand calibration, 1.20 mm for the 2D plate calibration using 8 control points and 0.73 mm for the 2D plane calibration using 16 control points. The results of the wand and 2D plate camera calibration methods were less associated to the rigid body position in the working volume and provided better accuracy than the nonlinear DLT. Wand and 2D plate camera calibration methods presented similar and highly accurate results, being alternatives for underwater 3D motion analysis.     

Analysis of swimmers’ velocity during the underwater gliding motion following grab start

The purpose of this study was to determine the swimmers’ loss of speed during the underwater gliding motion of a grab start. This study also set out to determine the kinematical variables influencing this loss of speed. Eight French national-level swimmers participated in this study. The swimmers were filmed using 4 mini-DV cameras during the entire underwater phase. Using the DLT technique and the Dempster's anthropometric data, swimmer's movement have been identified. Two principal components analysis (PCA) have been used to study the relations between the kinematical variables influencing the loss of speed. The swimmers reached a velocity between 2.2 and 1.9 m s^?1 after their centre of mass covered a distance ranging between 5.63 and 6.01 m from the start wall. For this range of velocity, head position was included between 6.02 and 6.51 m. First PCA show that the kinematical parameters at the immersion (first image at which the swimmers’ whole body was under water) are included in the first two components. Second PCA show that the knee, hip and shoulder angles can be included in the same component. The present study identified the optimal instant for initiating underwater leg movements after a grab start. This study also showed that the performance during the underwater gliding motion is determined as much by variables at the immersion as by the swimmer's loss of speed. It also seems that to hold the streamlined position the synergetic action of the knee, the hip and the shoulder is essential.     

A method to minimise error in 2D-DLT reconstruction of non-planar markers filmed with a moving camera

This article describes a method that allows estimating, with the 2D version of the direct linear transformation (DLT), the actual 2D coordinates of a point when the latter is not strictly in the calibration plane. Markers placed in vertical line, above, below and in the centre of a horizontal calibration plane were filmed by a moving camera. Without correction, strong errors (up to 64.5%) were noticed for markers out of the calibration plane. After correction, calculated coordinates were consistent with actual values (error < 0.55%). The method was then applied to slip distance measurement, using a marker fixed on the hoof of a horse trotting on a calibrated track while being followed with a camera. The correction effect represented 6.6% of slip distance. Combined with the 2D-DLT transformation, the proposed corrective method allows an accurate measurement of slip distances, for high-speed outdoor locomotion analysis, using a moving camera.     

3D kinematic and dynamic analysis of the front crawl tumble turn in elite male swimmers

The aim of this study was to identify kinematic and dynamic variables related to the best tumble turn times (3mRTT, the turn time from 3-m in to 3-m out, independent variable) in ten elite male swimmers using a three-dimensional (3D) underwater analysis protocol and the Lasso (least absolute shrinkage and selection operator) as statistical method. For each swimmer, the best-time turn was analyzed with five stationary and synchronized underwater cameras. The 3D reconstruction was performed using the Direct Linear Transformation algorithm. An underwater piezoelectric 3D force platform completed the set-up to compute dynamic variables. Data were smoothed by the Savitzky-Golay filtering method. Three variables were considered relevant in the best Lasso model (3mRTT=2.58-0.425 RD+0.204 VPe+0.0046 TD): the head-wall distance where rotation starts (RD), the horizontal speed at the force peak (VPe), and the 3D length of the path covered during the turn (TD). Furthermore, bivariate analysis showed that upper body (CUBei) and lower limb extension indexes at first contact (CLLei) were also linked to the turn time (r=-0.65 and p<0.05 for both variables). Thus the best turn times were associated with a long RD, slower VPe and reduced TD. By an early transverse rotation, male elite swimmers reach the wall with a slightly flexed posture that results in fast extension. These swimmers opt for a movement that is oriented forward and they focus on reducing the distance covered.     

High-level swimmers' kinetic efficiency during the underwater phase of a grab start

The purpose of the present work was to study swimmers' efficiency during the underwater phase of the grab start. Eight high-level swimmers participated in this study. They performed two types of start: a regular grab start (with underwater leg propulsion after the glide) and a grab start with no underwater movement (swimmers had to remain in a streamlined position). Four cameras filmed the entire underwater phase of all starts. Nine anatomic landmarks were identified on the swimmers' bodies and their positions were calculated using a modified double plan DLT technique. From these positions and Dempster's anthropometric data, the center of mass position and velocity were also determined. Kinetic energies were also calculated. This velocity and kinetic energies for the two types of start were compared. Swimmers began underwater leg propulsion 1.69 m too soon. The global and internal energies were significantly higher for the start with underwater leg propulsion. Nevertheless, swimmers' velocities were equivalent for both starts. These results suggest that the swimmers did not use the underwater phase of the start efficiently: By kicking too soon, they did not succeed in producing higher velocities and thus wasted energy.     

Fast 3D reconstruction of the lower limb using a parametric model and statistical inferences and clinical measurements calculation from biplanar X-rays

In clinical routine, lower limb analysis relies on conventional X-ray (2D view) or computerised tomography (CT) Scan (lying position). However, these methods do not allow 3D analysis in standing position. The aim of this study is to propose a fast and accurate 3D-reconstruction-method based on parametric models and statistical inferences from biplanar X-rays with clinical measurements' (CM) assessment in standing position for a clinical routine use. For the reproducibility study, the 95% CI was under 2.7° for all lower limbs' angular measurements except for tibial torsion, femoral torsion and tibiofemoral rotation ( < 5°). The 95% CI were under 2.5 mm for lower limbs' lengths and 1.5 to 3° for the pelvis' CM. Comparisons between X-rays and CT-scan based 3D shapes in vitro showed mean differences of 1.0 mm (95% CI = 2.4 mm). Comparisons of 2D lower limbs' and 3D pelvis' CM between standing ‘Shifted-Feet’ and ‘Non-Shifted-Feet’ position showed means differences of 0.0 to 1.4°. Significant differences were found only for pelvic obliquity and rotation. The reconstruction time was about 5 min.     

In-air versus underwater comparison of 3D reconstruction accuracy using action sport cameras

Action sport cameras (ASC) have achieved a large consensus for recreational purposes due to ongoing cost decrease, image resolution and frame rate increase, along with plug-and-play usability. Consequently, they have been recently considered for sport gesture studies and quantitative athletic performance evaluation. In this paper, we evaluated the potential of two ASCs (GoPro Hero3+) for in-air (laboratory) and underwater (swimming pool) three-dimensional (3D) motion analysis as a function of different camera setups involving the acquisition frequency, image resolution and field of view. This is motivated by the fact that in swimming, movement cycles are characterized by underwater and in-air phases what imposes the technical challenge of having a split volume configuration: an underwater measurement volume observed by underwater cameras and an in-air measurement volume observed by in-air cameras. The reconstruction of whole swimming cycles requires thus merging of simultaneous measurements acquired in both volumes. Characterizing and optimizing the instrumental errors of such a configuration makes mandatory the assessment of the instrumental errors of both volumes.In order to calibrate the camera stereo pair, black spherical markers placed on two calibration tools, used both in-air and underwater, and a two-step nonlinear optimization were exploited. The 3D reconstruction accuracy of testing markers and the repeatability of the estimated camera parameters accounted for system performance. For both environments, statistical tests were focused on the comparison of the different camera configurations. Then, each camera configuration was compared across the two environments. In all assessed resolutions, and in both environments, the reconstruction error (true distance between the two testing markers) was less than 3mm and the error related to the working volume diagonal was in the range of 1:2000 (3×1.3×1.5 m³) to 1:7000 (4.5×2.2×1.5 m³) in agreement with the literature. Statistically, the 3D accuracy obtained in the in-air environment was poorer (p<10⁻⁵) than the one in the underwater environment, across all the tested camera configurations. Related to the repeatability of the camera parameters, we found a very low variability in both environments (1.7% and 2.9%, in-air and underwater). This result encourage the use of ASC technology to perform quantitative reconstruction both in-air and underwater environments.     

Markerless analysis of front crawl swimming

Research on motion analysis of swimmers is commonly based on video recordings of the subject's motion, which are analyzed by manual digitization of feature points by an operator. This procedure has two main drawbacks: it is time-consuming, and it is affected by low repeatability. Therefore, the application of video-based, automatic approaches to motion analysis was investigated. A video-based, markerless system for the analysis of arm movements during front crawl swimming was developed. The method proposed by Corazza et al. (2010) was modified in order to be used into water environment. Three dimensional coordinates of shoulder, elbow and wrist joints centers of 5 sprint swimmers performing front crawl swimming were determined. Wrist joint velocity was also calculated. Accuracy and reliability of the proposed technique were evaluated by means of comparison with traditional manual digitization (SIMI Reality Motion Systems GmbH). Root mean square distance (RMSD) values between trajectories estimated with the two techniques were determined. Results show good accuracy for wrist joint (RMSD<56mm), and reliability, evaluated on one subject, comparable to the inter-operator variability associated with the manual digitization procedure. The proposed technique is therefore very promising for quantitative, wide-scale studies on swimmers' motion.     

Detecting binocular 3D motion in static 3D noise: no effect of viewing distance

Relative binocular disparity cannot tell us the absolute 3D shape of an object, nor the 3D trajectory of its motion, unless the visual system has independent access to how far away the object is at any moment. Indeed, as the viewing distance is changed, the same disparate retinal motions will correspond to very different real 3D trajectories. In this paper we were interested in whether binocular 3D motion detection is affected by viewing distance. A visual search task was used, in which the observer is asked to detect a target dot, moving in 3D, amidst 3D stationary distractor dots. We found that distance does not affect detection performance. Motion-in-depth is consistently harder to detect than the equivalent lateral motion, for all viewing distances. For a constant retinal motion with both lateral and motion-in-depth components, detection performance is constant despite variations in viewing distance that produce large changes in the direction of the 3D trajectory. We conclude that binocular 3D motion detection relies on retinal, not absolute, visual signals.     

Diel changes in the movement patterns of Ganges River dolphins monitored using stationed stereo acoustic data loggers

We monitored the underwater movements of Ganges River dolphins using stationed stereo acoustic data loggers. We estimated these movements using changes in the relative angle of the sound source direction (trajectory). Of the total acoustic recordings (66 h), 26.2% contained trajectories of dolphins, and 78.6% of these trajectories involved single animals, suggesting that dolphins tended to swim alone and were localized near the monitoring station. The observed trajectories were categorized as follows: staying type characterized by small changes in the sound source direction, moving type A (moving in the same direction), and moving type B (moving up and down the stream during recording). The average interpulse intervals of sounds in moving types A and B were significantly shorter than that of the staying type, suggesting that dolphins produce the former types of trajectories to echolocate across shorter distances during movement. The frequency of occurrence of moving type A increased during the night, whereas that of type B increased in the late afternoon and that of the staying type increased during the daytime. These results indicate that dolphins moving at night tended to use short‐range echolocation, whereas during the day, they remained in relatively small areas and used long‐range sonar.     

CT-based semi-automatic quantification of vertebral fracture restoration

Minimally invasive surgeries aiming to restore fractured vertebral body are increasing; therefore, our goals were to create a 3D vertebra reconstruction process and design clinical indices to assess the vertebral restoration in terms of heights, angles and volumes. Based on computed tomography (CT)-scan of the vertebral spine, a 3D reconstruction method as well as relevant clinical indices were developed. First, a vertebra initial solution requiring 5 min of manual adjustments is built. Then an image processing algorithm places this solution in the CT-scan images volume to adjust the model's nodes. On the vertebral body's anterior and posterior parts, nine robust heights, volume and endplate angle measurement methods were developed. These parameters were evaluated by reproducibility and accuracy studies. The vertebral body reconstruction accuracy was 1.0 mm; heights and volume accuracy were, respectively, 1.2 and 179 mm³. In conclusion, a 3D vertebra reconstruction process requiring little user time was proposed as well as 3D clinical indices assessing fractured and restored vertebra.     

Misperceptions in the trajectories of objects undergoing curvilinear motion

Trajectory perception is crucial in scene understanding and action. A variety of trajectory misperceptions have been reported in the literature. In this study, we quantify earlier observations that reported distortions in the perceived shape of bilinear trajectories and in the perceived positions of their deviation. Our results show that bilinear trajectories with deviation angles smaller than 90 deg are perceived smoothed while those with deviation angles larger than 90 degrees are perceived sharpened. The sharpening effect is weaker in magnitude than the smoothing effect. We also found a correlation between the distortion of perceived trajectories and the perceived shift of their deviation point. Finally, using a dual-task paradigm, we found that reducing attentional resources allocated to the moving target causes an increase in the perceived shift of the deviation point of the trajectory. We interpret these results in the context of interactions between motion and position systems.     

A novel supervised trajectory segmentation algorithm identifies distinct types of human adenovirus motion in host cells

Biological trajectories can be characterized by transient patterns that may provide insight into the interactions of the moving object with its immediate environment. The accurate and automated identification of trajectory motifs is important for the understanding of the underlying mechanisms. In this work, we develop a novel trajectory segmentation algorithm based on supervised support vector classification. The algorithm is validated on synthetic data and applied to the identification of trajectory fingerprints of fluorescently tagged human adenovirus particles in live cells. In virus trajectories on the cell surface, periods of confined motion, slow drift, and fast drift are efficiently detected. Additionally, directed motion is found for viruses in the cytoplasm. The algorithm enables the linking of microscopic observations to molecular phenomena that are critical in many biological processes, including infectious pathogen entry and signal transduction.     

Improved RSA accuracy with DLT and balanced calibration marker distributions with an assessment of initial-calibration

Roentgen stereophotogrammetric analysis (RSA) has been used for over 25 years for accurate micromotion measurement in a wide variety of orthopaedic applications. This study investigated two possible improvements to the method. First, direct linear transformation (DLT) was compared to the traditional RSA reconstruction algorithm. The two methods were considered with respect to standard extrapolation and interpolation calibration cages. Matlab simulations showed that reconstruction accuracy was greatly improved (>60%) by combining DLT with an even distribution of enclosing calibration markers. Second, a benchtop study using phantoms translated at 0.0254-mm intervals showed initial-calibration, followed by removal of the interpolation cage for subsequent exposures, was potentially twice as accurate as self-calibration with an extrapolation cage. These results showed optimizations for the application of RSA when unobstructed space is required.     

Evaluation and calibration of an electromagnetic tracking device for biomechanical analysis of lifting tasks.

Electromagnetic motion tracking devices are increasingly used as a kinematic measuring tool. The aim of this study was to evaluate a long-range transmitter in an environment with a conventional force plate present in order to assess its suitability for further biomechanical applications. Using a calibration apparatus developed in our lab and Optotrack measurements, the performances of the Motion Star were evaluated. Positions and orientations were measured in a 140 x 80 x 120 cm(3) space centered on the force plate. Using a mathematical model developed at Queen's University, these data were calibrated. Errors on position and orientation were less than 150 mm and 10 degrees before calibration of the Motion Star, and less than 20mm and 2 degrees after calibration, with no differences between data collected with the force plate switched on/off. These errors did not depend on sensor orientation. Variability of the signal was small indicating minimal noise. Field distortion was the largest source of measurement error, which increased with the distance between the transmitter and the sensor and the proximity of the sensor to the force plate. Before its use for biomechanical analysis of lifting tasks and validation of dynamic models using force plate data, the data from electromagnetic motion tracking devices must be calibrated to decrease the errors due to electromagnetic field distortion.     

A fluid-dynamic interpretation of the asymmetric motion of singly flagellated bacteria swimming close to a boundary

The singly flagellated bacterium, Vibrio alginolyticus, moves forward and backward by alternating the rotational direction of its flagellum. The bacterium has been observed retracing a previous path almost exactly and swimming in a zigzag pattern. In the presence of a boundary, however, the motion changes significantly, to something closer to a circular trajectory. Additionally, when the cell swims close to a wall, the forward and backward speeds differ noticeably. This study details a boundary element model for the motion of a bacterium swimming near a rigid boundary and the results of numerical analyses conducted using this model. The results reveal that bacterium motion is apparently influenced by pitch angle, i.e., the angle between the boundary and the swimming direction, and that forward motion is more stable than backward motion with respect to pitching of the bacterium. From these results, a set of diagrammatic representations have been created that explain the observed asymmetry in trajectory and speed between the forward and backward motions. For forward motion, a cell moving parallel to the boundary will maintain this trajectory. However, for backward motion, the resulting trajectory depends upon whether the bacterium is approaching or departing the boundary. Fluid-dynamic interactions between the flagellum and the boundary vary with cell orientation and cause peculiarities in the resulting trajectories.     

Irregular breathing during 4DCT scanning of lung cancer patients: Is the midventilation approach robust?

BackgroundWith 4DCT the risk of introducing positional systematic errors in lung cancer radiotherapy can be minimised. A common approach is to plan on the phase bin of the 4DCT best representing the tumour's time-weighted mean position also called the midventilation scan. However breathing irregularities can introduce uncertainties and potentially misrepresent both the tumour trajectory and the determination of the midventilation phase. In this study we evaluated the robustness of the midventilation approach in the presence of irregular breathing patterns.MethodsA LEGO Mindstorms^® phantom with compact balls simulating lung tumours was constructed. The breathing curves loaded in the phantom were either acquired from a human volunteer or constructed with various magnitudes (ranging from 12 to 29 mm) as well as various irregularities of motion pattern. Repeated 4DCT scans were performed while tumour trajectories were recorded with two motion tracking systems.ResultsThe time-weighted mean tumour position is accurately represented in 4DCT scans, even for irregular breathing patterns: the position presentation in the midventilation scan was always within in one standard deviation of the global position presentation (3 mm and 2 mm for regular and irregular breathing patterns, respectively). The displacement representation tended to be underestimated in 4DCT scans.ConclusionThe midventilation approach is robust even in the presence of breathing irregularity. The representation of the tumour trajectory in 4DCT scans is affected by breathing irregularity and the extent of tumour motion can be underestimated, which will affect the calculation of patient-individualised margins based on the 4DCT scan.     

Tracking of Ball and Players in Beach Volleyball Videos

This paper presents methods for the determination of players'' positions and contact time points by tracking the players and the ball in beach volleyball videos. Two player tracking methods are compared, a classical particle filter and a rigid grid integral histogram tracker. Due to mutual occlusion of the players and the camera perspective, results are best for the front players, with 74,6% and 82,6% of correctly tracked frames for the particle method and the integral histogram method, respectively. Results suggest an improved robustness against player confusion between different particle sets when tracking with a rigid grid approach. Faster processing and less player confusions make this method superior to the classical particle filter. Two different ball tracking methods are used that detect ball candidates from movement difference images using a background subtraction algorithm. Ball trajectories are estimated and interpolated from parabolic flight equations. The tracking accuracy of the ball is 54,2% for the trajectory growth method and 42,1% for the Hough line detection method. Tracking results of over 90% from the literature could not be confirmed. Ball contact frames were estimated from parabolic trajectory intersection, resulting in 48,9% of correctly estimated ball contact points.     

The static accuracy and calibration of inertial measurement units for 3D orientation

Inertial measurement units (IMUs) are integrated electronic devices that contain accelerometers, magnetometers and gyroscopes. Wearable motion capture systems based on IMUs have been advertised as alternatives to optical motion capture. In this paper, the accuracy of five different IMUs of the same type in measuring 3D orientation in static situations, as well as the calibration of the accelerometers and magnetometers within the IMUs, has been investigated. The maximum absolute static orientation error was 5.2°, higher than the 1° claimed by the vendor. If the IMUs are re-calibrated at the time of measurement with the re-calibration procedure described in this paper, it is possible to obtain an error of less than 1°, in agreement with the vendor's specifications (XSens Technologies B.V. 2005. Motion tracker technical documentation Mtx-B. Version 1.03. Available from: www.xsens.com).

The new calibration appears to be valid for at least 22 days providing the sensor is not exposed to high impacts. However, if several sensors are ‘daisy chained’ together changes to the magnetometer bias can cause heading errors of up to 15°. The results demonstrate the non-linear relationship between the vendor's orthogonality claim of < 0.1° and the accuracy of 3D orientation obtained from factory calibrated IMUs in static situations. The authors hypothesise that the high magnetic dip (64°) in our laboratory may have exacerbated the errors reported. For biomechanical research, small relative movements of a body segment from a calibrated position are likely to be more accurate than large scale global motion that may have an error of up to 9.8°.     

A comparison of calibration methods for stereo fluoroscopic imaging systems

Stereo (biplane) fluoroscopic imaging systems are considered the most accurate and precise systems to study joint kinematics in vivo. Calibration of a biplane fluoroscopy system consists of three steps: (1) correction for spatial image distortion; (2) calculation of the focus position; and (3) calculation of the relative position and orientation of the two fluoroscopy systems with respect to each other. In this study we compared 6 methods for calibrating a biplane fluoroscopy system including a new method using a novel nested-optimization technique. To quantify bias and precision, an electronic digital caliper instrumented with two tantalum markers on radiolucent posts was imaged in three configurations, and for each configuration placed in ten static poses distributed throughout the viewing volume. Bias and precision were calculated as the mean and standard deviation of the displacement of the markers measured between the three caliper configurations. The data demonstrated that it is essential to correct for image distortion when sub-millimeter accuracy is required. We recommend calibrating a stereo fluoroscopic imaging system using an accurately machined plate and a calibration cube, which improved accuracy 2-3 times compared to the other calibration methods. Once image distortion is properly corrected, the focus position should be determined using the Direct Linear Transformation (DLT) method for its increased speed and equivalent accuracy compared to the novel nested-optimization method. The DLT method also automatically provides the 3D fluoroscopy configuration. Using the recommended calibration methodology, bias and precision of 0.09 and 0.05 mm or better can be expected for measuring inter-marker distances.