A computer vision based method for 3D posture estimation of symmetrical lifting

Work-related musculoskeletal disorders (WMSD) are commonly observed among the workers involved in material handling tasks such as lifting. To improve work place safety, it is necessary to assess musculoskeletal and biomechanical risk exposures associated with these tasks. Such an assessment has been mainly conducted using surface marker-based methods, which is time consuming and tedious. During the past decade, computer vision based pose estimation techniques have gained an increasing interest and may be a viable alternative for surface marker-based human movement analysis. The aim of this study is to develop and validate a computer vision based marker-less motion capture method to assess 3D joint kinematics of lifting tasks. Twelve subjects performing three types of symmetrical lifting tasks were filmed from two views using optical cameras. The joints kinematics were calculated by the proposed computer vision based motion capture method as well as a surface marker-based motion capture method. The joint kinematics estimated from the computer vision based method were practically comparable to the joint kinematics obtained by the surface marker-based method. The mean and standard deviation of the difference between the joint angles estimated by the computer vision based method and these obtained by the surface marker-based method was 2.31 ± 4.00°. One potential application of the proposed computer vision based marker-less method is to noninvasively assess 3D joint kinematics of industrial tasks such as lifting.     

Kinematical models to reduce the effect of skin artifacts on marker-based human motion estimation

The estimation of the skeletal motion obtained from marker-based motion capture systems is known to be affected by significant bias caused by skin movement artifacts, which affects joint center and rotation axis estimation. Among different techniques proposed in the literature, that based on rigid body model, still the most used by commercial motion capture systems, can smooth only part of the above effects without eliminating their main components. In order to sensibly improve the accuracy of the motion estimation, a novel technique, named local motion estimation (LME), is proposed. This rests on a recently described approach that, using virtual humans and extended Kalman filters, estimates the kinematical variables directly from 2D measurements without requiring the 3D marker reconstruction. In this paper, we show how such method can be extended to include the computation of the local marker displacement due to skin artifacts. The 3D marker coordinates, expressed in the corresponding local reference coordinate frames, are inserted into the state vector of the filter and their dynamics is automatically estimated, with adequate accuracy, without assuming any particular deformation function. Simulated experiments of lower limb motion, involving systematic mislocations (5, 10, 20 mm) and random errors of the marker coordinates and joint center locations (+/-5, +/-10, +/-15 mm), have shown that artifact motion can be substantially decoupled from the global skeletal motion with an effective increase of the accuracy wrt standard techniques. In particular, the comparison between the nominal kinematical variables and the one recovered from markers attached to the skin surface proved LME to be sensibly superior (50% in the worse condition) to the methods imposing marker-bone rigidity. In conclusion, while requiring further validation on real movement data, we argue that the proposed method can constitute an appropriate approach toward the improvement of the human motion estimation.     

Continuous ambulatory hand force monitoring during manual materials handling using instrumented force shoes and an inertial motion capture suit

Hand forces (HFs) are commonly measured during biomechanical assessment of manual materials handling; however, it is often a challenge to directly measure HFs in field studies. Therefore, in a previous study we proposed a HF estimation method based on ground reaction forces (GRFs) and body segment accelerations and tested it with laboratory equipment: GFRs were measured with force plates (FPs) and segment accelerations were measured using optical motion capture (OMC). In the current study, we evaluated the HF estimation method based on an ambulatory measurement system, consisting of inertial motion capture (IMC) and instrumented force shoes (FSs).Sixteen participants lifted and carried a 10-kg crate from ground level while 3D full-body kinematics were measured using OMC and IMC, and 3D GRFs were measured using FPs and FSs. We estimated 3D hand force vectors based on: (1) FP+OMC, (2) FP+IMC and (3) FS+IMC. We calculated the root-mean-square differences (RMSDs) between the estimated HFs to reference HFs calculated based on crate kinematics and the GRFs of a FP that the crate was lifted from.Averaged over subjects and across 3D force directions, the HF RMSD ranged between 10-15N when using the laboratory equipment (FP + OMC), 11-18N when using the IMC instead of OMC data (FP+IMC), and 17-21N when using the FSs in combination with IMC (FS + IMC). This error is regarded acceptable for the assessment of spinal loading during manual lifting, as it would results in less than 5% error in peak moment estimates.     

Static and dynamic error of a biplanar videoradiography system using marker-based and markerless tracking techniques

The use of biplanar videoradiography technology has become increasingly popular for evaluating joint function in vivo. Two fundamentally different methods are currently employed to reconstruct 3D bone motions captured using this technology. Marker-based tracking requires at least three radio-opaque markers to be implanted in the bone of interest. Markerless tracking makes use of algorithms designed to match 3D bone shapes to biplanar videoradiography data. In order to reliably quantify in vivo bone motion, the systematic error of these tracking techniques should be evaluated. Herein, we present new markerless tracking software that makes use of modern GPU technology, describe a versatile method for quantifying the systematic error of a biplanar videoradiography motion capture system using independent gold standard instrumentation, and evaluate the systematic error of the W.M. Keck XROMM Facility's biplanar videoradiography system using both marker-based and markerless tracking algorithms under static and dynamic motion conditions. A polycarbonate flag embedded with 12 radio-opaque markers was used to evaluate the systematic error of the marker-based tracking algorithm. Three human cadaveric bones (distal femur, distal radius, and distal ulna) were used to evaluate the systematic error of the markerless tracking algorithm. The systematic error was evaluated by comparing motions to independent gold standard instrumentation. Static motions were compared to high accuracy linear and rotary stages while dynamic motions were compared to a high accuracy angular displacement transducer. Marker-based tracking was shown to effectively track motion to within 0.1?mm and 0.1 deg under static and dynamic conditions. Furthermore, the presented results indicate that markerless tracking can be used to effectively track rapid bone motions to within 0.15 deg for the distal aspects of the femur, radius, and ulna. Both marker-based and markerless tracking techniques were in excellent agreement with the gold standard instrumentation for both static and dynamic testing protocols. Future research will employ these techniques to quantify in vivo joint motion for high-speed upper and lower extremity impacts such as jumping, landing, and hammering.     

A learning-based markerless approach for full-body kinematics estimation in-natura from a single image

We present a supervised machine learning approach for markerless estimation of human full-body kinematics for a cyclist from an unconstrained colour image. This approach is motivated by the limitations of existing marker-based approaches restricted by infrastructure, environmental conditions, and obtrusive markers. By using a discriminatively learned mixture-of-parts model, we construct a probabilistic tree representation to model the configuration and appearance of human body joints. During the learning stage, a Structured Support Vector Machine (SSVM) learns body parts appearance and spatial relations. In the testing stage, the learned models are employed to recover body pose via searching in a test image over a pyramid structure. We focus on the movement modality of cycling to demonstrate the efficacy of our approach. In natura estimation of cycling kinematics using images is challenging because of human interaction with a bicycle causing frequent occlusions. We make no assumptions in relation to the kinematic constraints of the model, nor the appearance of the scene. Our technique finds multiple quality hypotheses for the pose. We evaluate the precision of our method on two new datasets using loss functions. Our method achieves a score of 91.1 and 69.3 on mean Probability of Correct Keypoint (PCK) measure and 88.7 and 66.1 on the Average Precision of Keypoints (APK) measure for the frontal and sagittal datasets respectively. We conclude that our method opens new vistas to robust user-interaction free estimation of full body kinematics, a prerequisite to motion analysis.     

CAT & MAUS: A novel system for true dynamic motion measurement of underlying bony structures with compensation for soft tissue movement

Optoelectronic motion capture systems are widely employed to measure the movement of human joints. However, there can be a significant discrepancy between the data obtained by a motion capture system (MCS) and the actual movement of underlying bony structures, which is attributed to soft tissue artefact. In this paper, a computer-aided tracking and motion analysis with ultrasound (CAT & MAUS) system with an augmented globally optimal registration algorithm is presented to dynamically track the underlying bony structure during movement. The augmented registration part of CAT & MAUS was validated with a high system accuracy of 80%. The Euclidean distance between the marker-based bony landmark and the bony landmark tracked by CAT & MAUS was calculated to quantify the measurement error of an MCS caused by soft tissue artefact during movement. The average Euclidean distance between the target bony landmark measured by each of the CAT & MAUS system and the MCS alone varied from 8.32 mm to 16.87 mm in gait. This indicates the discrepancy between the MCS measured bony landmark and the actual underlying bony landmark. Moreover, Procrustes analysis was applied to demonstrate that CAT & MAUS reduces the deformation of the body segment shape modeled by markers during motion. The augmented CAT & MAUS system shows its potential to dynamically detect and locate actual underlying bony landmarks, which reduces the MCS measurement error caused by soft tissue artefact during movement.     

Validation of a single camera three-dimensional motion tracking system

The ability to analyze human movement is an essential tool of biomechanical analysis for both sport and clinical applications. Traditional 3D motion capture technology limits the feasibility of large scale data collections and therefore the ability to address clinical questions. Ideally, the measurement system/protocol should be non-invasive, mobile, generate nearly instantaneous feedback to the clinician and athlete, and be relatively inexpensive. The retro-grate reflector (RGR) is a new technology that allows for three-dimensional motion capture using a single camera. Previous studies have shown that orientation and position information recorded by the RGR system has high measurement precision and is strongly correlated with a traditional multi-camera system across a series of static poses. The technology has since been refined to record moving pose information from multiple RGR targets at sampling rates adequate for assessment of athletic movements. The purpose of this study was to compare motion data for a standard athletic movement recorded simultaneously with the RGR and multi-camera (Motion Analysis Eagle) systems. Nine subjects performed three single-leg land-and-cut maneuvers. Thigh and shank three-dimensional kinematics were collected with the RGR and Eagle camera systems simultaneously at 100 Hz. Results showed a strong agreement between the two systems in all three planes, which demonstrates the ability of the RGR system to record moving pose information from multiple RGR targets at a sampling rate adequate for assessment of human movement and supports the ability to use the RGR technology as a valid 3D motion capture system.     

Accuracy and repeatability of joint angles measured using a single camera markerless motion capture system

Markerless motion capture systems have developed in an effort to evaluate human movement in a natural setting. However, the accuracy and reliability of these systems remain understudied. Therefore, the goals of this study were to quantify the accuracy and repeatability of joint angles using a single camera markerless motion capture system and to compare the markerless system performance with that of a marker-based system. A jig was placed in multiple static postures with marker trajectories collected using a ten camera motion analysis system. Depth and color image data were simultaneously collected from a single Microsoft Kinect camera, which was subsequently used to calculate virtual marker trajectories. A digital inclinometer provided a measure of ground-truth for sagittal and frontal plane joint angles. Joint angles were calculated with marker data from both motion capture systems using successive body-fixed rotations. The sagittal and frontal plane joint angles calculated from the marker-based and markerless system agreed with inclinometer measurements by <0.5°. The systems agreed with each other by <0.5° for sagittal and frontal plane joint angles and <2° for transverse plane rotation. Both systems showed a coefficient of reliability <0.5° for all angles. These results illustrate the feasibility of a single camera markerless motion capture system to accurately measure lower extremity kinematics and provide a first step in using this technology to discern clinically relevant differences in the joint kinematics of patient populations.     

A framework for the functional identification of joint centers using markerless motion capture, validation for the hip joint

The objective of the study was to develop a framework for the accurate identification of joint centers to be used for the calculation of human body kinematics and kinetics. The present work introduces a method for the functional identification of joint centers using markerless motion capture (MMC). The MMC system used 8 color VGA cameras. An automatic segmentation-registration algorithm was developed to identify the optimal joint center in a least-square sense. The method was applied to the hip joint center with a validation study conducted in a virtual environment. The results had an accuracy (6mm mean absolute error) below the current MMC system resolution (1cm voxel resolution). Direct experimental comparison with marker-based methods was carried out showing mean absolute deviations over the three anatomical directions of 11.9 and 15.3mm if compared with either a full leg or only thigh markers protocol, respectively. Those experimental results were presented only in terms of deviations between the two systems (marker-based and markerless) as no real gold standard was available. The methods presented in this paper provide an important enabling step towards the biomechanical and clinical applications of markerless motion capture.     

Dynamic accuracy of inertial measurement units during simple pendulum motion

A motion measurement system based on inertial measurement units (IMUs) has been suggested as an alternative to contemporary video motion capture. This paper reports an investigation into the accuracy of IMUs in estimating 3D orientation during simple pendulum motion. The IMU vendor's (XSens Technologies) accuracy claim of 3 degrees root mean squared (RMS) error is tested. IMUs are integrated electronic devices that contain accelerometers, magnetometers and gyroscopes. The motion of a pendulum swing was measured using both IMUs and video motion capture as a reference. The IMU raw data were processed by the Kalman filter algorithm supplied by the vendor and a custom fusion algorithm developed by the authors. The IMU measurement of pendulum motion using the vendor's Kalman filter algorithm did not compare well with the video motion capture with a RMS error of between 8.5 degrees and 11.7 degrees depending on the length and type of pendulum swing. The maximum orientation error was greater than 30 degrees , occurring approximately eight seconds into the motion. The custom fusion algorithm estimation of orientation compared well with the video motion capture with a RMS error of between 0.8 degrees and 1.3 degrees . Future research should concentrate on developing a general purpose fusion algorithm and vendors of IMUs should provide details about the errors to be expected in different measurement situations, not just those in a 'best case' scenario.     

Using an extended kalman filter for rigid body pose estimation

Rigid body pose is commonly represented as the rigid body transformation from one (often reference) pose to another This is usually computed for each frame of data without any assumptions or restrictions on the temporal change of the pose. The most common algorithm was proposed by S?derkvist and Wedin (1993, "Determining the Movements of the Skeleton Using Well-configured Markers," J. Biomech., 26, pp. 1473-1477), and implies the assumption that measurement errors are isotropic and homogenous. This paper describes an alternative method based on a state space formulation and the application of an extended Kalman filter (EKF). State space models are formulated, which describe the kinematics of the rigid body. The state vector consists of six generalized coordinates (corresponding to the 6 degrees of freedom), and their first time derivatives. The state space models have linear dynamics, while the measurement function is a non-linear relation between the state vector and the observations (marker positions). An analytical expression for the linearized measurement function is derived. Tracking the rigid body motion using an EKF enables the use of a priori information on the measurement noise and type of motion to tune the filter. The EKF is time variant, which allows for a natural way of handling temporarily missing marker data. State updates are based on all the information available at each time step, even when data from fewer than three markers are available. Comparison with the method of S?derkvist and Wedin on simulated data showed a considerable improvement in accuracy with the proposed EKF method when marker data was temporarily missing. The proposed method offers an improvement in accuracy of rigid body pose estimation by incorporating knowledge of the characteristics of the movement and the measurement errors. Analytical expressions for the linearized system equations are provided, which eliminate the need for approximate discrete differentiation and which facilitate a fast implementation.     

Dynamic accuracy of inertial measurement units during simple pendulum motion

A motion measurement system based on inertial measurement units (IMUs) has been suggested as an alternative to contemporary video motion capture. This paper reports an investigation into the accuracy of IMUs in estimating 3D orientation during simple pendulum motion. The IMU vendor's (XSens Technologies) accuracy claim of 3° root mean squared (RMS) error is tested. IMUs are integrated electronic devices that contain accelerometers, magnetometers and gyroscopes. The motion of a pendulum swing was measured using both IMUs and video motion capture as a reference. The IMU raw data were processed by the Kalman filter algorithm supplied by the vendor and a custom fusion algorithm developed by the authors. The IMU measurement of pendulum motion using the vendor's Kalman filter algorithm did not compare well with the video motion capture with a RMS error of between 8.5° and 11.7° depending on the length and type of pendulum swing. The maximum orientation error was greater than 30°, occurring approximately eight seconds into the motion. The custom fusion algorithm estimation of orientation compared well with the video motion capture with a RMS error of between 0.8° and 1.3°. Future research should concentrate on developing a general purpose fusion algorithm and vendors of IMUs should provide details about the errors to be expected in different measurement situations, not just those in a ‘best case’ scenario.     

Two-dimensional video-based analysis of human gait using pose estimation

Human gait analysis is often conducted in clinical and basic research, but many common approaches (e.g., three-dimensional motion capture, wearables) are expensive, immobile, data-limited, and require expertise. Recent advances in video-based pose estimation suggest potential for gait analysis using two-dimensional video collected from readily accessible devices (e.g., smartphones). To date, several studies have extracted features of human gait using markerless pose estimation. However, we currently lack evaluation of video-based approaches using a dataset of human gait for a wide range of gait parameters on a stride-by-stride basis and a workflow for performing gait analysis from video. Here, we compared spatiotemporal and sagittal kinematic gait parameters measured with OpenPose (open-source video-based human pose estimation) against simultaneously recorded three-dimensional motion capture from overground walking of healthy adults. When assessing all individual steps in the walking bouts, we observed mean absolute errors between motion capture and OpenPose of 0.02 s for temporal gait parameters (i.e., step time, stance time, swing time and double support time) and 0.049 m for step lengths. Accuracy improved when spatiotemporal gait parameters were calculated as individual participant mean values: mean absolute error was 0.01 s for temporal gait parameters and 0.018 m for step lengths. The greatest difference in gait speed between motion capture and OpenPose was less than 0.10 m s⁻¹. Mean absolute error of sagittal plane hip, knee and ankle angles between motion capture and OpenPose were 4.0°, 5.6° and 7.4°. Our analysis workflow is freely available, involves minimal user input, and does not require prior gait analysis expertise. Finally, we offer suggestions and considerations for future applications of pose estimation for human gait analysis.     

A novel validation and calibration method for motion capture systems based on micro-triangulation

Motion capture systems are widely used to measure human kinematics. Nevertheless, users must consider system errors when evaluating their results. Most validation techniques for these systems are based on relative distance and displacement measurements. In contrast, our study aimed to analyse the absolute volume accuracy of optical motion capture systems by means of engineering surveying reference measurement of the marker coordinates (uncertainty: 0.75 mm). The method is exemplified on an 18 camera OptiTrack Flex13 motion capture system. The absolute accuracy was defined by the root mean square error (RMSE) between the coordinates measured by the camera system and by engineering surveying (micro-triangulation). The original RMSE of 1.82 mm due to scaling error was managed to be reduced to 0.77 mm while the correlation of errors to their distance from the origin reduced from 0.855 to 0.209. A simply feasible but less accurate absolute accuracy compensation method using tape measure on large distances was also tested, which resulted in similar scaling compensation compared to the surveying method or direct wand size compensation by a high precision 3D scanner. The presented validation methods can be less precise in some respects as compared to previous techniques, but they address an error type, which has not been and cannot be studied with the previous validation methods.     

Measuring and modeling the motor system with machine learning

The utility of machine learning in understanding the motor system is promising a revolution in how to collect, measure, and analyze data. The field of movement science already elegantly incorporates theory and engineering principles to guide experimental work, and in this review we discuss the growing use of machine learning: from pose estimation, kinematic analyses, dimensionality reduction, and closed-loop feedback, to its use in understanding neural correlates and untangling sensorimotor systems. We also give our perspective on new avenues, where markerless motion capture combined with biomechanical modeling and neural networks could be a new platform for hypothesis-driven research.     

Investigation of soft tissue movement during level walking: translations and rotations of skin markers

Skin marker-based stereophotogrammetry is the most widely used technique for human motion analysis but its accuracy is mainly limited by soft tissue artifact (STA) which reflects the non-rigidity of human body segments during activities. To compensate for the effects of STA and improve the accuracy of motion analysis, it is critical to understand the behavior and characteristics of soft tissue movement. By using a non-invasive approach, this study investigated the soft tissue movement on the thigh and shank of twenty healthy subjects during level walking which is one of the most important human daily activities and the basic content of clinical gait analysis. With the measurement of inter-marker translations and rotations on each segment, a 4D picture (3D space and time) of soft tissue deformation on the thigh and shank during walking was quantified in terms of the positional and orientational change between different skin locations. Soft tissue deformation showed nonuniform distribution at different locations as well as along different directions. The range of inter-marker movement was found to be up to 19.1mm/19.6 degrees on the thigh and 9.3mm/8.6 degrees on the shank. Results in this study provide useful information for understanding soft tissue movement behavior and exploring better marker configurations. Inter-marker movement exhibited similar patterns across subjects. This finding suggests the possibility that STA has inter-subject similarity, which is contrary to the prevailing opinion. This new insight may lead to more effective STA compensation strategies for skin marker-based motion analysis.     

Head and pelvic vertical displacement in dogs with induced swinging limb lameness: an experimental study

Background

Swinging limb lameness is defined as a motion disturbance ascribed to a limb in swing phase. Little is known about its biomechanics in dogs, particularly about the body motions that accompany it, such as vertical head and pelvic motion asymmetry. The aim of this study was to describe the changes in vertical head and pelvic motion asymmetry in dogs with induced swinging limb motion disturbance, mimicking a swinging limb lameness. Fore- and hind-limb lameness was induced in ten sound dogs by placing a weight (200 g) proximal to the carpus or tarsus, respectively. Marker-based motion capture by eight infrared light emitting video cameras recorded the dogs when trotting on a treadmill. Body symmetry parameters were calculated, including differences between the two highest positions of the head (HDmax) and pelvis (PDmax) and between the two lowest positions of the head (HDmin) and pelvis (PDmin), with a value of zero indicating perfect symmetry.

Results

Induction of swinging forelimb lameness showed significant changes in HDmax (median and range: sound 1.3 mm [??4.7 to 3.1], in the left side ??28.5 mm [??61.2 to ??17.9] and in the right side 20.1 mm [??4.4 to 47.5]) and, induction of swinging hind limb lameness showed significant changes in PDmax (sound 2.7 mm [??7.4 to 7.2], in the left side ??10.9 mm [??22.4 to 0.5] and in the right side 8.6 mm [??3 to 30]), as well as an increased hip movement asymmetry (sound 1.6 mm [??8.6 to 19.9], in the left side ??18.1 mm [??36.7 to 5.4] and in the right side 15 mm [??20.7 to 32.1]) (P?<?0.05).

Conclusions

Induced swinging fore- and hind limb lameness resulted in significant increased asymmetry of the maximal vertical displacement movement of the head and pelvis, due to decreased lifting of the head in forelimb lameness and of the pelvis in hind limb lameness. The results suggest that asymmetry of the maximal vertical displacement of the head and pelvis (i.e. lifting) is a key lameness sign to evaluate during examination of swinging limb lameness.

     

Validation of a model-based inverse kinematics approach based on wearable inertial sensors

Abstract

Wearable inertial measurement units (IMUs) are a promising solution to human motion estimation. Using IMUs 3D orientations, a model-driven inverse kinematics methodology to estimate joint angles is presented. Estimated joint angles were validated against encoder-measured kinematics (robot) and against marker-based kinematics (passive mechanism). Results are promising, with RMS angular errors respectively lower than 3 and 6?deg over a minimum range of motion of 50?deg (robot) and 160?deg (passive mechanism). Moreover, a noise robustness analysis revealed that the model-driven approach reduces the effects of experimental noises, making the proposed technique particularly suitable for application in human motion analysis.     

The efficacy of a video-based marker-less tracking system for gait analysis

Most clinical gait analyses are conducted using motion capture systems which track retro-reflective markers that are placed on key landmarks of the participants. An alternative to a three-dimensional (3D) motion capture, marker-based, optical camera system may be a marker-less video-based tracking system. The aim of our study was to investigate the efficacy of the use of a marker-less tracking system in the calculation of 3D joint angles for possible use in clinical gait analysis. Ten participants walked and jogged on a treadmill and their kinematic data were captured with a marker and marker-less tracking system simultaneously. The hip, knee and ankle angles in the frontal, sagittal and transverse planes were computed. Root Mean Square differences (RMS_diff) between corresponding angles for each participant’s support phase were calculated and averaged to derive the mean within-subject RMS_diff. These within-subject means were averaged to obtain the mean between-subject RMS_diff for the relevant joint angles in the two gait conditions (walking and jogging). The RMS_diff between the two tracking systems was less than 1° for all rotations of the three joint angles of the hip and knee. However, there were slightly larger differences in the ankle joint angles. The results of this study suggest a potential application in gait analysis in clinical settings where observations of anatomical motions may provide meaningful feedback.