Validity of a novel method to measure vertical oscillation during running using a depth camera

Recent advancements in low-cost depth cameras may provide a clinically accessible alternative to conventional three-dimensional (3D) multi-camera motion capture systems for gait analysis. However, there remains a lack of information on the validity of clinically relevant running gait parameters such as vertical oscillation (VO). The purpose of this study was to assess the validity of measures of VO during running gait using raw depth data, in comparison to a 3D multi-camera motion capture system. Sixteen healthy adults ran on a treadmill at a standard speed of 2.7 m/s. The VO of their running gait was simultaneously collected from raw depth data (Microsoft Kinect v2) and 3D marker data (Vicon multi-camera motion capture system). The agreement between the VO measures obtained from the two systems was assessed using a Bland-Altman plot with 95% limits of agreement (LOA), a Pearson’s correlation coefficient (r), and a Lin’s concordance correlation coefficient (r_c). The depth data from the Kinect v2 demonstrated excellent results across all measures of validity (r = 0.97; r_c = 0.97; 95% LOA = −8.0 mm – 8.7 mm), with an average absolute error and percent error of 3.7 (2.1) mm and 4.0 (2.0)%, respectively. The findings of this study have demonstrated the ability of a low cost depth camera and a novel tracking method to accurately measure VO in running gait.     

A Deep Neural Network-based method for estimation of 3D lifting motions

The aim of this study is developing and validating a Deep Neural Network (DNN) based method for 3D pose estimation during lifting. The proposed DNN based method addresses problems associated with marker-based motion capture systems like excessive preparation time, movement obstruction, and controlled environment requirement. Twelve healthy adults participated in a protocol and performed nine lifting tasks with different vertical heights and asymmetry angles. They lifted a crate and placed it on a shelf while being filmed by two camcorders and a synchronized motion capture system, which directly measured their body movement. A DNN with two-stage cascaded structure was designed to estimate subjects’ 3D body pose from images captured by camcorders. Our DNN augmented Hourglass network for monocular 2D pose estimation with a novel 3D pose generator subnetwork, which synthesized information from all available views to predict accurate 3D pose. We validated the results against the marker-based motion capture system as a reference and examined the method performance under different lifting conditions. The average Euclidean distance between the estimated 3D pose and reference (3D pose error) on the whole dataset was 14.72 ± 2.96 mm. Repeated measures ANOVAs showed lifting conditions can affect the method performance e.g. 60° asymmetry angle and shoulder height lifting showed higher 3D pose error compare to other lifting conditions. The results demonstrated the capability of the proposed method for 3D pose estimation with high accuracy and without limitations of marker-based motion capture systems. The proposed method may be utilized as an on-site biomechanical analysis tool.     

Accuracy and repeatability of joint angles measured using a single camera markerless motion capture system

Markerless motion capture systems have developed in an effort to evaluate human movement in a natural setting. However, the accuracy and reliability of these systems remain understudied. Therefore, the goals of this study were to quantify the accuracy and repeatability of joint angles using a single camera markerless motion capture system and to compare the markerless system performance with that of a marker-based system. A jig was placed in multiple static postures with marker trajectories collected using a ten camera motion analysis system. Depth and color image data were simultaneously collected from a single Microsoft Kinect camera, which was subsequently used to calculate virtual marker trajectories. A digital inclinometer provided a measure of ground-truth for sagittal and frontal plane joint angles. Joint angles were calculated with marker data from both motion capture systems using successive body-fixed rotations. The sagittal and frontal plane joint angles calculated from the marker-based and markerless system agreed with inclinometer measurements by <0.5°. The systems agreed with each other by <0.5° for sagittal and frontal plane joint angles and <2° for transverse plane rotation. Both systems showed a coefficient of reliability <0.5° for all angles. These results illustrate the feasibility of a single camera markerless motion capture system to accurately measure lower extremity kinematics and provide a first step in using this technology to discern clinically relevant differences in the joint kinematics of patient populations.     

Motion vectors and deep neural networks for video camera traps

Commercial camera traps are usually triggered by a Passive Infra-Red (PIR) motion sensor necessitating a delay between triggering and the image being captured. This often seriously limits the ability to record images of small and fast moving animals. It also results in many “empty” images, e.g., owing to moving foliage against a background of different temperature. In this paper we detail a new triggering mechanism based solely on the camera sensor. This is intended for use by citizen scientists and for deployment on an affordable, compact, low-power Raspberry Pi computer (RPi). Our system introduces a video frame filtering pipeline consisting of movement and image-based processing. This makes use of Machine Learning (ML) feasible on a live camera stream on an RPi. We describe our free and open-source software implementation of the system; introduce a suitable ecology efficiency measure that mediates between specificity and recall; provide ground-truth for a video clip collection from camera traps; and evaluate the effectiveness of our system thoroughly. Overall, our video camera trap turns out to be robust and effective.     

Validation of magneto-inertial measuring units for measuring hip joint angles

Camera-based motion capture systems are the current gold standard for motion analysis. However, the use of wireless inertial sensor-based systems is increasing in popularity, largely due to convenient portability. The purpose of this study was to validate the use of wireless inertial sensors for measuring hip joint motion with a functional calibration requiring only one motion (walking) and neutral standing. Data were concurrently collected using a 10-camera motion capture system and a wireless inertial sensor-based system. Hip joint angles were measured for 10 participants during walking, jumping jack, and bilateral squat tasks and for a subset (n = 5) a jump turn task. Camera-based system hip joint angles were calculated from retro-reflective marker positions and sensor-based system angles were calculated in MATLAB using the sensor output quaternions. Most hip joint angles measured with the sensor-based system were within 6° of angles measured with the camera motion capture system. Accurate measurement of motion outside of a laboratory setting has broad implications for diagnosing movement abnormalities, monitoring sports performance, and assessing rehabilitation progress.     

A reconfigurable real-time compressive-sampling camera for biological applications

Many applications in biology, such as long-term functional imaging of neural and cardiac systems, require continuous high-speed imaging. This is typically not possible, however, using commercially available systems. The frame rate and the recording time of high-speed cameras are limited by the digitization rate and the capacity of on-camera memory. Further restrictions are often imposed by the limited bandwidth of the data link to the host computer. Even if the system bandwidth is not a limiting factor, continuous high-speed acquisition results in very large volumes of data that are difficult to handle, particularly when real-time analysis is required. In response to this issue many cameras allow a predetermined, rectangular region of interest (ROI) to be sampled, however this approach lacks flexibility and is blind to the image region outside of the ROI. We have addressed this problem by building a camera system using a randomly-addressable CMOS sensor. The camera has a low bandwidth, but is able to capture continuous high-speed images of an arbitrarily defined ROI, using most of the available bandwidth, while simultaneously acquiring low-speed, full frame images using the remaining bandwidth. In addition, the camera is able to use the full-frame information to recalculate the positions of targets and update the high-speed ROIs without interrupting acquisition. In this way the camera is capable of imaging moving targets at high-speed while simultaneously imaging the whole frame at a lower speed. We have used this camera system to monitor the heartbeat and blood cell flow of a water flea (Daphnia) at frame rates in excess of 1500 fps.     

便携式运动能力检测系统的研制

目的:检测儿童的平衡能力和下肢力量,研究评估儿童运动能力的相关指标。方法:设计了一套便携式运动能力检测系统,检测儿童的平衡能力和下肢力量。本系统由运动能力检测设备和上位机数据分析存储展示部分组成。结果:研制的便携式运动能力检测系统具有可穿戴、功耗低等优点,能够准确地记录儿童的运动数据。结论:便携式运动能力检测系统能够检测使用者在运动过程中的平衡数据和下肢力量数据,及时发现运动能力的缺陷,为儿童的日常生活和锻炼提供参考。     

A novel nest‐monitoring camera system using a Raspberry Pi micro‐computer

The utility, availability, cost‐effectiveness, and reliability of prefabricated video systems designed to monitor wildlife have lagged behind the unique and varied needs of many researchers. Many systems are limited by inflexible video settings, lack of adequate data storage, and cannot be programmed by the user. More sophisticated systems can be cost prohibitive, and the literature describing remote wildlife video monitoring has, for the most part, not incorporated advances in camera and computer technology. Here, we present details of a pilot study to design and construct a lower cost (US $340) nest camera system to record the behavior of Acorn Woodpeckers (Melanerpes formicivorus) in artificial tree cavity nests. This system incorporates a Raspberry Pi micro‐computer, Pi NoIR infrared camera, a wireless adapter to transmit video over the Internet, and Deka rechargeable gel batteries for power. We programmed the system to motion‐sense, to record exclusively during daylight hours, and to automatically upload videos to the cloud over wireless Internet. The Raspberry Pi micro‐computer does not require advanced programming or electrical engineering skills to build and configure and, because it is programmable, provides unprecedented flexibility for field researchers who wish to configure the system to the specific needs of their study.     

Evaluation of optical motion information by movement detectors

Summary The paper is dealing in its first part with a system-theoretical approach for the decomposition of multi-input systems into the sum of simpler systems. By this approach the algorithm for the computations underlying the extraction of motion information from the optical environment by biological movement detectors is analysed. In the second part it concentrates on a specific model for motion computation known to be realized by the visual system of insects and of man. These motion detectors provide the visual system with information on both, velocity and structural properties of a moving pattern. The last part of the paper deals with the functional properties of two-dimensional arrays of movement detectors. They are analyzed and their relations to meaningful physiological responses are discussed.     

Computation of optical motion by movement detectors

The first part of this paper deals with a system-theoretical approach for the decomposition of multi-input systems into the sum of simpler systems. This approach is applied here to analyse the algorithm which represents the computations underlying the extraction of motion information from the optical environment by biological movement detectors. The second part concentrates on a specific model for motion computation known to be realized by the visual system of insects and of man. These detectors provide the visual system with information on both the velocity and structural properties of a moving pattern. In the third part of this article the properties of two-dimensional arrays of movement detectors are analyzed and their relations to meaningful physiological responses are discussed.     

Method for testing motion analysis laboratory measurement systems

This paper proposes a method for comparing data from accelerometers, optical based 3D motion capture systems, and force platforms (FPs) in the context of spatial and temporal differences. Testing method is based on the motion laboratory accreditation test (MLAT), which can be used to test FP and camera based motion capture components of a motion analysis laboratory. This study extends MLAT to include accelerometer data. Accelerometers were attached to a device similar to the MLAT rod. The elevation of the rod from the plane of the floor is computed and compared with the force platform vector orientation and the rod orientation obtained by optical motion capture system. Orientation of the test device is achieved by forming nonlinear equation group, which describes the components of the measured accelerations. Solution for this equation group is estimated by using the Gauss-Newton method. This expanded MLAT procedure can be used in the laboratory setting were either FP, camera based motion capture, or any other motion capture system is used along with accelerometer measurements.     

Automated Assessment of Upper Extremity Movement Impairment due to Stroke

Current diagnosis and treatment of movement impairment post-stroke is based on the subjective assessment of select movements by a trained clinical specialist. However, modern low-cost motion capture technology allows for the development of automated quantitative assessment of motor impairment. Such outcome measures are crucial for advancing post-stroke treatment methods. We sought to develop an automated method of measuring the quality of movement in clinically-relevant terms from low-cost motion capture. Unconstrained movements of upper extremity were performed by people with chronic hemiparesis and recorded by standard and low-cost motion capture systems. Quantitative scores derived from motion capture were compared to qualitative clinical scores produced by trained human raters. A strong linear relationship was found between qualitative scores and quantitative scores derived from both standard and low-cost motion capture. Performance of the automated scoring algorithm was matched by averaged qualitative scores of three human raters. We conclude that low-cost motion capture combined with an automated scoring algorithm is a feasible method to assess objectively upper-arm impairment post stroke. The application of this technology may not only reduce the cost of assessment of post-stroke movement impairment, but also promote the acceptance of objective impairment measures into routine medical practice.     

Measuring and modeling the motor system with machine learning

The utility of machine learning in understanding the motor system is promising a revolution in how to collect, measure, and analyze data. The field of movement science already elegantly incorporates theory and engineering principles to guide experimental work, and in this review we discuss the growing use of machine learning: from pose estimation, kinematic analyses, dimensionality reduction, and closed-loop feedback, to its use in understanding neural correlates and untangling sensorimotor systems. We also give our perspective on new avenues, where markerless motion capture combined with biomechanical modeling and neural networks could be a new platform for hypothesis-driven research.     

Model of a predatory stealth behaviour camouflaging motion

A computational model of a stealth strategy inspired by the apparent mating tactics of male hoverflies is presented. The stealth strategy (motion camouflage) paradoxically allows a predator to approach a moving prey in such a way that it appears to be a stationary object. In the model, the predators are controlled by neural sensorimotor systems that base their decisions on realistic levels of input information. They are shown to be able to employ motion camouflage to approach prey that move along both real hoverfly flight paths and artificially generated flight paths. The camouflaged approaches made demonstrate that the control systems have an ability to predict future prey movements. This is illustrated using two- and three-dimensional simulations.     

The evolution of patterning during movement in a large-scale citizen science game

The motion dazzle hypothesis posits that high contrast geometric patterns can cause difficulties in tracking a moving target and has been argued to explain the patterning of animals such as zebras. Research to date has only tested a small number of patterns, offering equivocal support for the hypothesis. Here, we take a genetic programming approach to allow patterns to evolve based on their fitness (time taken to capture) and thus find the optimal strategy for providing protection when moving. Our ‘Dazzle Bug’ citizen science game tested over 1.5 million targets in a touch screen game at a popular visitor attraction. Surprisingly, we found that targets lost pattern elements during evolution and became closely background matching. Modelling results suggested that targets with lower motion energy were harder to catch. Our results indicate that low contrast, featureless targets offer the greatest protection against capture when in motion, challenging the motion dazzle hypothesis.     

A three dimensional drive system for use with fillable emission phantoms for SPECT and PET imaging

Respiratory motion artefacts pose significant challenges for imaging of the lung and thorax. Dynamic phantoms have previously been applied to the study of respiratory motion, however, most moving platforms have been capable of movement in either one or two dimensions only. We describe a moving platform suitable for SPECT-CT and PET-CT imaging. The platform allows a fillable emission phantom to simulate rigid motion in three dimensions. Elliptical periodical motion of 1.5 cm in all three orthogonal planes was simulated using a series of cams moving a baseplate up and across a slope of 45°. The frequency of movement can be varied manually between 5 and 25 cycles per minute in a known calibrated and reproducible manner (This encompasses the range of physiological respiratory motion). Preliminary studies demonstrated that the phantom can be used to identify motion parameters and for the qualitative assessment of motion blurring in reconstructed images.     

Representation of planar motion of complex joints by means of rolling pairs. Application to neck motion

We propose to model planar movements between two human segments by means of rolling-without-slipping kinematic pairs. We compute the path traced by the instantaneous center of rotation (ICR) as seen from the proximal and distal segments, thus obtaining the fixed and moving centrodes, respectively. The joint motion is then represented by the rolling-without-slipping of one centrode on the other. The resulting joint kinematic model is based on the real movement and accounts for nonfixed axes of rotation; therefore it could improve current models based on revolute pairs in those cases where joint movement implies displacement of the ICR. Previous authors have used the ICR to characterize human joint motion, but they only considered the fixed centrode. Such an approach is not adequate for reproducing motion because the fixed centrode by itself does not convey information about body position. The combination of the fixed and moving centrodes gathers the kinematic information needed to reproduce the position and velocities of moving bodies. To illustrate our method, we applied it to the flexion-extension movement of the head relative to the thorax. The model provides a good estimation of motion both for position variables (mean R(pos)=0.995) and for velocities (mean R(vel)=0.958). This approach is more realistic than other models of neck motion based on revolute pairs, such as the dual-pivot model. The geometry of the centrodes can provide some information about the nature of the movement. For instance, the ascending and descending curves of the fixed centrode suggest a sequential movement of the cervical vertebrae.     

A Novel Method for Tracking Individuals of Fruit Fly Swarms Flying in a Laboratory Flight Arena

The growing interest in studying social behaviours of swarming fruit flies, Drosophila melanogaster, has heightened the need for developing tools that provide quantitative motion data. To achieve such a goal, multi-camera three-dimensional tracking technology is the key experimental gateway. We have developed a novel tracking system for tracking hundreds of fruit flies flying in a confined cubic flight arena. In addition to the proposed tracking algorithm, this work offers additional contributions in three aspects: body detection, orientation estimation, and data validation. To demonstrate the opportunities that the proposed system offers for generating high-throughput quantitative motion data, we conducted experiments on five experimental configurations. We also performed quantitative analysis on the kinematics and the spatial structure and the motion patterns of fruit fly swarms. We found that there exists an asymptotic distance between fruit flies in swarms as the population density increases. Further, we discovered the evidence for repulsive response when the distance between fruit flies approached the asymptotic distance. Overall, the proposed tracking system presents a powerful method for studying flight behaviours of fruit flies in a three-dimensional environment.     

Data acquisition and management software for camera trap data: A case study from the TEAM Network

Camera traps and the images they generate are becoming an essential tool for field biologists studying and monitoring terrestrial animals, in particular medium to large terrestrial mammals and birds. In the last five years, camera traps have made the transition to digital technology, where these devices now produce hundreds of instantly available images per month and a large amount of ancillary metadata (e.g., date, time, temperature, image size, etc.). Despite this accelerated pace in the development of digital image capture, field biologists still lack adequate software solutions to process and manage the increasing amount of information in a cost efficient way. In this paper we describe a software system that we have developed, called DeskTEAM, to address this issue. DeskTEAM has been developed in the context of the Tropical Ecology Assessment and Monitoring Network (TEAM), a global network that monitors terrestrial vertebrates. We describe the software architecture and functionality and its utility in managing and processing large amounts of digital camera trap data collected throughout the global TEAM network. DeskTEAM incorporates software features and functionality that make it relevant to the broad camera trapping community. These include the ability to run the application locally on a laptop or desktop computer, without requiring an Internet connection, as well as the ability to run on multiple operating systems; an intuitive navigational user interface with multiple levels of detail (from individual images, to whole groups of images) which allows users to easily manage hundreds or thousands of images; ability to automatically extract EXIF and custom metadata information from digital images to increase standardization; availability of embedded taxonomic lists to allow users to easily tag images with species identities; and the ability to export data packages consisting of data, metadata and images in standardized formats so that they can be transferred to online data warehouses for easy archiving and dissemination. Lastly, building these software tools for wildlife scientists provides valuable lessons for the ecoinformatics community.     

Two-dimensional video-based analysis of human gait using pose estimation

Human gait analysis is often conducted in clinical and basic research, but many common approaches (e.g., three-dimensional motion capture, wearables) are expensive, immobile, data-limited, and require expertise. Recent advances in video-based pose estimation suggest potential for gait analysis using two-dimensional video collected from readily accessible devices (e.g., smartphones). To date, several studies have extracted features of human gait using markerless pose estimation. However, we currently lack evaluation of video-based approaches using a dataset of human gait for a wide range of gait parameters on a stride-by-stride basis and a workflow for performing gait analysis from video. Here, we compared spatiotemporal and sagittal kinematic gait parameters measured with OpenPose (open-source video-based human pose estimation) against simultaneously recorded three-dimensional motion capture from overground walking of healthy adults. When assessing all individual steps in the walking bouts, we observed mean absolute errors between motion capture and OpenPose of 0.02 s for temporal gait parameters (i.e., step time, stance time, swing time and double support time) and 0.049 m for step lengths. Accuracy improved when spatiotemporal gait parameters were calculated as individual participant mean values: mean absolute error was 0.01 s for temporal gait parameters and 0.018 m for step lengths. The greatest difference in gait speed between motion capture and OpenPose was less than 0.10 m s⁻¹. Mean absolute error of sagittal plane hip, knee and ankle angles between motion capture and OpenPose were 4.0°, 5.6° and 7.4°. Our analysis workflow is freely available, involves minimal user input, and does not require prior gait analysis expertise. Finally, we offer suggestions and considerations for future applications of pose estimation for human gait analysis.