3D HUMAN MOTION RETRIEVAL BASED ON HUMAN HIERARCHICAL INDEX STRUCTURE

With the development and wide application of motion capture technology, the captured motion data sets are becoming larger and larger. For this reason, an efficient retrieval method for the motion database is very important. The retrieval method needs an appropriate indexing scheme and an effective similarity measure that can organize the existing motion data well. In this paper, we represent a human motion hierarchical index structure and adopt a nonlinear method to segment motion sequences. Based on this, we extract motion patterns and then we employ a fast similarity measure algorithm for motion pattern similarity computation to efficiently retrieve motion sequences. The experiment results show that the approach proposed in our paper is effective and efficient.     

Accuracy map of an optical motion capture system with 42 or 21 cameras in a large measurement volume

Optical motion capture is commonly used in biomechanics to measure human kinematics. However, no studies have yet examined the accuracy of optical motion capture in a large capture volume (>100 m³), or how accuracy varies from the center to the extreme edges of the capture volume. This study measured the dynamic 3D errors of an optical motion capture system composed of 42 OptiTrack Prime 41 cameras (capture volume of 135 m³) by comparing the motion of a single marker to the motion reported by a ThorLabs linear motion stage. After spline interpolating the data, it was found that 97% of the capture area had error below 200 μm. When the same analysis was performed using only half (21) of the cameras, 91% of the capture area was below 200 μm of error. The only locations that exceeded this threshold were at the extreme edges of the capture area, and no location had a mean error exceeding 1 mm. When measuring human kinematics with skin-mounted markers, uncertainty of marker placement relative to underlying skeletal features and soft tissue artifact produce errors that are orders of magnitude larger than the errors attributed to the camera system itself. Therefore, the accuracy of this OptiTrack optical motion capture system was found to be more than sufficient for measuring full-body human kinematics with skin-mounted markers in a large capture volume (>100 m³).     

Direct evidence for encoding of motion streaks in human visual cortex

Temporal integration in the visual system causes fast-moving objects to generate static, oriented traces (‘motion streaks’), which could be used to help judge direction of motion. While human psychophysics and single-unit studies in non-human primates are consistent with this hypothesis, direct neural evidence from the human cortex is still lacking. First, we provide psychophysical evidence that faster and slower motions are processed by distinct neural mechanisms: faster motion raised human perceptual thresholds for static orientations parallel to the direction of motion, whereas slower motion raised thresholds for orthogonal orientations. We then used functional magnetic resonance imaging to measure brain activity while human observers viewed either fast (‘streaky’) or slow random dot stimuli moving in different directions, or corresponding static-oriented stimuli. We found that local spatial patterns of brain activity in early retinotopic visual cortex reliably distinguished between static orientations. Critically, a multivariate pattern classifier trained on brain activity evoked by these static stimuli could then successfully distinguish the direction of fast (‘streaky’) but not slow motion. Thus, signals encoding static-oriented streak information are present in human early visual cortex when viewing fast motion. These experiments show that motion streaks are present in the human visual system for faster motion.     

Dynamic accuracy of inertial measurement units during simple pendulum motion

A motion measurement system based on inertial measurement units (IMUs) has been suggested as an alternative to contemporary video motion capture. This paper reports an investigation into the accuracy of IMUs in estimating 3D orientation during simple pendulum motion. The IMU vendor's (XSens Technologies) accuracy claim of 3° root mean squared (RMS) error is tested. IMUs are integrated electronic devices that contain accelerometers, magnetometers and gyroscopes. The motion of a pendulum swing was measured using both IMUs and video motion capture as a reference. The IMU raw data were processed by the Kalman filter algorithm supplied by the vendor and a custom fusion algorithm developed by the authors. The IMU measurement of pendulum motion using the vendor's Kalman filter algorithm did not compare well with the video motion capture with a RMS error of between 8.5° and 11.7° depending on the length and type of pendulum swing. The maximum orientation error was greater than 30°, occurring approximately eight seconds into the motion. The custom fusion algorithm estimation of orientation compared well with the video motion capture with a RMS error of between 0.8° and 1.3°. Future research should concentrate on developing a general purpose fusion algorithm and vendors of IMUs should provide details about the errors to be expected in different measurement situations, not just those in a ‘best case’ scenario.     

Automated Assessment of Upper Extremity Movement Impairment due to Stroke

Current diagnosis and treatment of movement impairment post-stroke is based on the subjective assessment of select movements by a trained clinical specialist. However, modern low-cost motion capture technology allows for the development of automated quantitative assessment of motor impairment. Such outcome measures are crucial for advancing post-stroke treatment methods. We sought to develop an automated method of measuring the quality of movement in clinically-relevant terms from low-cost motion capture. Unconstrained movements of upper extremity were performed by people with chronic hemiparesis and recorded by standard and low-cost motion capture systems. Quantitative scores derived from motion capture were compared to qualitative clinical scores produced by trained human raters. A strong linear relationship was found between qualitative scores and quantitative scores derived from both standard and low-cost motion capture. Performance of the automated scoring algorithm was matched by averaged qualitative scores of three human raters. We conclude that low-cost motion capture combined with an automated scoring algorithm is a feasible method to assess objectively upper-arm impairment post stroke. The application of this technology may not only reduce the cost of assessment of post-stroke movement impairment, but also promote the acceptance of objective impairment measures into routine medical practice.     

Design of a marker-based human motion tracking system

In this paper a complete design of a high speed optical motion analyzer system has been described. The main core of the image processing unit has been implemented by the differential algorithm procedure. Some intelligent and conservative procedures that facilitate the search algorithm have also been proposed and implemented for the processing of human motions. Moreover, an optimized modified direct linear transformation (MDLT) method has been used to reconstruct 3D markers positions which are used for deriving kinematic characteristics of the motion. Consequently, a set of complete tests using some simple mechanical devices were conducted to verify the system outputs. Considering the system verification for human motion analysis, we used the system for gait analysis and the results including joint angles showed good compatibility with other investigations. Furthermore, a sport application example of the system has been quantitatively presented and discussed for Iranian National Karate-kas. The low computational cost, the high precision in detecting and reconstructing marker position with 2.39 mm error, and the capability of capturing from any number of cameras to increase the domain of operation of the subject, has made the proposed method a reliable approach for real-time human motion analysis. No special environment limitation, portability, low cost hardware and built in units for simulations and kinematic analysis are the other significant specifications of this system.     

Feedback control in active sensing: rat exploratory whisking is modulated by environmental contact

Rats sweep their facial whiskers back and forth to generate tactile sensory information through contact with environmental structure. The neural processes operating on the signals arising from these whisker contacts are widely studied as a model of sensing in general, even though detailed knowledge of the natural circumstances under which such signals are generated is lacking. We used digital video tracking and wireless recording of mystacial electromyogram signals to assess the effects of whisker-object contact on whisking in freely moving animals exploring simple environments. Our results show that contact leads to reduced protraction (forward whisker motion) on the side of the animal ipsilateral to an obstruction and increased protraction on the contralateral side. Reduced ipsilateral protraction occurs rapidly and in the same whisk cycle as the initial contact. We conclude that whisker movements are actively controlled so as to increase the likelihood of environmental contacts while constraining such interactions to involve a gentle touch. That whisking pattern generation is under strong feedback control has important implications for understanding the nature of the signals reaching upstream neural processes.     

Supervised Neural Q_learning based Motion Control for Bionic Underwater Robots

Bionic underwater robots have been a hot research area in recent years. The motion control methods for a kind of bionic underwater robot with two undulating fins are discussed in this paper. The equations of motion for the bionic underwater robot are described. To apply the reinforcement learning to the actual robot control, a Supervised Neural Q_learning (SNQL) algorithm is put forward. This algorithm is based on conventional Q_learning algorithm, but has three remarkable distinctions: (1) using a feedforward neural network to approximate the Q_function table; (2) adopting a learning sample database to speed up learning and improve the stability of learning system; (3) introducing a supervised control in the earlier stage of learning for safety and to speed up learning again. Experiments of swimming straightforward are carried out with SNQL algorithm. Results indicate that the SNQL algorithm is more effective than pure neural Q_learning or supervised control. It is a feasible approach to figure out the motion control for bionic underwater robots.     

人体无支撑运动的数学模型

基于Hanavan[1]人体模型,本文建立了人体空中运动的一般数学模型,并利用微分方程数值解法对某些实际运动进行了模拟．结果表明该模型能用于反映人体空中无支撑运动的运动过程,并可应用于宇航、体操、技巧、跳水、舞蹈等等运动的研究．     

Estimating joint kinematics from skin motion observation: modelling and validation

Modelling of soft tissue motion is required in many areas, such as computer animation, surgical simulation, 3D motion analysis and gait analysis. In this paper, we will focus on the use of modelling of skin deformation during 3D motion analysis. The most frequently used method in 3D human motion analysis involves placing markers on the skin of the analysed segment which is composed of the rigid bone and the surrounding soft tissues. Skin and soft tissue deformations introduce a significant artefact which strongly influences the resulting bone position, orientation and joint kinematics. For this study, we used a statistical solid dynamics approach which is a combination of several previously reported tools: the point cluster technique (PCT) and a Kalman filter which was added to the PCT. The methods were tested and evaluated on controlled human-arm motions, using an optical motion capture system (Vicon^TM).

The addition of a Kalman filter to the PCT for rigid body motion estimation results in a smoother signal that better represents the joint motion. Calculations indicate less signal distortion than when using a digital low-pass filter. Furthermore, adding a Kalman filter to the PCT substantially reduces the dispersion of the maximal and minimal instantaneous frequencies. For controlled human movements, the result indicated that adding a Kalman filter to the PCT produced a more accurate signal. However, it could not be concluded that the proposed Kalman filter is better than a low-pass filter for estimation of the motion. We suggest that implementation of a Kalman filter with a better biomechanical motion model will be more likely to improve the results.     

Multifunctional Setup for Studying Human Motor Control Using Transcranial Magnetic Stimulation,Electromyography, Motion Capture,and Virtual Reality

The study of neuromuscular control of movement in humans is accomplished with numerous technologies. Non-invasive methods for investigating neuromuscular function include transcranial magnetic stimulation, electromyography, and three-dimensional motion capture. The advent of readily available and cost-effective virtual reality solutions has expanded the capabilities of researchers in recreating “real-world” environments and movements in a laboratory setting. Naturalistic movement analysis will not only garner a greater understanding of motor control in healthy individuals, but also permit the design of experiments and rehabilitation strategies that target specific motor impairments (e.g. stroke). The combined use of these tools will lead to increasingly deeper understanding of neural mechanisms of motor control. A key requirement when combining these data acquisition systems is fine temporal correspondence between the various data streams. This protocol describes a multifunctional system’s overall connectivity, intersystem signaling, and the temporal synchronization of recorded data. Synchronization of the component systems is primarily accomplished through the use of a customizable circuit, readily made with off the shelf components and minimal electronics assembly skills.     

Robotics-based synthesis of human motion

The synthesis of human motion is a complex procedure that involves accurate reconstruction of movement sequences, modeling of musculoskeletal kinematics, dynamics and actuation, and characterization of reliable performance criteria. Many of these processes have much in common with the problems found in robotics research. Task-based methods used in robotics may be leveraged to provide novel musculoskeletal modeling methods and physiologically accurate performance predictions. In this paper, we present (i) a new method for the real-time reconstruction of human motion trajectories using direct marker tracking, (ii) a task-driven muscular effort minimization criterion and (iii) new human performance metrics for dynamic characterization of athletic skills. Dynamic motion reconstruction is achieved through the control of a simulated human model to follow the captured marker trajectories in real-time. The operational space control and real-time simulation provide human dynamics at any configuration of the performance. A new criteria of muscular effort minimization has been introduced to analyze human static postures. Extensive motion capture experiments were conducted to validate the new minimization criterion. Finally, new human performance metrics were introduced to study in details an athletic skill. These metrics include the effort expenditure and the feasible set of operational space accelerations during the performance of the skill. The dynamic characterization takes into account skeletal kinematics as well as muscle routing kinematics and force generating capacities. The developments draw upon an advanced musculoskeletal modeling platform and a task-oriented framework for the effective integration of biomechanics and robotics methods.     

Neural adaptive control of nonlinear multivariable systems with application to a class of inverted pendulums

In this paper multilayer neural networks (MNNs) are used to control the balancing of a class of inverted pendulums. Unlike normal inverted pendulums, the pendulum discussed here has two degrees of rotational freedom and the base-point moves randomly in three-dimensional space. The goal is to apply control torques to keep the pendulum in a prescribed position in spite of the random movement at the base-point. Since the inclusion of the base-point motion leads to a non-autonomous dynamic system with time-varying parametric excitation, the design of the control system is a challenging task. A feedback control algorithm is proposed that utilizes a set of neural networks to compensate for the effect of the system's nonlinearities. The weight parameters of neural networks updated on-line, according to a learning algorithm that guarantees the Lyapunov stability of the control system. Furthermore, since the base-point movement is considered unmeasurable, a neural inverse model is employed to estimate it from only measured state variables. The estimate is then utilized within the main control algorithm to produce compensating control signals. The examination of the proposed control system, through simulations, demonstrates the promise of the methodology and exhibits positive aspects, which cannot be achieved by the previously developed techniques on the same problem. These aspects include fast, yet well-maintained damped responses with reasonable control torques and no requirement for knowledge of the model or the model parameters. The work presented here can benefit practical problems such as the study of stable locomotion of human upper body and bipedal robots.     

Simulating the approximate irregular field dose distribution in radiotherapy using an ultrasound tracking technique

PurposeThis study used an ultrasound image tracking algorithm (UITA) in combination with a proposed simulation program for the approximate irregular field dose distribution (SPAD) to assess the feasibility of performing dose distribution simulations for two-dimensional radiotherapy.MethodsThis study created five different types of multileaf collimator openings, and applied a SPAD to analyze the matrix position parameters for each regular field to generate a static program-simulation dose distribution map (PDDM), whose similarity was then compared with a static radiochromic film experimental-measurement dose distribution map (EDDM). A two-dimensional respiration motion simulation system (RMSS) was used to reproduce the respiration motion, and the UITA was used to capture the respiration signals. Respiration signals were input to the SPAD to generate two dynamic PDDMs, which were compared for similarity with the dynamic EDDM.ResultsIn order to verify the dose distribution between different dose measurement techniques, the gamma passing rate with 2%/2 mm criterion was used for the EDDM and PDDM, the passing rates were between 94.31% and 99.71% in the static field analyses, and between 84.45% and 96.09% for simulations with the UITA signal input and between 89.35% and 97.78% for simulations with the original signal input in the dynamic field analyses.ConclusionsStatic and dynamic dose distribution maps can be simulated based on the proposed matrix position parameters of various fields and by using the UITA to track respiration signals during radiation therapy. The present findings indicate that it is possible to develop a reusable and time-saving dose distribution measurement tool.     

Two-dimensional video-based analysis of human gait using pose estimation

Human gait analysis is often conducted in clinical and basic research, but many common approaches (e.g., three-dimensional motion capture, wearables) are expensive, immobile, data-limited, and require expertise. Recent advances in video-based pose estimation suggest potential for gait analysis using two-dimensional video collected from readily accessible devices (e.g., smartphones). To date, several studies have extracted features of human gait using markerless pose estimation. However, we currently lack evaluation of video-based approaches using a dataset of human gait for a wide range of gait parameters on a stride-by-stride basis and a workflow for performing gait analysis from video. Here, we compared spatiotemporal and sagittal kinematic gait parameters measured with OpenPose (open-source video-based human pose estimation) against simultaneously recorded three-dimensional motion capture from overground walking of healthy adults. When assessing all individual steps in the walking bouts, we observed mean absolute errors between motion capture and OpenPose of 0.02 s for temporal gait parameters (i.e., step time, stance time, swing time and double support time) and 0.049 m for step lengths. Accuracy improved when spatiotemporal gait parameters were calculated as individual participant mean values: mean absolute error was 0.01 s for temporal gait parameters and 0.018 m for step lengths. The greatest difference in gait speed between motion capture and OpenPose was less than 0.10 m s⁻¹. Mean absolute error of sagittal plane hip, knee and ankle angles between motion capture and OpenPose were 4.0°, 5.6° and 7.4°. Our analysis workflow is freely available, involves minimal user input, and does not require prior gait analysis expertise. Finally, we offer suggestions and considerations for future applications of pose estimation for human gait analysis.     

Quantification of gait parameters with inertial sensors and inverse kinematics

Measuring human gait is important in medicine to obtain outcome parameter for therapy, for instance in Parkinson’s disease. Recently, small inertial sensors became available which allow for the registration of limb-position outside of the limited space of gait laboratories. The computation of gait parameters based on such recordings has been the subject of many scientific papers. We want to add to this knowledge by presenting a 4-segment leg model which is based on inverse kinematic and Kalman filtering of data from inertial sensors. To evaluate the model, data from four leg segments (shanks and thighs) were recorded synchronously with accelerometers and gyroscopes and a 3D motion capture system while subjects (n = 12) walked at three different velocities on a treadmill. Angular position of leg segments was computed from accelerometers and gyroscopes by Kalman filtering and compared to data from the motion capture system. The four-segment leg model takes the stance foot as a pivotal point and computes the position of the remaining segments as a kinematic chain (inverse kinematics). Second, we evaluated the contribution of pelvic movements to the model and evaluated a five segment model (shanks, thighs and pelvis) against ground-truth data from the motion capture system and the path of the treadmill.ResultsWe found the precision of the Kalman filtered angular position is in the range of 2–6° (RMS error). The 4-segment leg model computed stride length and length of gait path with a constant undershoot of 3% for slow and 7% for fast gait. The integration of a 5th segment (pelvis) into the model increased its precision. The advantages of this model and ideas for further improvements are discussed.     

Differentiation of reprogrammed human adipose mesenchymal stem cells toward neural cells with defined transcription factors

Somatic cell reprogramming may become a powerful approach to generate specific human cell types for cell-fate determination studies and potential transplantation therapies of neurological diseases. Here we report a reprogramming methodology with which human adipose stem cells (hADSCs) can be differentiated into neural cells. After being reprogrammed with polycistronic plasmid carrying defined factor OCT3/4, SOX2, KLF4 and c-MYC, and further treated with neural induce medium, the hADSCs switched to differentiate toward neural cell lineages. The generated cells had normal karyotypes and exogenous vector sequences were not inserted in the genomes. Therefore, this cell lineage conversion methodology bypasses the risk of mutation and gene instability, and provides a novel strategy to obtain patient-specific neural cells for basic research and therapeutic application.     

Validation of magneto-inertial measuring units for measuring hip joint angles

Camera-based motion capture systems are the current gold standard for motion analysis. However, the use of wireless inertial sensor-based systems is increasing in popularity, largely due to convenient portability. The purpose of this study was to validate the use of wireless inertial sensors for measuring hip joint motion with a functional calibration requiring only one motion (walking) and neutral standing. Data were concurrently collected using a 10-camera motion capture system and a wireless inertial sensor-based system. Hip joint angles were measured for 10 participants during walking, jumping jack, and bilateral squat tasks and for a subset (n = 5) a jump turn task. Camera-based system hip joint angles were calculated from retro-reflective marker positions and sensor-based system angles were calculated in MATLAB using the sensor output quaternions. Most hip joint angles measured with the sensor-based system were within 6° of angles measured with the camera motion capture system. Accurate measurement of motion outside of a laboratory setting has broad implications for diagnosing movement abnormalities, monitoring sports performance, and assessing rehabilitation progress.     

暗示性运动加工的认知神经机制

暗示性运动是指个体观看静止图片时从中知觉到的运动.研究者采用高低认知水平两类暗示性运动刺激材料,借助"冻结帧"、直接观看、运动后效和f MRI适应等任务范式,探讨了注意和意识在暗示性运动加工中的作用及其记忆特点;并借助脑成像等技术,考察了颞中区、颞上皮层区、颞上沟、镜像神经元系统等脑区在暗示性运动加工中的作用.但由于暗示性运动加工涉及"视觉腹侧通路与背侧通路功能的分离与整合"问题,目前对相关研究结果和解释还存在争议,暗示性运动加工的认知神经机制仍有待于进一步研究.     

A computer vision based method for 3D posture estimation of symmetrical lifting

Work-related musculoskeletal disorders (WMSD) are commonly observed among the workers involved in material handling tasks such as lifting. To improve work place safety, it is necessary to assess musculoskeletal and biomechanical risk exposures associated with these tasks. Such an assessment has been mainly conducted using surface marker-based methods, which is time consuming and tedious. During the past decade, computer vision based pose estimation techniques have gained an increasing interest and may be a viable alternative for surface marker-based human movement analysis. The aim of this study is to develop and validate a computer vision based marker-less motion capture method to assess 3D joint kinematics of lifting tasks. Twelve subjects performing three types of symmetrical lifting tasks were filmed from two views using optical cameras. The joints kinematics were calculated by the proposed computer vision based motion capture method as well as a surface marker-based motion capture method. The joint kinematics estimated from the computer vision based method were practically comparable to the joint kinematics obtained by the surface marker-based method. The mean and standard deviation of the difference between the joint angles estimated by the computer vision based method and these obtained by the surface marker-based method was 2.31 ± 4.00°. One potential application of the proposed computer vision based marker-less method is to noninvasively assess 3D joint kinematics of industrial tasks such as lifting.