Biview Learning for Human Posture Segmentation from 3D Points Cloud

Posture segmentation plays an essential role in human motion analysis. The state-of-the-art method extracts sufficiently high-dimensional features from 3D depth images for each 3D point and learns an efficient body part classifier. However, high-dimensional features are memory-consuming and difficult to handle on large-scale training dataset. In this paper, we propose an efficient two-stage dimension reduction scheme, termed biview learning, to encode two independent views which are depth-difference features (DDF) and relative position features (RPF). Biview learning explores the complementary property of DDF and RPF, and uses two stages to learn a compact yet comprehensive low-dimensional feature space for posture segmentation. In the first stage, discriminative locality alignment (DLA) is applied to the high-dimensional DDF to learn a discriminative low-dimensional representation. In the second stage, canonical correlation analysis (CCA) is used to explore the complementary property of RPF and the dimensionality reduced DDF. Finally, we train a support vector machine (SVM) over the output of CCA. We carefully validate the effectiveness of DLA and CCA utilized in the two-stage scheme on our 3D human points cloud dataset. Experimental results show that the proposed biview learning scheme significantly outperforms the state-of-the-art method for human posture segmentation.     

A learning-based markerless approach for full-body kinematics estimation in-natura from a single image

We present a supervised machine learning approach for markerless estimation of human full-body kinematics for a cyclist from an unconstrained colour image. This approach is motivated by the limitations of existing marker-based approaches restricted by infrastructure, environmental conditions, and obtrusive markers. By using a discriminatively learned mixture-of-parts model, we construct a probabilistic tree representation to model the configuration and appearance of human body joints. During the learning stage, a Structured Support Vector Machine (SSVM) learns body parts appearance and spatial relations. In the testing stage, the learned models are employed to recover body pose via searching in a test image over a pyramid structure. We focus on the movement modality of cycling to demonstrate the efficacy of our approach. In natura estimation of cycling kinematics using images is challenging because of human interaction with a bicycle causing frequent occlusions. We make no assumptions in relation to the kinematic constraints of the model, nor the appearance of the scene. Our technique finds multiple quality hypotheses for the pose. We evaluate the precision of our method on two new datasets using loss functions. Our method achieves a score of 91.1 and 69.3 on mean Probability of Correct Keypoint (PCK) measure and 88.7 and 66.1 on the Average Precision of Keypoints (APK) measure for the frontal and sagittal datasets respectively. We conclude that our method opens new vistas to robust user-interaction free estimation of full body kinematics, a prerequisite to motion analysis.     

Remote Safety Monitoring for Elderly Persons Based on Omni-Vision Analysis

Remote monitoring service for elderly persons is important as the aged populations in most developed countries continue growing. To monitor the safety and health of the elderly population, we propose a novel omni-directional vision sensor based system, which can detect and track object motion, recognize human posture, and analyze human behavior automatically. In this work, we have made the following contributions: (1) we develop a remote safety monitoring system which can provide real-time and automatic health care for the elderly persons and (2) we design a novel motion history or energy images based algorithm for motion object tracking. Our system can accurately and efficiently collect, analyze, and transfer elderly activity information and provide health care in real-time. Experimental results show that our technique can improve the data analysis efficiency by 58.5% for object tracking. Moreover, for the human posture recognition application, the success rate can reach 98.6% on average.     

A model of neuro-musculo-skeletal system for human locomotion under position constraint condition

The human locomotion was studied on the basis of the interaction of the musculo-skeletal system, the neural system and the environment. A mathematical model of human locomotion under position constraint condition was established. Besides the neural rhythm generator, the posture controller and the sensory system, the environment feedback controller and the stability controller were taken into account in the model. The environment feedback controller was proposed for two purposes, obstacle avoidance and target position control of the swing foot. The stability controller was proposed to imitate the self-balancing ability of a human body and improve the stability of the model. In the stability controller, the ankle torque was used to control the velocity of the body gravity center. A prediction control algorithm was applied to calculate the torque magnitude of the stability controller. As an example, human stairs climbing movement was simulated and the results were given. The simulation result proved that the mathematical modeling of the task was successful.     

Self versus Environment Motion in Postural Control

To stabilize our position in space we use visual information as well as non-visual physical motion cues. However, visual cues can be ambiguous: visually perceived motion may be caused by self-movement, movement of the environment, or both. The nervous system must combine the ambiguous visual cues with noisy physical motion cues to resolve this ambiguity and control our body posture. Here we have developed a Bayesian model that formalizes how the nervous system could solve this problem. In this model, the nervous system combines the sensory cues to estimate the movement of the body. We analytically demonstrate that, as long as visual stimulation is fast in comparison to the uncertainty in our perception of body movement, the optimal strategy is to weight visually perceived movement velocities proportional to a power law. We find that this model accounts for the nonlinear influence of experimentally induced visual motion on human postural behavior both in our data and in previously published results.     

On designing geometric motion planners to solve regulating and trajectory tracking problems for robotic locomotion systems

Based on a geometric fiber bundle structure, a generalized method to solve both regulation and trajectory tracking problems for locomotion systems is presented. The method is especially applied to two case studies of robotic locomotion systems; a three link articulated fish-like robot as a prototype of locomotion systems with symmetry, and the snakeboard as a prototype of mixed locomotion systems. Our results show that although these motion planners have an open loop structure, due to their generalities, they can steer case studies with negligible errors for almost any complicated path.     

Effect of posture and locomotion on energy expenditure

Energy expenditure for human adults and infants and for dogs was measured in resting (supine or lateral) posture, in bipedal posture and locomotion, and in quadrupedal posture and locomotion. Variations in respiratory and heart rate and in body temperature were utilized in this comparative study. Oxygen consumption was also measured in human adults. In human adults, bipedal posture and locomotion were shown to be much less energy-consuming than corresponding quadrupedal posture and locomotion. The opposite was observed in adult dogs, where bipedalism was shown to be much more energy-consuming than quadrupedalism. In addition, this study demonstrated, for human adults in their natural erect posture, an energy expenditure barely higher than in supine or lateral resting posture, while the dogs in their natural quadrupedal stance, the energy expenditure is much higher than in their resting posture. With respect to energy, therefore, humans are more adapted to bipedalism than dogs to quadrupedalism. Human children, at the transitional stage between quadrupedalism and bipedalism, have high and almost equal requirements for all postures and locomotions. This demonstrates, in term of energy, their incomplete adaptation to erect behavior.     

What information do Kármán streets offer to flow sensing?

In this work, we focus on biomimetic lateral line sensing in Kármán vortex streets. After generating a Kármán street in a controlled environment, we examine the hydrodynamic images obtained with digital particle image velocimetry (DPIV). On the grounds that positioning in the flow and interaction with the vortices govern bio-inspired underwater locomotion, we inspect the fluid in the swimming robot frame of reference. We spatially subsample the flow field obtained using DPIV to emulate the local flow around the body. In particular, we look at various sensor configurations in order to reliably identify the vortex shedding frequency, wake wavelength and downstream flow speed. Moreover, we propose methods that differentiate between being in and out of the Kármán street with >70% accuracy, distinguish right from left with respect to Kármán vortex street centreline (>80%) and highlight when the sensor system enters the vortex formation zone (>75%). Finally, we present a method that estimates the relative position of a sensor array with respect to the vortex formation point within 15% error margin.     

Corollary discharge enables proprioception from lateral line sensory feedback

Animals modulate sensory processing in concert with motor actions. Parallel copies of motor signals, called corollary discharge (CD), prepare the nervous system to process the mixture of externally and self-generated (reafferent) feedback that arises during locomotion. Commonly, CD in the peripheral nervous system cancels reafference to protect sensors and the central nervous system from being fatigued and overwhelmed by self-generated feedback. However, cancellation also limits the feedback that contributes to an animal’s awareness of its body position and motion within the environment, the sense of proprioception. We propose that, rather than cancellation, CD to the fish lateral line organ restructures reafference to maximize proprioceptive information content. Fishes’ undulatory body motions induce reafferent feedback that can encode the body’s instantaneous configuration with respect to fluid flows. We combined experimental and computational analyses of swimming biomechanics and hair cell physiology to develop a neuromechanical model of how fish can track peak body curvature, a key signature of axial undulatory locomotion. Without CD, this computation would be challenged by sensory adaptation, typified by decaying sensitivity and phase distortions with respect to an input stimulus. We find that CD interacts synergistically with sensor polarization to sharpen sensitivity along sensors’ preferred axes. The sharpening of sensitivity regulates spiking to a narrow interval coinciding with peak reafferent stimulation, which prevents adaptation and homogenizes the otherwise variable sensor output. Our integrative model reveals a vital role of CD for ensuring precise proprioceptive feedback during undulatory locomotion, which we term external proprioception.

Animals modulate sensory processing in concert with motor actions. A study of the corollary discharge in zebrafish reveals that it modulates the sensitivity of the lateral line during swimming to prevent sensor adaptation and maintain the high-quality feedback necessary for kinematic control.     

A real-time predictive simulation of abdominal viscera positions during quiet free breathing

Prediction of abdominal viscera and tumour positions during free breathing is a major challenge from which several medical applications could benefit. For instance, in radiotherapy it would reduce the healthy tissue irradiation. In this paper, we present a new approach to predict real-time abdominal viscera positions during free breathing. Our method needs an abdo-thoracic 3D preoperative CT or MR image, a second one limited to the diaphragmatic area, and a tracking of the patient's skin position. First, a physical analysis of the breathing motion shows it is possible to predict accurately abdominal viscera positions from the skin position and a modelling of the diaphragm motion. Secondly, a quantitative analysis of the skin and organ motion allows us to define the demands our real-time simulation has to fulfill. Then, we present in detail all the necessary steps of our original method to compute a deformation field from data extracted in both 3D preoperative image and skin surface tracking. Finally, experiments carried out with two human data show that our simulation model predicts abdominal viscera positions, such as liver, kidneys or spleen, at 50 Hz with an accuracy within 2-3 mm.     

Variations in blood supply allocations for quadrupedal and for bipedal posture and locomotion

Hemodynamics and orthodynamics were investigated in quadrupeds (dogs) and in bipeds (humans). The subjects were investigated at rest in supine or lateral posture, in quadrupedal and then in bipedal posture, and during locomotion. Quadrupedalism in humans was with subjects on their hands and knees. Bipedalism in dogs was on hindlimbs with the forelimbs held by a technician. Blood flow in the main arteries of the body (aorta, external and internal carotid, subclavian, and femoral) was measured by sonography. Positional variations between the main bones of the body were determined from X-rays. This study investigated the reallocation of blood supply to different regions of the body when it switches from quadrupedal to bipedal posture and locomotion. Compared with resting posture, the principal findings are 1) cardiac output shows a minimal increase for humans in bipedal stance and a noticeable increase for dogs as well as humans in quadrupedal stance; 2) quadrupedal stance in humans and dogs and bipedal stance in dogs require increased blood supply to the muscles of the neck, back, and limbs, while human bipedal stance requires none of these; 3) cerebral blood flow (internal carotid) in humans did not change as a result of bipedal posture or locomotion, but showed a noticeable drop in quadrupedal posture and an even further drop in quadrupedal locomotion. The conclusion is that erect posture and encephalization produced a noticeable readjustment and reallocation of blood flow among the different regions of the body: This consisted in shifting a large volume of blood supply from the musculature to the human brain.     

Suspending loads decreases load stability but may slightly improve body stability

Here, we seek to determine how compliantly suspended loads could affect the dynamic stability of legged locomotion. We theoretically model the dynamic stability of a human carrying a load using a coupled spring-mass-damper model and an actuated spring-loaded inverted pendulum model, as these models have demonstrated the ability to correctly predict other aspects of locomotion with a load in prior work, such as body forces and energetic cost. We report that minimizing the load suspension natural frequency and damping ratio significantly reduces the stability of the load mass but may slightly improve the body stability of locomotion when compared to a rigidly attached load. These results imply that a highly-compliant load suspension could help stabilize body motion during human, animal, or robot load carriage, but at the cost of a more awkward (less stable) load.     

Requirements for the valid quantification of immunostains on tissue microarray materials using image analysis

Antibody‐based proteomics applied to tissue microarray (TMA) technology provides a very efficient means of visualizing and locating antigen expression in large collections of normal and pathological tissue samples. To characterize antigen expression on TMAs, the use of image analysis methods avoids the effects of human subjectivity evidenced in manual microscopical analysis. Thus, these methods have the potential to significantly enhance both precision and reproducibility. Although some commercial systems include tools for the quantitative evaluation of immunohistochemistry‐stained images, there exists no clear agreement on best practices to allow for correct and reproducible quantification results. Our study focuses on practical aspects regarding (i) image acquisition (ii) segmentation of staining and counterstaining areas and (iii) extraction of quantitative features. We illustrate our findings using a commercial system to quantify different immunohistochemistry markers targeting proteins with different expression patterns (cytoplasmic, nuclear or membranous) in colon cancer or brain tumor TMAs. Our investigations led us to identify several steps that we consider essential for standardizing computer‐assisted immunostaining quantification experiments. In addition, we propose a data normalization process based on reference materials to be able to compare measurements between studies involving different TMAs. In conclusion, we recommend certain critical prerequisites that commercial or in‐house systems should satisfy in order to permit valid immunostaining quantification.     

A general model for multiphase texture segmentation and its applications to retinal image analysis

In this paper we propose a general variational segmentation model for multiphase texture segmentation based on fuzzy region competition principle. An important strength of the proposed framework is that different region terms (e.g. mutual information Kim et al. (2005) [1], local histogram Ni et al. (2009) [2] models for texture-based segmentation, and piecewise constant intensity model Chan and Vese (2001) [3] for intensity-based segmentation) can be included as appropriate to the problem. Constraints of different phases are considered by introducing Lagrangian multipliers into the energy functional, and a fast numerical solution is achieved by employing the fast dual projection algorithm Chambolle (2004) [4]. The proposed model has been applied to synthetic and natural images in order to make comparisons with other competing models in literature. Our results demonstrate superiority in dealing with multiphase texture segmentation problems. To demonstrate its usefulness in biomedical applications we have applied the new model to two retinal image segmentation problems: segmentation of capillary non-perfusion regions in fluorescein angiogram and segmentation of cellular layers of the retina in optical coherence tomography, and evaluated against the gold standard set by experts. The generalized overlap analysis shows good agreement for both applications. As a generic segmentation technique our new model has the potential to be extended for wider applications.     

An investigation of rugby scrimmaging posture and individual maximum pushing force

Although rugby is a popular contact sport and the isokinetic muscle torque assessment has recently found widespread application in the field of sports medicine, little research has examined the factors associated with the performance of game-specific skills directly by using the isokinetic-type rugby scrimmaging machine. This study is designed to (a) measure and observe the differences in the maximum individual pushing forward force produced by scrimmaging in different body postures (3 body heights x 2 foot positions) with a self-developed rugby scrimmaging machine and (b) observe the variations in hip, knee, and ankle angles at different body postures and explore the relationship between these angle values and the individual maximum pushing force. Ten national rugby players were invited to participate in the examination. The experimental equipment included a self-developed rugby scrimmaging machine and a 3-dimensional motion analysis system. Our results showed that the foot positions (parallel and nonparallel foot positions) do not affect the maximum pushing force; however, the maximum pushing force was significantly lower in posture I (36% body height) than in posture II (38%) and posture III (40%). The maximum forward force in posture III (40% body height) was also slightly greater than for the scrum in posture II (38% body height). In addition, it was determined that hip, knee, and ankle angles under parallel feet positioning are factors that are closely negatively related in terms of affecting maximum pushing force in scrimmaging. In cross-feet postures, there was a positive correlation between individual forward force and hip angle of the rear leg. From our results, we can conclude that if the player stands in an appropriate starting position at the early stage of scrimmaging, it will benefit the forward force production.     

Motion vectors and deep neural networks for video camera traps

Commercial camera traps are usually triggered by a Passive Infra-Red (PIR) motion sensor necessitating a delay between triggering and the image being captured. This often seriously limits the ability to record images of small and fast moving animals. It also results in many “empty” images, e.g., owing to moving foliage against a background of different temperature. In this paper we detail a new triggering mechanism based solely on the camera sensor. This is intended for use by citizen scientists and for deployment on an affordable, compact, low-power Raspberry Pi computer (RPi). Our system introduces a video frame filtering pipeline consisting of movement and image-based processing. This makes use of Machine Learning (ML) feasible on a live camera stream on an RPi. We describe our free and open-source software implementation of the system; introduce a suitable ecology efficiency measure that mediates between specificity and recall; provide ground-truth for a video clip collection from camera traps; and evaluate the effectiveness of our system thoroughly. Overall, our video camera trap turns out to be robust and effective.     

Mantisbot is a robotic model of visually guided motion in the praying mantis

Insects use highly distributed nervous systems to process exteroception from head sensors, compare that information with state-based goals, and direct posture or locomotion toward those goals. To study how descending commands from brain centers produce coordinated, goal-directed motion in distributed nervous systems, we have constructed a conductance-based neural system for our robot MantisBot, a 29 degree-of-freedom, 13.3:1 scale praying mantis robot. Using the literature on mantis prey tracking and insect locomotion, we designed a hierarchical, distributed neural controller that establishes the goal, coordinates different joints, and executes prey-tracking motion. In our controller, brain networks perceive the location of prey and predict its future location, store this location in memory, and formulate descending commands for ballistic saccades like those seen in the animal. The descending commands are simple, indicating only 1) whether the robot should walk or stand still, and 2) the intended direction of motion. Each joint's controller uses the descending commands differently to alter sensory-motor interactions, changing the sensory pathways that coordinate the joints' central pattern generators into one cohesive motion. Experiments with one leg of MantisBot show that visual input produces simple descending commands that alter walking kinematics, change the walking direction in a predictable manner, enact reflex reversals when necessary, and can control both static posture and locomotion with the same network.     

Evaluation of "unique" aspects of human vertebral bodies and pedicles with a consideration of Australopithecus africanus

Recent functional studies of human vertebrae have revealed that loads borne by the axial skeleton during bipedal postures and locomotion pass through the pedicles and posterior elements as well as through the bodies and discs. Accordingly, particular morphological attributes of these vertebral elements have been linked exclusively with bipedalism. In order to test the validity of current form-function associations in human vertebral anatomy, this study considers the morphology of human thoracolumbar vertebral bodies and pedicles in the context of a wide comparative primate sample. The last lumbar vertebra of STS 14 (Australopithecus africanus) is also included in the analysis. Results indicate that certain features of human vertebrae previously thought to reflect bipedalism are characteristic of several nonhuman primates, including those whose posture is habitually pronograde. These features include the decrease in vertebral body surface area and the increase in cross-sectional area of the pedicle between the penultimate and last lumbar vertebra. In addition, although humans have relatively large and wide last lumbar pedicles, the enlargement and widening of the pedicle between the penultimate and last lumbar vertebra is not unique to humans. On the other hand, human vertebrae do exhibit several unique adaptations to bipedal posture and locomotion: (1) the vertebral body surface areas of the lower lumbar vertebrae and the cross-sectional areas of the last lumbar pedicles are large relative to body size, and (2) the last lumbar pedicles are wider relative to length and to body size than are those of nonhuman primates. The last lumbar vertebra of STS 14 does not exhibit any of these human-like vertebral features—its pedicles and body surface areas are relatively small, and its pedicles are not relatively wide, but relatively short.     

Mechanical interaction of center of pressure and force direction in the upright human

Humans maintain upright bipedal posture by producing appropriate force against the environment through the interaction of neural controlled muscle force with the mechanics of the skeletal system. Characterizing these mechanics facilitates understanding of the neural control. We used a mechanical model of an upright human to analyze how the mechanical linkage aspects of the human body affect the force between the feet and the ground (F). Key parameters of F that directly regulate upright body posture are the direction of F (θ(F)) and its point of application (x(CP), anterior-posterior position of the center of pressure). Instantaneous analysis of the equations of motion demonstrated that θ(F) varied systematically with x(CP) such that the F vectors intersected at a point called the Posture-specific force Intersection point or PI (Π). The Π was located above the center of mass when the hip and knee joints were modeled as rigid and was located near the knee when the hip and knee torques were held constant. Limb posture and the knee torque affected the location of Π. This Π behavior quantifies the purely mechanical effect of anterior-posterior center of pressure shifts on the direction of F, which has consequences for the control of whole body posture.