Schema generation in recurrent neural nets for intercepting a moving target

The grasping of a moving object requires the development of a motor strategy to anticipate the trajectory of the target and to compute an optimal course of interception. During the performance of perception-action cycles, a preprogrammed prototypical movement trajectory, a motor schema, may highly reduce the control load. Subjects were asked to hit a target that was moving along a circular path by means of a cursor. Randomized initial target positions and velocities were detected in the periphery of the eyes, resulting in a saccade toward the target. Even when the target disappeared, the eyes followed the target’s anticipated course. The Gestalt of the trajectories was dependent on target velocity. The prediction capability of the motor schema was investigated by varying the visibility range of cursor and target. Motor schemata were determined to be of limited precision, and therefore visual feedback was continuously required to intercept the moving target. To intercept a target, the motor schema caused the hand to aim ahead and to adapt to the target trajectory. The control of cursor velocity determined the point of interception. From a modeling point of view, a neural network was developed that allowed the implementation of a motor schema interacting with feedback control in an iterative manner. The neural net of the Wilson type consists of an excitation-diffusion layer allowing the generation of a moving bubble. This activation bubble runs down an eye-centered motor schema and causes a planar arm model to move toward the target. A bubble provides local integration and straightening of the trajectory during repetitive moves. The schema adapts to task demands by learning and serves as forward controller. On the basis of these model considerations the principal problem of embedding motor schemata in generalized control strategies is discussed.     

Oculocentric frames of reference for limb movement

We have reviewed evidence that suggests that the target for limb motion is encoded in a retinocentric frame of reference. Errors in pointing that are elicited by an illusion that distorts the perceived motion of a target are strongly correlated with errors in gaze position. The modulations in the direction and speed of ocular smooth pursuit and of the hand show remarkable similarities, even though the inertia of the arm is much larger than that of the eye. We have suggested that ocular motion is constrained so that gaze provides an appropriate target signal for the hand. Finally, ocular and manual tracking deficits in patients with cerebellar ataxia are very similar. These deficits are also consistent with the idea that a gaze signal provides the target for hand motion; in some cases limb ataxia would be a consequence of optic ataxia rather than reflecting a deficit in the control of limb motion per se. These results, as well as neurophysiological data summarized here, have led us to revise a hypothesis we have previously put forth to account for the initial stages of sensorimotor transformations underlying targeted limb motions. In the original hypothesis, target location and initial arm posture were ultimately encoded in a common frame of reference tied to somatosensation, i.e. a body-centered frame of reference, and a desired change in posture was derived from the difference between the two. In our new scheme, a movement vector is derived from the difference between variables encoded in a retinocentric frame of reference. Accordingly, gaze, with its exquisite ability to stabilize a target image even under dynamic conditions, would be used as a reference signal. Consequently, this scheme would facilitate the processing of information under conditions in which the body and the target are moving relative to each other.     

Distance in motion: response trajectories reveal the dynamics of number comparison

Cognitive and neuroscientific evidence has challenged the widespread view that perception, cognition and action constitute independent, discrete stages. For example, in continuous response trajectories toward a target response location, evidence suggests that a decision on which target to reach for (i.e., the cognition stage) is not reached before the movement starts (i.e., the action stage). As a result, instead of a straight trajectory to the correct target response, movement trajectories may curve toward competing responses or away from inhibited responses. In the present study, we examined response trajectories during a number comparison task. Participants had to decide whether a target number was smaller or larger than 5. They had to respond by moving to a left or a right response location. Replicating previous results, response trajectories were more curved toward the incorrect response location when distance to 5 was small (e.g., target number 4) than when distance to 5 was large (e.g., target number 1). Importantly, we manipulated the response mapping, which allowed us to demonstrate that this response trajectory effect results from the relative amount of evidence for the available responses across time. In this way, the present study stresses the tight coupling of number representations (i.e., cognition) and response related processes (i.e., action) and shows that these stages are not separable in time.     

Formation and control of optimal trajectory in human multijoint arm movement

In this paper, we study trajectory planning and control in voluntary, human arm movements. When a hand is moved to a target, the central nervous system must select one specific trajectory among an infinite number of possible trajectories that lead to the target position. First, we discuss what criterion is adopted for trajectory determination. Several researchers measured the hand trajectories of skilled movements and found common invariant features. For example, when moving the hand between a pair of targets, subjects tended to generate roughly straight hand paths with bell-shaped speed profiles. On the basis of these observations and dynamic optimization theory, we propose a mathematical model which accounts for formation of hand trajectories. This model is formulated by defining an objective function, a measure of performance for any possible movement: square of the rate of change of torque integrated over the entire movement. That is, the objective function CT is defined as follows: (formula; see text) We overcome this difficult by developing an iterative scheme, with which the optimal trajectory and the associated motor command are simultaneously computed. To evaluate our model, human hand trajectories were experimentally measured under various behavioral situations. These results supported the idea that the human hand trajectory is planned and controlled in accordance with the minimum torque-change criterion.     

Experimental Test of Spatial Updating Models for Monkey Eye-Head Gaze Shifts

How the brain maintains an accurate and stable representation of visual target locations despite the occurrence of saccadic gaze shifts is a classical problem in oculomotor research. Here we test and dissociate the predictions of different conceptual models for head-unrestrained gaze-localization behavior of macaque monkeys. We adopted the double-step paradigm with rapid eye-head gaze shifts to measure localization accuracy in response to flashed visual stimuli in darkness. We presented the second target flash either before (static), or during (dynamic) the first gaze displacement. In the dynamic case the brief visual flash induced a small retinal streak of up to about 20 deg at an unpredictable moment and retinal location during the eye-head gaze shift, which provides serious challenges for the gaze-control system. However, for both stimulus conditions, monkeys localized the flashed targets with accurate gaze shifts, which rules out several models of visuomotor control. First, these findings exclude the possibility that gaze-shift programming relies on retinal inputs only. Instead, they support the notion that accurate eye-head motor feedback updates the gaze-saccade coordinates. Second, in dynamic trials the visuomotor system cannot rely on the coordinates of the planned first eye-head saccade either, which rules out remapping on the basis of a predictive corollary gaze-displacement signal. Finally, because gaze-related head movements were also goal-directed, requiring continuous access to eye-in-head position, we propose that our results best support a dynamic feedback scheme for spatial updating in which visuomotor control incorporates accurate signals about instantaneous eye- and head positions rather than relative eye- and head displacements.     

A simulation study of a programme generator for centrally programmed fast two-joint arm movements: responses to single- and double-step target displacements

From earlier experimental results we concluded that a sudden displacement of a target causes the internal representation of the target to shift gradually from the first to the second target location. This internal target is used as an input to the motor programme generator. The aim of the present simulation study is to investigate whether the CNS can programme fast arm movements towards the internal representation of the target without taking into account velocity-dependent mechanical interactions between the upper arm and forearm and velocity-dependent biomechanical properties of the muscles. The simulations of single- and double-step responses show that on the basis of the assumptions mentioned above, movement trajectories and velocity profiles are in fair agreement with experimental results. However, at the end of fast arm movements a stiffness and viscosity control is needed to halt the movement at the target location.     

Prospective Control in Catching: The Persistent Angle-of-Approach Effect in Lateral Interception

In lateral interception tasks balls converging onto the same interception location via different trajectories give rise to systematic differences in the kinematics of hand movement. While it is generally accepted that this angle-of-approach effect reflects the prospective (on-line) control of movement, controversy exists with respect to the information used to guide the hand to the future interception location. Based on the pattern of errors observed in a task requiring visual extrapolation of line segments to their intersection with a second line, angle-of-approach effects in lateral interception have been argued to result from perceptual biases in the detection of information about the ball''s future passing distance along the axis of hand movement. Here we demonstrate that this account does not hold under experimental scrutiny: The angle-of-approach effect still emerged when participants intercepted balls moving along trajectories characterized by a zero perceptual bias with respect to the ball''s future arrival position (Experiment 4). Designing and validating such bias-controlled trajectories were done using the line-intersection extrapolation task (Experiments 2 and 3). The experimental set-up used in the present series of experiments was first validated for the lateral interception and the line-intersection extrapolation tasks: In Experiment 1 we used rectilinear ball trajectories to replicate the angle-of-approach effect in lateral interception of virtual balls. Using line segments extracted from these rectilinear ball trajectories, in Experiment 2 we replicated the reported pattern of errors in the estimated locus of intersection with the axis of hand movement. We used these errors to develop a set of bias-free trajectories. Experiment 3 confirmed that the perceptual biases had been corrected for successfully. We discuss the implications on the information-based regulation of hand movement of our finding that the angle-of-approach effect in lateral interception cannot not explained by perceptual biases in information about the ball''s future passing distance.     

Motor control of voluntary arm movements

The motor control of pointing and reaching-to-grasp movements was investigated using two different approaches (kinematic and modelling) in order to establish whether the type of control varies according to modifications of arm kinematics. Kinematic analysis of arm movements was performed on subjects' hand trajectories directed to large and small stimuli located at two different distances. The subjects were required either to grasp and to point to each stimulus. The kinematics of the subsequent movement, during which subject's hand came back to the starting position, were also studied. For both movements, kinematic analysis was performed on hand linear trajectories as well as on joint angular trajectories of shoulder and elbow. The second approach consisted in the parametric identification of the black box (ARMAX) model of the controller driving the arm movement. Such controller is hypothesized to work for the correct execution of the motor act. The order of the controller ARMAX model was analyzed with respect to the different experimental conditions (distal task, stimulus size and distance). Results from kinematic analysis showed that target distance and size influenced kinematic parameters both of angular and linear displacements. Nevertheless, the structure of the motor program was found to remain constant with distane and distal task, while it varied with precision requirements due to stimulus size. The estimated model order of the controller confirmed the invariance of the control law with regard to movement amplitude, whereas it was sensitive to target size.     

Predictive adjustment of the perceived direction of gaze during saccadic eye movements

When we look at a stationary object, the perceived direction of gaze (where we are looking) is aligned with the physical direction of eyes (where our eyes are oriented) by which the object is foveated. However, this alignment may not hold in a dynamic situation. Our experiments assessed the perceived locations of two brief stimuli (1 ms) simultaneously displayed at two different physical locations during a saccade. The first stimulus was in the instantaneous location to which the eyes were oriented and the second one was always in the same location as the initial fixation point. When the timing of these stimuli was changed intra-saccadically, their perceived locations were dissociated. The first stimuli were consistently perceived near the target that will be foveated at saccade termination. The second stimuli once perceived near the target location, shifted in the direction opposite to that of saccades, as its latency from saccades increased. These results suggested an independent adjustment of gaze orientation from the physical orientation of eyes during saccades. The spatial dissociation of two stimuli may reflect sensorimotor control of gaze during saccades.     

Analysis of kinematic invariances of multijoint reaching movement

 There is a no unique relationship between the trajectory of the hand, represented in cartesian or extrinsic space, and its trajectory in joint angle or intrinsic space in the general condition of joint redundancy. The goal of this work is to analyze the relation between planning the trajectory of a multijoint movement in these two coordinate systems. We show that the cartesian trajectory can be planned based on the task parameters (target coordinates, etc.) prior to and independently of angular trajectories. Angular time profiles are calculated from the cartesian trajectory to serve as a basis for muscle control commands. A unified differential equation that allows planning trajectories in cartesian and angular spaces simultaneously is proposed. Due to joint redundancy, each cartesian trajectory corresponds to a family of angular trajectories which can account for the substantial variability of the latter. A set of strategies for multijoint motor control following from this model is considered; one of them coincides with the frog wiping reflex model and resolves the kinematic inverse problem without inversion. The model trajectories exhibit certain properties observed in human multijoint reaching movements such as movement equifinality, straight end-point paths, bell-shaped tangential velocity profiles, speed-sensitive and speed-insensitive movement strategies, peculiarities of the response to double-step targets, and variations of angular trajectory without variations of the limb end-point trajectory in cartesian space. In humans, those properties are almost independent of limb configuration, target location, movement duration, and load. In the model, these properties are invariant to an affine transform of cartesian space. This implies that these properties are not a special goal of the motor control system but emerge from movement kinematics that reflect limb geometry, dynamics, and elementary principles of motor control used in planning. All the results are given analytically and, in order to compare the model with experimental results, by computer simulations. Received: 6 April 1994/Accepted in revised form: 25 April 1995     

Smooth Pursuit Eye Movements Improve Temporal Resolution for Color Perception

Human observers see a single mixed color (yellow) when different colors (red and green) rapidly alternate. Accumulating evidence suggests that the critical temporal frequency beyond which chromatic fusion occurs does not simply reflect the temporal limit of peripheral encoding. However, it remains poorly understood how the central processing controls the fusion frequency. Here we show that the fusion frequency can be elevated by extra-retinal signals during smooth pursuit. This eye movement can keep the image of a moving target in the fovea, but it also introduces a backward retinal sweep of the stationary background pattern. We found that the fusion frequency was higher when retinal color changes were generated by pursuit-induced background motions than when the same retinal color changes were generated by object motions during eye fixation. This temporal improvement cannot be ascribed to a general increase in contrast gain of specific neural mechanisms during pursuit, since the improvement was not observed with a pattern flickering without changing position on the retina or with a pattern moving in the direction opposite to the background motion during pursuit. Our findings indicate that chromatic fusion is controlled by a cortical mechanism that suppresses motion blur. A plausible mechanism is that eye-movement signals change spatiotemporal trajectories along which color signals are integrated so as to reduce chromatic integration at the same locations (i.e., along stationary trajectories) on the retina that normally causes retinal blur during fixation.     

Independent control of acceleration and direction of the hand when hitting moving targets

Human subjects were asked to hit moving targets as quickly as they could. Nevertheless the speed with which the subjects moved toward identical stimuli differed between trials. We examined whether the subjects compensated for a lower initial acceleration by aiming further ahead of the target. We found that the initial acceleration of the hand and its initial direction were hardly correlated. Thus subjects did not aim further ahead when they hit more slowly. This supports our earlier suggestion that the acceleration of the hand and the direction in which it moves are controlled separately.     

Sensory Agreement Guides Kinetic Energy Optimization of Arm Movements during Object Manipulation

The laws of physics establish the energetic efficiency of our movements. In some cases, like locomotion, the mechanics of the body dominate in determining the energetically optimal course of action. In other tasks, such as manipulation, energetic costs depend critically upon the variable properties of objects in the environment. Can the brain identify and follow energy-optimal motions when these motions require moving along unfamiliar trajectories? What feedback information is required for such optimal behavior to occur? To answer these questions, we asked participants to move their dominant hand between different positions while holding a virtual mechanical system with complex dynamics (a planar double pendulum). In this task, trajectories of minimum kinetic energy were along curvilinear paths. Our findings demonstrate that participants were capable of finding the energy-optimal paths, but only when provided with veridical visual and haptic information pertaining to the object, lacking which the trajectories were executed along rectilinear paths.     

Novel tracking function of moving target using chaotic dynamics in a recurrent neural network model

Chaotic dynamics introduced in a recurrent neural network model is applied to controlling an object to track a moving target in two-dimensional space, which is set as an ill-posed problem. The motion increments of the object are determined by a group of motion functions calculated in real time with firing states of the neurons in the network. Several cyclic memory attractors that correspond to several simple motions of the object in two-dimensional space are embedded. Chaotic dynamics introduced in the network causes corresponding complex motions of the object in two-dimensional space. Adaptively real-time switching of control parameter results in constrained chaos (chaotic itinerancy) in the state space of the network and enables the object to track a moving target along a certain trajectory successfully. The performance of tracking is evaluated by calculating the success rate over 100 trials with respect to nine kinds of trajectories along which the target moves respectively. Computer experiments show that chaotic dynamics is useful to track a moving target. To understand the relations between these cases and chaotic dynamics, dynamical structure of chaotic dynamics is investigated from dynamical viewpoint.     

Effects of age on orientation of attention

The present experiment investigated the influence of age on the magnitude of attentional effects. Subjects were cued to attend to one of two possible stimulus locations horizontally arranged. The instructions were to respond as fast as possible to the occurrence of a visual stimulus, regardless of whether it occurred in a cued or in a non-cued location. When the stimulus occurred in a non-cued location, the subject had to reorient attention to the non-attended, but stimulated, location. Reorienting of attention was directed toward either the left or the right side. The results showed that for the elders, time for reorienting was longer than for younger subjects.     

Postural Sway and Gaze Can Track the Complex Motion of a Visual Target

Variability is an inherent and important feature of human movement. This variability has form exhibiting a chaotic structure. Visual feedback training using regular predictive visual target motions does not take into account this essential characteristic of the human movement, and may result in task specific learning and loss of visuo-motor adaptability. In this study, we asked how well healthy young adults can track visual target cues of varying degree of complexity during whole-body swaying in the Anterior-Posterior (AP) and Medio-Lateral (ML) direction. Participants were asked to track three visual target motions: a complex (Lorenz attractor), a noise (brown) and a periodic (sine) moving target while receiving online visual feedback about their performance. Postural sway, gaze and target motion were synchronously recorded and the degree of force-target and gaze-target coupling was quantified using spectral coherence and Cross-Approximate entropy. Analysis revealed that both force-target and gaze-target coupling was sensitive to the complexity of the visual stimuli motions. Postural sway showed a higher degree of coherence with the Lorenz attractor than the brown noise or sinusoidal stimulus motion. Similarly, gaze was more synchronous with the Lorenz attractor than the brown noise and sinusoidal stimulus motion. These results were similar regardless of whether tracking was performed in the AP or ML direction. Based on the theoretical model of optimal movement variability tracking of a complex signal may provide a better stimulus to improve visuo-motor adaptation and learning in postural control.     

The capacity of human subjects to process directional information provided at two skin sites

The ability of human subjects to discriminate direction of tactile stimulus motion on the dorsum of the hand was determined (1) in the absence and (2) in the presence of a moving stimulus delivered to a second skin site on the ipsilateral or contralateral forelimb. When the two skin sites were simultaneously contacted by stimuli moving in the same direction, directional sensitivity was typically below that predicted for a hypothetical subject who could independently process the information provided at each of the two skin sites. Even when the stimulus delivered to a second site was deliberately ignored, it could still alter a subject's perception of stimulus direction on the dorsal hand. Moreover, its influence was greatest whenever it moved in a direction opposite to that of the attended stimulus. Whenever the two moving stimuli were delivered nonsimultaneously to two skin sites, directional sensitivity rarely matched the levels predicted for a hypothetical subject who could independently process the information provided at each site. This, in part, resulted from the subjects' utilization of "long-range" cues provided by the temporal order of stimulation. Subjects frequently failed to distinguish these cues from the sensation of stimulus direction provided at each skin site.     

Determining finger segmental centers of rotation in flexion-extension based on surface marker measurement

This paper describes the development of a novel algorithm for deriving finger segmental center of rotation (COR) locations during flexion-extension from measured surface marker motions in vivo. The algorithm employs an optimization routine minimizing the time-variance of the internal link lengths, and incorporates an empirically quantifiable relationship between the local movement of a surface marker around a joint (termed "surface marker excursion") and the joint flexion-extension. The latter relationship constrains and simplifies the optimization routine to make it computationally tractable. To empirically investigate this relationship and test the proposed algorithm, an experiment was conducted, in which hand cylinder-grasping movements were performed by 24 subjects (12 males and 12 females). Spherical retro-reflective markers were placed at various surface landmarks on the dorsal aspect of each subject's right (grasping) hand, and were measured during the movements by an opto-electronic system. Analysis of experimental data revealed a highly linear relationship between the "surface marker excursion" and the marker-defined flexion-extension angle: the average R(2) in linear regression ranged from 0.89 to 0.97. The algorithm successfully determined the CORs of the distal interphalangeal, proximal interphalangeal, and metacarpophalangeal joints of digits 2-5 during measured motions. The derived CORs appeared plausible as examined in terms of the physical locations relative to surface marker trajectories and the congruency across different joints and individuals.     

Manipulating intent: evidence for a causal role of the superior colliculus in target selection

The superior colliculus (SC) is well known for its role in the motor control of saccades. Recent work has shown that it also plays a role in the selection of saccades, but a causal role in the process of target selection has not been demonstrated. We applied subthreshold microstimulation to the SC while monkeys performed a task requiring them to select a stimulus as the target for a pursuit or saccade movement. Stimulation increased the proportion of selections toward the stimulus that appeared contralateral to the site of stimulation and also decreased their latencies. For pursuit, this stimulation-induced contralateral response bias was with respect to the initial target location and not the direction of eye movement, demonstrating a causal effect on target choice distinct from any effect on motor preparation. These results show that the SC helps decide the object of the next movement, beyond its traditional responsibility of saccade production.     

The psychomotor test for research of eye-hand coordination at performance of monotonous activity on tracking target

Visual-motor coordination is necessary for successful performance of everyday activities. Many tasks, such as driving or operating devices in the workplace, require a variety of coordination patterns with different levels of compatibility between the eyes and the hand. The psychomotor test was developed which makes it possible to analyze visual-motor coordination disorders caused by a decrease in the level of wakefulness. A small circular target (14 mm in diameter) was moving with a low constant velosity (12 mm/s) in a circular trajectory (80 mm in diameter) with a period of 20 s. Subjects were instructed to keep the mouse-driven cursor inside a target, overstepping the limits of the moving target was considered as an error. To test the attention level, an additional stimulus was introduced which appeared for 3 seconds with an interval of 15 to 40 s. When the stimulus appeared, it was required to touch it with the cursor and click the mouse button. Cursor trajectories have a temporal resolution of 120 Hz. Eye movements were recorded with a PC-based Eyegaze Development System (LC Technologies, USA) measuring corneal reflectance with a collection rate of 120 Hz. Monotonous character of the test performance induced drowsiness and led to errors 25-30 minutes after the beginning of the experiment. Changes in physiological vigilance level was evaluated with EEG recording. Analysis of the dynamic characteristics of smooth and saccadic eye movements and hand movements showed their high sensitivity to a decrease in efficiency of operator's activity caused by a drop in the level of wakefulness. It is suggested that further development of this approach to measuring eye-hand coordination will promote working out a contactless method for the express diagnostics of the critical levels of drowsiness as well as for the definition of professional characteristics of an operator.