A model of handwriting

The research reported here is concerned with hand trajectory planning for the class of movements involved in handwriting. Previous studies show that the kinematics of human two-joint arm movements in the horizontal plane can be described by a model which is based on dynamic minimization of the square of the third derivative of hand position (jerk), integrated over the entire movement. We extend this approach to both the analysis and the synthesis of the trajectories occurring in the generation of handwritten characters. Several basic strokes are identified and possible stroke concatenation rules are suggested. Given a concise symbolic representation of a stroke shape, a simple algorithm computes the complete kinematic specification of the corresponding trajectory. A handwriting generation model based on a kinematics from shape principle and on dynamic optimization is formulated and tested. Good qualitative and quantitative agreement was found between subject recordings and trajectories generated by the model. The simple symbolic representation of hand motion suggested here may permit the central nervous system to learn, store and modify motor action plans for writing in an efficient manner.     

A theory for cursive handwriting based on the minimization principle

 We propose a trajectory planning and control theory which provides explanations at the computation, algorithm, representation, and hardware levels for continuous movement such as connected cursive handwriting. The hardware is based on our previously proposed forward-inverse-relaxation neural network. Computationally, the optimization principle is the minimum torque-change criterion. At the representation level, hard constraints satisfied by a trajectory are represented as a set of via-points extracted from handwritten characters. Accordingly, we propose a via-point estimation algorithm that estimates via-points by repeating trajectory formation of a character and via-point extraction from the character. It is shown experimentally that for movements with a single via-point target, the via-point estimation algorithm can assign a point near the actual via-point target. Good quantitative agreement is found between human movement data and the trajectories generated by the proposed model. Received: 23 June 1994 / Accepted in revised form: 3 February 1995     

The ontogeny of visual–motor memory and its importance in handwriting and reading: a developing construct

Humans have evolved a remarkable ability to remember visual shapes and use these representations to generate motor activity (from Palaeolithic cave drawings through Jiahu symbols to cursive handwriting). The term visual–motor memory (VMM) describes this psychological ability, which must have conveyed an evolutionary advantage and remains critically important to humans (e.g. when learning to write). Surprisingly, little empirical investigation of this unique human ability exists—almost certainly because of the technological difficulties involved in measuring VMM. We deployed a novel technique for measuring this construct in 87 children (6–11 years old, 44 females). Children drew novel shapes presented briefly on a tablet laptop screen, drawing their responses from memory on the screen using a digitizer stylus. Sophisticated algorithms (using point-registration techniques) objectively quantified the accuracy of the children''s reproductions. VMM improved with age and performance decreased with shape complexity, indicating that the measure captured meaningful developmental changes. The relationship between VMM and scores on nationally standardized writing assessments were explored with the results showing a clear relationship between these measures, even after controlling for age. Moreover, a relationship between VMM and the nationally standardized reading test was mediated via writing ability, suggesting VMM''s wider importance within language development.     

Grip force variability and its effects on children's handwriting legibility, form, and strokes

A comprehensive understanding of the underlying biomechanical processes during handwriting is needed to accurately guide clinical interventions. To date, quantitative measurement of such biomechanical processes has largely excluded measurements of the forces exerted radially on the barrel of the writing utensil (grip forces) and how they vary over time during a handwriting task. An instrumented writing utensil was deployed for a direct measurement of kinematic and temporal information during a writing task, as well as forces exerted on the writing surface and on the barrel of the pen. The writing utensil was used by a cohort of 35 students (19 males), 16 in first grade and 19 in second grade, as they performed the Minnesota Handwriting Assessment (MHA) test. Quantitative grip force variability measures were computed and tested as correlates of handwriting legibility, form, and strokes. Grip force variability was shown to correlate strongly with handwriting quality, in particular for students classified by the MHA as nonproficient writers. More specifically, static grip force patterns were shown to result in poor handwriting quality and in greater variation in handwriting stroke durations. Grip force variability throughout the writing task was shown to be significantly lower for nonproficient writers (t-test, p<0.01) while the number of strokes and per-stroke durations were shown to be higher (p<0.03). The results suggest that grip force dynamics play a key role in determining handwriting quality and stroke characteristics. In particular, students with writing difficulties exhibited more static grip force patterns, lower legibility and form scores, as well as increased variation in stroke durations. These findings shed light on the underlying processes of handwriting and grip force modulation and may help to improve intervention planning.     

The forces behind the words: development of the kinetic pen

This paper describes the creation of a Kinetic Pen capable of measuring the six-component force and torque that each of four individual contacts applies to the pen during writing. This was done by staggering the mounting of the four sensors along the long axis of the pen and having an extended arm run from the sensor to the grip site, preventing a clustering of the sensors where the digit tips meet while grasping. The implications of this tool allow handwriting studies to be expanded from two-dimensional pen-tip kinematics to three-dimensional dynamics at each contact point between the hand and pen.     

Motor program coding representation from a handwriting generator model: The production of line responses

In this paper, a model is employed that describes handwriting behaviors activated by curvilinear and angular velocity generators postulated to initiate and regulate pen tip velocity profiles. This model accounts for the observed differences between straight and curved line production and the effect of movement precues on these responses. Of particular interest is the observed interaction between precue information and line execution type for reaction time. It is shown that differences in reaction time can be explained by the model as a function of the number of parameters that need to be specified. Moreover, there is some evidence that the biomechanical system reacts in a privileged manner to command pulses for specific directions, and that the central nervous system attempts to compensate for these asymmetries. These data are some of the first to show that the benefits of precue extend beyond reaction time and that movement execution characteristics are influenced by motor preparation.     

“Let Me Hear Your Handwriting!” Evaluating the Movement Fluency from Its Sonification

The quality of handwriting is evaluated from the visual inspection of its legibility and not from the movement that generates the trace. Although handwriting is achieved in silence, adding sounds to handwriting movement might help towards its perception, provided that these sounds are meaningful. This study evaluated the ability to judge handwriting quality from the auditory perception of the underlying sonified movement, without seeing the written trace. In a first experiment, samples of a word written by children with dysgraphia, proficient children writers, and proficient adult writers were collected with a graphic tablet. Then, the pen velocity, the fluency, and the axial pen pressure were sonified in order to create forty-five audio files. In a second experiment, these files were presented to 48 adult listeners who had to mark the underlying unseen handwriting. In order to evaluate the relevance of the sonification strategy, two experimental conditions were compared. In a first ‘implicit’ condition, the listeners made their judgment without any knowledge of the mapping between the sounds and the handwriting variables. In a second ‘explicit’ condition, they knew what the sonified variables corresponded to and the evaluation criteria. Results showed that, under the implicit condition, two thirds of the listeners marked the three groups of writers differently. In the explicit condition, all listeners marked the dysgraphic handwriting lower than that of the two other groups. In a third experiment, the scores given from the auditory evaluation were compared to the scores given by 16 other adults from the visual evaluation of the trace. Results revealed that auditory evaluation was more relevant than the visual evaluation for evaluating a dysgraphic handwriting. Handwriting sonification might therefore be a relevant tool allowing a therapist to complete the visual assessment of the written trace by an auditory control of the handwriting movement quality.     

Handwriting performance in the absence of visual control in writer's cramp patients: Initial observations

Background  

The present study was aimed at investigating the writing parameters of writer's cramp patients and control subjects during handwriting of a test sentence in the absence of visual control.     

Decoding Hindlimb Movement for a Brain Machine Interface after a Complete Spinal Transection

Stereotypical locomotor movements can be made without input from the brain after a complete spinal transection. However, the restoration of functional gait requires descending modulation of spinal circuits to independently control the movement of each limb. To evaluate whether a brain-machine interface (BMI) could be used to regain conscious control over the hindlimb, rats were trained to press a pedal and the encoding of hindlimb movement was assessed using a BMI paradigm. Off-line, information encoded by neurons in the hindlimb sensorimotor cortex was assessed. Next neural population functions, or weighted representations of the neuronal activity, were used to replace the hindlimb movement as a trigger for reward in real-time (on-line decoding) in three conditions: while the animal could still press the pedal, after the pedal was removed and after a complete spinal transection. A novel representation of the motor program was learned when the animals used neural control to achieve water reward (e.g. more information was conveyed faster). After complete spinal transection, the ability of these neurons to convey information was reduced by more than 40%. However, this BMI representation was relearned over time despite a persistent reduction in the neuronal firing rate during the task. Therefore, neural control is a general feature of the motor cortex, not restricted to forelimb movements, and can be regained after spinal injury.     

Psychophysiological Mechanisms of Writing Difficulties in Schoolchildren

The temporal and qualitative organization of writing movements was studied in six- to seven-year-old and nine- to ten-year-old right-handed and left-handed children with underdeveloped visuomotor coordination. The characteristic features of the temporal and qualitative organization of these movements at different stages of motor skill formation were found. Regarding the temporal structure of the series of writing movements, a pause was observed between individual movements in the series of motor task formation. The duration of this pause may serve as a criterion of the degree and efficiency of writing skill formation. The disturbance of the temporal structure of movements during the formation of handwriting speed at the initial stages of learning deteriorated the temporal organization of movements, drastically decreased the quality, and impeded the formation of the skill. The destructive processes were more pronounced in the case of accelerated handwriting in left-handed children and especially in children with underdeveloped visuomotor coordination.     

Impaired Retention of Motor Learning of Writing Skills in Patients with Parkinson’s Disease with Freezing of Gait

Background

Patients with Parkinson’s disease (PD) and freezing of gait (FOG) suffer from more impaired motor and cognitive functioning than their non-freezing counterparts. This underlies an even higher need for targeted rehabilitation programs in this group. However, so far it is unclear whether FOG affects the ability for consolidation and generalization of motor learning and thus the efficacy of rehabilitation.

Objective

To investigate the hallmarks of motor learning in people with FOG compared to those without by comparing the effects of an intensive motor learning program to improve handwriting.

Methods

Thirty five patients with PD, including 19 without and 16 with FOG received six weeks of handwriting training consisting of exercises provided on paper and on a touch-sensitive writing tablet. Writing training was based on single- and dual-task writing and was supported by means of visual target zones. To investigate automatization, generalization and retention of learning, writing performance was assessed before and after training in the presence and absence of cues and dual tasking and after a six-week retention period. Writing amplitude was measured as primary outcome measure and variability of writing and dual-task accuracy as secondary outcomes.

Results

Significant learning effects were present on all outcome measures in both groups, both for writing under single- and dual-task conditions. However, the gains in writing amplitude were not retained after a retention period of six weeks without training in the patient group without FOG. Furthermore, patients with FOG were highly dependent on the visual target zones, reflecting reduced generalization of learning in this group.

Conclusions

Although short-term learning effects were present in both groups, generalization and retention of motor learning were specifically impaired in patients with PD and FOG. The results of this study underscore the importance of individualized rehabilitation protocols.     

Bayesian action-perception computational model: interaction of production and recognition of cursive letters

In this paper, we study the collaboration of perception and action representations involved in cursive letter recognition and production. We propose a mathematical formulation for the whole perception-action loop, based on probabilistic modeling and bayesian inference, which we call the Bayesian Action-Perception (BAP) model. Being a model of both perception and action processes, the purpose of this model is to study the interaction of these processes. More precisely, the model includes a feedback loop from motor production, which implements an internal simulation of movement. Motor knowledge can therefore be involved during perception tasks. In this paper, we formally define the BAP model and show how it solves the following six varied cognitive tasks using bayesian inference: i) letter recognition (purely sensory), ii) writer recognition, iii) letter production (with different effectors), iv) copying of trajectories, v) copying of letters, and vi) letter recognition (with internal simulation of movements). We present computer simulations of each of these cognitive tasks, and discuss experimental predictions and theoretical developments.     

Modeling of control and learning in a stepping motion

In a previous study (Beuter et al. 1986) the authors modeled a stepping motion using a three-body linkage with four degrees of freedom. Stepping was simulated by using three task parameters (i.e., step height, length, and duration) and sinusoidal joint angular velocity profiles. The results supported the concept of a hierarchical control structure with open-loop control during normal operation. In this study we refine the dynamic model and improve the simulation technique by incorporating the dynamics of the leg after landing, adding a foot segment to the model, and preprogramming the complete step motion using cycloids. The equations of the forces and torques developed on the ground by the foot during the landing phase are derived using the Lagrangian method. Simulation results are compared to experimental data collected on a subject stepping four times over an obstacle using a Selspot motion analysis system. A hierarchical control model that incorporates a learning process is proposed. The model allows an efficient combination of open and closed loop control strategies and involves hardwired movement segments. We also test the hypothesis of cycloidal velocity profiles in the joint programs against experimental data using a novel curve-fitting procedure based on analytical rather than numerical differentiation. The results suggest multiob-jective optimization of the joint's motion. The control and learning model proposed here will help the understanding of the mechanisms responsible for assembling selected movement segments into goaldirected movement sequences in humans.     

A model for sensorimotor control and learning

A model for motor learning, generalization, and adaptation is presented. It is shown that the equations of motion of a limb can be expressed in a parametric form that facilitates transformation of desired trajectories into plans. These parametric equations are used in conjunction with a quantized multidimensional memory organized by state variables. The memory is supplied with data derived from the analysis of practice movements. A small computer and mechanical arm are used to implement the model and study its properties. Results verify the ability to acquire new movements, adapt to mechanical loads, and generalize between similar movements.This research was done while the author was a graduate student at the Massachusetts Institute of Technology in the Artificial Intelligence Laboratory and Department of Psychology. It was supported in part by training grant NGMS 5-T01-GM01064-15     

Fractal dimensions of pacing and grip force in drawing and handwriting production

We performed a repeated measures experiment to show that the pacing of a cyclic, ballistic drawing task has a fractal dimension. We also estimated the dimensionality of the force used to grip the drawing implement. Finally, we present an analysis of pediatric data to show that grip force has a fractal dimension in an actual handwriting task. In our experiment, subjects drew circles of varying sizes and at varying rates on a digitizing tablet, using a pen instrumented to measure radial force applied to its barrel. Subjects also drew circles in synchrony with a metronome. We found strong evidence for fractal scaling of both drawing period and grip force in the circle-drawing study. The dimensionality ranged from fractal Gaussian noise (fGn) to fractal Brownian motion, with Hurst coefficients clustering around the value for 1/f noise. When the subjects were required to synchronize their drawing with a metronome, the Hurst coefficient for the drawing period decreased, while the coefficient for grip force did not. This result indicates that independent processes control the variations in pacing and grip force. Grip force in the handwriting study also displayed fractal properties, with Hurst coefficients in the range of correlated fGn. We draw parallels between our handwriting measurements and studies of human gait.     

Cursive word recognition based on interactive activation and early visual processing models

We present an off-line cursive word recognition system based completely on neural networks: reading models and models of early visual processing. The first stage (normalization) preprocesses the input image in order to reduce letter position uncertainty; the second stage (feature extraction) is based on the feedforward model of orientation selectivity; the third stage (letter pre-recognition) is based on a convolutional neural network, and the last stage (word recognition) is based on the interactive activation model.