Computational approaches to motor learning by imitation

Movement imitation requires a complex set of mechanisms that map an observed movement of a teacher onto one's own movement apparatus. Relevant problems include movement recognition, pose estimation, pose tracking, body correspondence, coordinate transformation from external to egocentric space, matching of observed against previously learned movement, resolution of redundant degrees-of-freedom that are unconstrained by the observation, suitable movement representations for imitation, modularization of motor control, etc. All of these topics by themselves are active research problems in computational and neurobiological sciences, such that their combination into a complete imitation system remains a daunting undertaking-indeed, one could argue that we need to understand the complete perception-action loop. As a strategy to untangle the complexity of imitation, this paper will examine imitation purely from a computational point of view, i.e. we will review statistical and mathematical approaches that have been suggested for tackling parts of the imitation problem, and discuss their merits, disadvantages and underlying principles. Given the focus on action recognition of other contributions in this special issue, this paper will primarily emphasize the motor side of imitation, assuming that a perceptual system has already identified important features of a demonstrated movement and created their corresponding spatial information. Based on the formalization of motor control in terms of control policies and their associated performance criteria, useful taxonomies of imitation learning can be generated that clarify different approaches and future research directions.     

Viewing lip forms: cortical dynamics

Viewing other persons' actions automatically activates brain areas belonging to the mirror-neuron system (MNS) assumed to link action execution and observation. We followed, by magnetoencephalographic cortical dynamics, subjects who observed still pictures of lip forms, on-line imitated them, or made similar forms in a self-paced manner. In all conditions and in both hemispheres, cortical activation progressed in 20-70 ms steps from the occipital cortex to the superior temporal region (where the strongest activation took place), the inferior parietal lobule, and the inferior frontal lobe (Broca's area), and finally, 50-140 ms later, to the primary motor cortex. The signals of Broca's area and motor cortex were significantly stronger during imitation than other conditions. These results demonstrate that still pictures, only implying motion, activate the human MNS in a well-defined temporal order.     

Human primary motor cortex is both activated and stabilized during observation of other person's phasic motor actions

When your favourite athlete flops over the high-jump bar, you may twist your body in front of the TV screen. Such automatic motor facilitation, ‘mirroring’ or even overt imitation is not always appropriate. Here, we show, by monitoring motor-cortex brain rhythms with magnetoencephalography (MEG) in healthy adults, that viewing intermittent hand actions of another person, in addition to activation, phasically stabilizes the viewer''s primary motor cortex, with the maximum of half a second after the onset of the seen movement. Such a stabilization was evident as enhanced cortex–muscle coherence at 16–20 Hz, despite signs of almost simultaneous suppression of rolandic rhythms of approximately 7 and 15 Hz as a sign of activation of the sensorimotor cortex. These findings suggest that inhibition suppresses motor output during viewing another person''s actions, thereby withholding unintentional imitation.     

Imitation as faithful copying of a novel technique in marmoset monkeys

Imitative learning has received great attention due to its supposed role in the development of culture and the cognitive demands it poses on the individual. Evidence for imitation in non-human primate species, therefore, could shed light on the early origins of proto-cultural traits in the primate order. Imitation has been defined as the learning of an act by seeing it done or, more specifically, as the copying of a novel or otherwise improbable act. But despite a century of research and the detection of mirror neurons the empirical basis for this most advanced form of observational learning is weak. Few, if any, studies have shown that the observer has learned the response topography, i.e., the specific action by which the response is made. In an experimental set-up we confronted marmoset monkeys (Callithrix jacchus) with a conspecific model that was previously trained to open a plastic box in a peculiar way. Employing detailed motion analyses we show that the observers precisely copied the movement patterns of the novel action demonstrated by the model. A discriminant analysis classified 13 out of 14 observer movements (92.86%) as model movements and only one as non-observer movement. This evidence of imitation in non-human primates questions the dominant opinion that imitation is a human-specific ability. Furthermore, the high matching degree suggests that marmosets possess the neuronal mechanism to code the actions of others and to map them onto their own motor repertoire, rather than priming existing motor-templates.     

Influence of Action-Effect Associations Acquired by Ideomotor Learning on Imitation

According to the ideomotor theory, actions are represented in terms of their perceptual effects, offering a solution for the correspondence problem of imitation (how to translate the observed action into a corresponding motor output). This effect-based coding of action is assumed to be acquired through action-effect learning. Accordingly, performing an action leads to the integration of the perceptual codes of the action effects with the motor commands that brought them about. While ideomotor theory is invoked to account for imitation, the influence of action-effect learning on imitative behavior remains unexplored. In two experiments, imitative performance was measured in a reaction time task following a phase of action-effect acquisition. During action-effect acquisition, participants freely executed a finger movement (index or little finger lifting), and then observed a similar (compatible learning) or a different (incompatible learning) movement. In Experiment 1, finger movements of left and right hands were presented as action-effects during acquisition. In Experiment 2, only right-hand finger movements were presented during action-effect acquisition and in the imitation task the observed hands were oriented orthogonally to participants’ hands in order to avoid spatial congruency effects. Experiments 1 and 2 showed that imitative performance was improved after compatible learning, compared to incompatible learning. In Experiment 2, although action-effect learning involved perception of finger movements of right hand only, imitative capabilities of right- and left-hand finger movements were equally affected. These results indicate that an observed movement stimulus processed as the effect of an action can later prime execution of that action, confirming the ideomotor approach to imitation. We further discuss these findings in relation to previous studies of action-effect learning and in the framework of current ideomotor approaches to imitation.     

Nonvisual motor learning influences abstract action observation

Neuroimaging studies have recently provided support for the existence of a human equivalent of the "mirror-neuron" system as first described in monkeys [1], involved in both the execution of movements as well as the observation and imitation of actions performed by others (e.g., [2-6]). A widely held conception concerning this system is that the understanding of observed actions is mediated by a covert simulation process [7]. In the present fMRI experiment, this simulation process was probed by asking subjects to discriminate between visually presented trajectories that either did or did not match previously performed but unseen continuous movement sequences. A specific network of learning-related premotor and parietal areas was found to be reactivated when participants were confronted with their movements' visual counterpart. Moreover, the strength of these reactivations was dependent on the observers' experience with executing the corresponding movement sequence. These findings provide further support for the emerging view that embodied simulations during action observation engage widespread activations in cortical motor regions beyond the classically defined mirror-neuron system. Furthermore, the obtained results extend previous work by showing experience-dependent perceptual modulations at the neural systems level based on nonvisual motor learning.     

Enhanced Activation of Motor Execution Networks Using Action Observation Combined with Imagination of Lower Limb Movements

The combination of first-person observation and motor imagery, i.e. first-person observation of limbs with online motor imagination, is commonly used in interactive 3D computer gaming and in some movie scenes. These scenarios are designed to induce a cognitive process in which a subject imagines himself/herself acting as the agent in the displayed movement situation. Despite the ubiquity of this type of interaction and its therapeutic potential, its relationship to passive observation and imitation during observation has not been directly studied using an interactive paradigm. In the present study we show activation resulting from observation, coupled with online imagination and with online imitation of a goal-directed lower limb movement using functional MRI (fMRI) in a mixed block/event-related design. Healthy volunteers viewed a video (first-person perspective) of a foot kicking a ball. They were instructed to observe-only the action (O), observe and simultaneously imagine performing the action (O-MI), or imitate the action (O-IMIT). We found that when O-MI was compared to O, activation was enhanced in the ventralpremotor cortex bilaterally, left inferior parietal lobule and left insula. The O-MI and O-IMIT conditions shared many activation foci in motor relevant areas as confirmed by conjunction analysis. These results show that (i) combining observation with motor imagery (O-MI) enhances activation compared to observation-only (O) in the relevant foot motor network and in regions responsible for attention, for control of goal-directed movements and for the awareness of causing an action, and (ii) it is possible to extensively activate the motor execution network using O-MI, even in the absence of overt movement. Our results may have implications for the development of novel virtual reality interactions for neurorehabilitation interventions and other applications involving training of motor tasks.     

The Study of Object-Oriented Motor Imagery Based on EEG Suppression

Motor imagery is a conventional method for brain computer interface and motor learning. To avoid the great individual difference of the motor imagery ability, object-oriented motor imagery was applied, and the effects were studied. Kinesthetic motor imagery and visual observation were administered to 15 healthy volunteers. The EEG during cue-based simple imagery (SI), object-oriented motor imagery (OI), non-object-oriented motor imagery (NI) and visual observation (VO) was recorded. Study results showed that OI and NI presented significant contralateral suppression in mu rhythm (p < 0.05). Besides, OI exhibited significant contralateral suppression in beta rhythm (p < 0.05). While no significant mu or beta contralateral suppression could be found during VO or SI (p > 0.05). Compared with NI, OI showed significant difference (p < 0.05) in mu rhythm and weak significant difference (p = 0.0612) in beta rhythm over the contralateral hemisphere. The ability of motor imagery can be reflected by the suppression degree of mu and beta frequencies which are the motor related rhythms. Thus, greater enhancement of activation in mirror neuron system is involved in response to object-oriented motor imagery. The object-oriented motor imagery is favorable for improvement of motor imagery ability.     

Causality in the association between P300 and alpha event-related desynchronization

Recent findings indicated that both P300 and alpha event-related desynchronization (α-ERD) were associated, and similarly involved in cognitive brain functioning, e.g., attention allocation and memory updating. However, an explicit causal influence between the neural generators of P300 and α-ERD has not yet been investigated. In the present study, using an oddball task paradigm, we assessed the task effect (target vs. non-target) on P300 and α-ERD elicited by stimuli of four sensory modalities, i.e., audition, vision, somatosensory, and pain, estimated their respective neural generators, and investigated the information flow among their neural generators using time-varying effective connectivity in the target condition. Across sensory modalities, the scalp topographies of P300 and α-ERD were similar and respectively maximal at parietal and occipital regions in the target condition. Source analysis revealed that P300 and α-ERD were mainly generated from posterior cingulate cortex and occipital lobe respectively. As revealed by time-varying effective connectivity, the cortical information was consistently flowed from α-ERD sources to P300 sources in the target condition for all four sensory modalities. All these findings showed that P300 in the target condition is modulated by the changes of α-ERD, which would be useful to explore neural mechanism of cognitive information processing in the human brain.     

Neurofeedback training produces normalization in behavioural and electrophysiological measures of high-functioning autism

Autism spectrum disorder (ASD) is a neurodevelopmental condition exhibiting impairments in behaviour, social and communication skills. These deficits may arise from aberrant functional connections that impact synchronization and effective neural communication. Neurofeedback training (NFT), based on operant conditioning of the electroencephalogram (EEG), has shown promise in addressing abnormalities in functional and structural connectivity. We tested the efficacy of NFT in reducing symptoms in children with ASD by targeting training to the mirror neuron system (MNS) via modulation of EEG mu rhythms. The human MNS has provided a neurobiological substrate for understanding concepts in social cognition relevant to behavioural and cognitive deficits observed in ASD. Furthermore, mu rhythms resemble MNS phenomenology supporting the argument that they are linked to perception and action. Thirty hours of NFT on ASD and typically developing (TD) children were assessed. Both groups completed an eyes-open/-closed EEG session as well as a mu suppression index assessment before and after training. Parents filled out pre- and post-behavioural questionnaires. The results showed improvements in ASD subjects but not in TDs. This suggests that induction of neuroplastic changes via NFT can normalize dysfunctional mirroring networks in children with autism, but the benefits are different for TD brains.     

Attention modulation regulates both motor and non-motor performance: a high-density EEG study in Parkinson's disease

We have previously shown that, in early stages of Parkinson's disease (PD), patients with higher reaction times are also more impaired in visual sequence learning, suggesting that movement preparation shares resources with the learning of visuospatial sequences. Here, we ascertained whether, in patients with PD, the pattern of the neural correlates of attentional processes of movement planning predict sequence learning and working memory abilities. High density Electroencephalography (EEG, 256 electrodes) was recorded in 19 patients with PD performing reaching movements in a choice reaction time paradigm. Patients were also tested with Digit Span and performed a visuomotor sequence learning task that has an important declarative learning component. We found that attenuation of alpha/beta oscillatory activity before the stimulus presentation in frontoparietal regions significantly correlated with reaction time in the choice reaction time task, similarly to what we had previously found in normal subjects. In addition, such activity significantly predicted the declarative indices of sequence learning and the scores in the Digit Span task. These findings suggest that some motor and non motor PD signs might have common neural bases, and thus, might have a similar response to the same behavioral therapy. In addition, these results might help in designing and testing the efficacy of novel rehabilitative approaches to improve specific aspects of motor performance in PD and other neurological disorders.     

Topographic characteristics of cortex activity in delta-, alpha2-, and gamma2- frequency bands related to social creativity

Social creativity-related topographic changes of the delta, alpha2, and gamma2 power were studied using 19-channel EEG. Originality and fluency indices of social thinking were evaluated on basis of specially developed 'divergent' task solution that stimulated different interpretations of social scenes; complex visual stimuli (architectural building) were used as control task. 'Divergent' task performing was characterized by both the greater power of the delta and alpha2 rhythms mostly in the right hemisphere and widespread increase in the gamma2 power as compared with the control task. Positive correlations between delta and gamma2 rhythms in baseline condition were revealed in the fronto-parietal cortex, and this relationship between low- and high-frequency oscillations while 'divergent' task performing was found in the right posterior cortex. Special topographic patterns of delta and gamma2 activity as predictors of social creativity were obtained using multiple regression analysis. These patterns can be interpreted as EEG correlates of a summation of endogenous and exogenous components of social thinking.     

Aplasics born without hands mirror the goal of hand actions with their feet

The premotor and parietal mirror neuron system (MNS) is thought to contribute to the understanding of observed actions by mapping them onto "corresponding" motor programs of the observer [1-24], but how would the MNS respond to the observation of hand actions if the observer never had hands? Would it not show changes of blood-oxygen-level dependent (BOLD) signal, because the observer lacks motor programs that can resonate [12, 25, 26], or would it show significant changes because the observer has motor programs for the foot or mouth with corresponding goals [15, 17, 19, 27, 28]? We scanned two aplasic subjects, born without arms or hands, while they watched hand actions and compared their brain activity with that of 16 control subjects. All subjects additionally executed actions with different effectors (feet, mouth, and, for controls, hands). The BOLD signal of aplasic individuals within the putative MNS was augmented when they watched hand actions, demonstrating the brain's capacity to mirror actions that deviate from the embodiment of the observer by recruiting voxels involved in the execution of actions that achieve corresponding goals by different effectors. This sheds light on the functional organization of the MNS and predominance of goals in imitation.     

Automatic Imitation in Rhythmical Actions: Kinematic Fidelity and the Effects of Compatibility,Delay, and Visual Monitoring

We demonstrate that observation of everyday rhythmical actions biases subsequent motor execution of the same and of different actions, using a paradigm where the observed actions were irrelevant for action execution. The cycle time of the distractor actions was subtly manipulated across trials, and the cycle time of motor responses served as the main dependent measure. Although distractor frequencies reliably biased response cycle times, this imitation bias was only a small fraction of the modulations in distractor speed, as well as of the modulations produced when participants intentionally imitated the observed rhythms. Importantly, this bias was not only present for compatible actions, but was also found, though numerically reduced, when distractor and executed actions were different (e.g., tooth brushing vs. window wiping), or when the dominant plane of movement was different (horizontal vs. vertical). In addition, these effects were equally pronounced for execution at 0, 4, and 8 s after action observation, a finding that contrasts with the more short-lived effects reported in earlier studies. The imitation bias was also unaffected when vision of the hand was occluded during execution, indicating that this effect most likely resulted from visuomotor interactions during distractor observation, rather than from visual monitoring and guidance during execution. Finally, when the distractor was incompatible in both dimensions (action type and plane) the imitation bias was not reduced further, in an additive way, relative to the single-incompatible conditions. This points to a mechanism whereby the observed action’s impact on motor processing is generally reduced whenever this is not useful for motor planning. We interpret these findings in the framework of biased competition, where intended and distractor actions can be represented as competing and quasi-encapsulated sensorimotor streams.     

How frontoparietal brain regions mediate imitative and complementary actions: an FMRI study

The 'mirror neuron system' (MNS), located within inferior parietal lobe (IPL) and inferior frontal gyrus (IFG), creates an internal motor representation of the actions we see and has been implicated in imitation. Recently, the MNS has been implicated in non-identical responses: when the actions we must execute do not match those that we observe. However, in such conflicting situations non action-specific cognitive control networks also located in frontoparietal regions may be involved. In the present functional magnetic resonance imaging (fMRI) study participants made both similar and dissimilar actions within two action contexts: imitative and complementary. We aimed to determine whether activity within IPL/IFG depends on (i) responding under an imitative versus complementary context (ii) responding with similar versus dissimilar responses, and (iii) observing hand actions versus symbolic arrow cue stimuli. Activity within rIPL/rIFG regions was largest during observation of hand actions compared with arrow cues. Specifically, rIPL/rIFG were recruited only during the imitative context, when participants responded with similar actions. When responding to symbolic arrow cues, rIPL/rIFG activity increased during dissimilar responses, reflecting increased demands placed on general cognitive control mechanisms. These results suggest a specific role of rIPL/rIFG during imitation of hand actions, and also a general role of frontoparietal areas in mediating dissimilar responses to both hand actions and symbolic stimuli. We discuss our findings in relation to recent work that has examined the role of frontoparietal brain structures in joint-actions and inter-actor cooperation. We conclude that the specific brain regions identified here to show increased activation during action observation conditions are likely to form part of a mechanism specifically involved in matching observed actions directly with internal motor plans. Conversely, observation of arrow cues recruited part of a wider cognitive control network involved in the rapid remapping of stimulus-response associations.     

Autism and the mirror neuron system: insights from learning and teaching

Individuals with autism have difficulties in social learning domains which typically involve mirror neuron system (MNS) activation. However, the precise role of the MNS in the development of autism and its relevance to treatment remain unclear. In this paper, we argue that three distinct aspects of social learning are critical for advancing knowledge in this area: (i) the mechanisms that allow for the implicit mapping of and learning from others'' behaviour, (ii) the motivation to attend to and model conspecifics and (iii) the flexible and selective use of social learning. These factors are key targets of the Early Start Denver Model, an autism treatment approach which emphasizes social imitation, dyadic engagement, verbal and non-verbal communication and affect sharing. Analysis of the developmental processes and treatment-related changes in these different aspects of social learning in autism can shed light on the nature of the neuropsychological mechanisms underlying social learning and positive treatment outcomes in autism. This knowledge in turn may assist in developing more successful pedagogic approaches to autism spectrum disorder. Thus, intervention research can inform the debate on relations among neuropsychology of social learning, the role of the MNS, and educational practice in autism.     

Infants’ Somatotopic Neural Responses to Seeing Human Actions: I’ve Got You under My Skin

Human infants rapidly learn new skills and customs via imitation, but the neural linkages between action perception and production are not well understood. Neuroscience studies in adults suggest that a key component of imitation–identifying the corresponding body part used in the acts of self and other–has an organized neural signature. In adults, perceiving someone using a specific body part (e.g., hand vs. foot) is associated with activation of the corresponding area of the sensory and/or motor strip in the observer’s brain–a phenomenon called neural somatotopy. Here we examine whether preverbal infants also exhibit somatotopic neural responses during the observation of others’ actions. 14-month-old infants were randomly assigned to watch an adult reach towards and touch an object using either her hand or her foot. The scalp electroencephalogram (EEG) was recorded and event-related changes in the sensorimotor mu rhythm were analyzed. Mu rhythm desynchronization was greater over hand areas of sensorimotor cortex during observation of hand actions and was greater over the foot area for observation of foot actions. This provides the first evidence that infants’ observation of someone else using a particular body part activates the corresponding areas of sensorimotor cortex. We hypothesize that this somatotopic organization in the developing brain supports imitation and cultural learning. The findings connect developmental cognitive neuroscience, adult neuroscience, action representation, and behavioral imitation.     

Automatic versus voluntary motor imitation: effect of visual context and stimulus velocity

Automatic imitation is the tendency to reproduce observed actions involuntarily. Though this topic has been widely treated, at present little is known about the automatic imitation of the kinematic features of an observed movement. The present study was designed to understand if the kinematics of a previously seen stimulus primes the executed action, and if this effect is sensitive to the kinds of stimuli presented. We proposed a simple imitation paradigm in which a dot or a human demonstrator moved in front of the participant who was instructed either to reach the final position of the stimulus or to imitate its motion with his or her right arm. Participants' movements were automatically contaminated by stimulus velocity when it moved according to biological laws, suggesting that automatic imitation was kinematic dependent. Despite that the performance, in term of reproduced velocity, improved in a context of voluntary imitation, subjects did not replicate the observed motions exactly. These effects were not affected by the kind of stimuli used, i.e., motor responses were influenced in the same manner after dot or human observation. These findings support the existence of low-level sensory-motor matching mechanisms that work on movement planning and represent the basis for higher levels of social interaction.     

View-Invariant Visuomotor Processing in Computational Mirror Neuron System for Humanoid

Mirror neurons are visuo-motor neurons found in primates and thought to be significant for imitation learning. The proposition that mirror neurons result from associative learning while the neonate observes his own actions has received noteworthy empirical support. Self-exploration is regarded as a procedure by which infants become perceptually observant to their own body and engage in a perceptual communication with themselves. We assume that crude sense of self is the prerequisite for social interaction. However, the contribution of mirror neurons in encoding the perspective from which the motor acts of others are seen have not been addressed in relation to humanoid robots. In this paper we present a computational model for development of mirror neuron system for humanoid based on the hypothesis that infants acquire MNS by sensorimotor associative learning through self-exploration capable of sustaining early imitation skills. The purpose of our proposed model is to take into account the view-dependency of neurons as a probable outcome of the associative connectivity between motor and visual information. In our experiment, a humanoid robot stands in front of a mirror (represented through self-image using camera) in order to obtain the associative relationship between his own motor generated actions and his own visual body-image. In the learning process the network first forms mapping from each motor representation onto visual representation from the self-exploratory perspective. Afterwards, the representation of the motor commands is learned to be associated with all possible visual perspectives. The complete architecture was evaluated by simulation experiments performed on DARwIn-OP humanoid robot.     

When Money Is Not Enough: Awareness,Success, and Variability in Motor Learning

When performing a skill such as throwing a dart, many different combinations of joint motions suffice to hit the target. The motor system adapts rapidly to reduce bias in the desired outcome (i.e., the first-order moment of the error); however, the essence of skill is to produce movements with less variability (i.e., to reduce the second-order moment). It is easy to see how feedback about success or failure could sculpt performance to achieve this aim. However, it is unclear whether the dimensions responsible for success or failure need to be known explicitly by the subjects, or whether learning can proceed without explicit awareness of the movement parameters that need to change. Here, we designed a redundant, two-dimensional reaching task in which we could selectively manipulate task success and the variability of action outcomes, whilst also manipulating awareness of the dimension along which performance could be improved. Variability was manipulated either by amplifying natural errors, leaving the correlation between the executed movement and the visual feedback intact, or by adding extrinsic noise, decorrelating movement and feedback. We found that explicit, binary, feedback about success or failure was only sufficient for learning when participants were aware of the dimension along which motor behavior had to change. Without such awareness, learning was only present when extrinsic noise was added to the feedback, but not when task success or variability was manipulated in isolation; learning was also much slower. Our results highlight the importance of conscious awareness of the relevant dimension during motor learning, and suggest that higher-order moments of outcome signals are likely to play a significant role in skill learning in complex tasks.