Motor Contagion during Human-Human and Human-Robot Interaction

Motor resonance mechanisms are known to affect humans'' ability to interact with others, yielding the kind of “mutual understanding” that is the basis of social interaction. However, it remains unclear how the partner''s action features combine or compete to promote or prevent motor resonance during interaction. To clarify this point, the present study tested whether and how the nature of the visual stimulus and the properties of the observed actions influence observer''s motor response, being motor contagion one of the behavioral manifestations of motor resonance. Participants observed a humanoid robot and a human agent move their hands into a pre-specified final position or put an object into a container at various velocities. Their movements, both in the object- and non-object- directed conditions, were characterized by either a smooth/curvilinear or a jerky/segmented trajectory. These trajectories were covered with biological or non-biological kinematics (the latter only by the humanoid robot). After action observation, participants were requested to either reach the indicated final position or to transport a similar object into another container. Results showed that motor contagion appeared for both the interactive partner except when the humanoid robot violated the biological laws of motion. These findings suggest that the observer may transiently match his/her own motor repertoire to that of the observed agent. This matching might mediate the activation of motor resonance, and modulate the spontaneity and the pleasantness of the interaction, whatever the nature of the communication partner.     

Macaques (Macaca nemestrina) recognize when they are being imitated

This study investigated whether monkeys recognize when a human experimenter imitates their actions towards an object. Two experimenters faced 10 pigtailed macaques, who were given access to an interesting object. One experimenter imitated the monkeys' object-directed actions, the other performed temporally contingent but structurally different object-directed actions. Results show a significant visual preference for the imitator during manual object manipulations, but not mouthing actions. We argue that the ability to match actions could be based on both visual-visual and kinaesthetic-visual matching skills, and that mirror neurons, which have both visual and motor properties, could serve as a neural basis for recognizing imitation. However, imitation recognition as assessed by visual preference does not necessarily imply the capacity to attribute imitative intentionality to the imitator. The monkeys might implicitly recognize when they are being imitated without deeper insight into the mental processes of others.     

Homo imitans? Seven reasons why imitation couldn't possibly be associative

Many comparative and developmental psychologists believe that we are Homo imitans; humans are more skilled and prolific imitators than other animals, because we have a special, inborn ‘intermodal matching’ mechanism that integrates representations of others with representations of the self. In contrast, the associative sequence learning (ASL) model suggests that human infants learn to imitate using mechanisms that they share with other animals, and the rich resources provided by their sociocultural environments. This article answers seven objections to the ASL model: (i) it presents evidence that newborns do not imitate; (ii) argues that infants receive a plentiful supply of the kind of experience necessary for learning to imitate; (iii) suggests that neither infants nor adults can imitate elementally novel actions; (iv) explains why non-human animals have a limited capacity for imitation; (v) discusses the goal-directedness of imitation; (vi) presents evidence that improvement in imitation depends on visual feedback; and (vii) reflects on the view that associative theories steal ‘the soul of imitation’. The empirical success of the ASL model indicates that the mechanisms which make imitation possible, by aligning representations of self with representations of others, have been tweaked by cultural evolution, not built from scratch by genetic evolution.     

A Bayesian Developmental Approach to Robotic Goal-Based Imitation Learning

A fundamental challenge in robotics today is building robots that can learn new skills by observing humans and imitating human actions. We propose a new Bayesian approach to robotic learning by imitation inspired by the developmental hypothesis that children use self-experience to bootstrap the process of intention recognition and goal-based imitation. Our approach allows an autonomous agent to: (i) learn probabilistic models of actions through self-discovery and experience, (ii) utilize these learned models for inferring the goals of human actions, and (iii) perform goal-based imitation for robotic learning and human-robot collaboration. Such an approach allows a robot to leverage its increasing repertoire of learned behaviors to interpret increasingly complex human actions and use the inferred goals for imitation, even when the robot has very different actuators from humans. We demonstrate our approach using two different scenarios: (i) a simulated robot that learns human-like gaze following behavior, and (ii) a robot that learns to imitate human actions in a tabletop organization task. In both cases, the agent learns a probabilistic model of its own actions, and uses this model for goal inference and goal-based imitation. We also show that the robotic agent can use its probabilistic model to seek human assistance when it recognizes that its inferred actions are too uncertain, risky, or impossible to perform, thereby opening the door to human-robot collaboration.     

Hebbian learning and predictive mirror neurons for actions,sensations and emotions

Spike-timing-dependent plasticity is considered the neurophysiological basis of Hebbian learning and has been shown to be sensitive to both contingency and contiguity between pre- and postsynaptic activity. Here, we will examine how applying this Hebbian learning rule to a system of interconnected neurons in the presence of direct or indirect re-afference (e.g. seeing/hearing one''s own actions) predicts the emergence of mirror neurons with predictive properties. In this framework, we analyse how mirror neurons become a dynamic system that performs active inferences about the actions of others and allows joint actions despite sensorimotor delays. We explore how this system performs a projection of the self onto others, with egocentric biases to contribute to mind-reading. Finally, we argue that Hebbian learning predicts mirror-like neurons for sensations and emotions and review evidence for the presence of such vicarious activations outside the motor system.     

Human motor cortex excitability during the perception of others' action

Neuroscience research during the past ten years has fundamentally changed the traditional view of the motor system. In monkeys, the finding that premotor neurons also discharge during visual stimulation (visuomotor neurons) raises new hypotheses about the putative role played by motor representations in perceptual functions. Among visuomotor neurons, mirror neurons might be involved in understanding the actions of others and might, therefore, be crucial in interindividual communication. Functional brain imaging studies enabled us to localize the human mirror system, but the demonstration that the motor cortex dynamically replicates the observed actions, as if they were executed by the observer, can only be given by fast and focal measurements of cortical activity. Transcranial magnetic stimulation enables us to instantaneously estimate corticospinal excitability, and has been used to study the human mirror system at work during the perception of actions performed by other individuals. In the past ten years several TMS experiments have been performed investigating the involvement of motor system during others' action observation. Results suggest that when we observe another individual acting we strongly 'resonate' with his or her action. In other words, our motor system simulates underthreshold the observed action in a strictly congruent fashion. The involved muscles are the same as those used in the observed action and their activation is temporally strictly coupled with the dynamics of the observed action.     

The mirror mechanism: recent findings and perspectives

Mirror neurons are a specific type of visuomotor neuron that discharge both when a monkey executes a motor act and when it observes a similar motor act performed by another individual. In this article, we review first the basic properties of these neurons. We then describe visual features recently investigated which indicate that, besides encoding the goal of motor acts, mirror neurons are modulated by location in space of the observed motor acts, by the perspective from which the others’ motor acts are seen, and by the value associated with the object on which others’ motor acts are performed. In the last part of this article, we discuss the role of the mirror mechanism in planning actions and in understanding the intention underlying the others’ motor acts. We also review some human studies suggesting that motor intention in humans may rely, as in the monkey, on the mirror mechanism.     

Learning to understand others' actions

Despite nearly two decades of research on mirror neurons, there is still much debate about what they do. The most enduring hypothesis is that they enable 'action understanding'. However, recent critical reviews have failed to find compelling evidence in favour of this view. Instead, these authors argue that mirror neurons are produced by associative learning and therefore that they cannot contribute to action understanding. The present opinion piece suggests that this argument is flawed. We argue that mirror neurons may both develop through associative learning and contribute to inferences about the actions of others.     

Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons

A mirror system is active both when an animal executes a class of actions (self-actions) and when it sees another execute an action of that class. Much attention has been given to the possible roles of mirror systems in responding to the actions of others but there has been little attention paid to their role in self-actions. In the companion article (Bonaiuto et al. Biol Cybern 96:9–38, 2007) we presented MNS2, an extension of the Mirror Neuron System model of the monkey mirror system trained to recognize the external appearance of its own actions as a basis for recognizing the actions of other animals when they perform similar actions. Here we further extend the study of the mirror system by introducing the novel hypotheses that a mirror system may additionally help in monitoring the success of a self-action and may also be activated by recognition of one’s own apparent actions as well as efference copy from one’s intended actions. The framework for this computational demonstration is a model of action sequencing, called augmented competitive queuing, in which action choice is based on the desirability of executable actions. We show how this “what did I just do?” function of mirror neurons can contribute to the learning of both executability and desirability which in certain cases supports rapid reorganization of motor programs in the face of disruptions.     

View-based encoding of actions in mirror neurons of area f5 in macaque premotor cortex

Converging experimental evidence indicates that mirror neurons in the monkey premotor area F5 encode the goals of observed motor acts [1-3]. However, it is unknown whether they also contribute to encoding the perspective from which the motor acts of others are seen. In order to address this issue, we recorded the visual responses of mirror neurons of monkey area F5 by using a novel experimental paradigm based on the presentation of movies showing grasping motor acts from different visual perspectives. We found that the majority of the tested mirror neurons?(74%) exhibited view-dependent activity with responses tuned to specific points of view. A minority of the tested mirror neurons (26%) exhibited view-independent responses. We conclude that view-independent mirror neurons encode action goals irrespective of the details of the observed motor acts, whereas the view-dependent ones might either form an intermediate step in the formation of view independence or contribute to a modulation of view-dependent representations in higher-level visual areas, potentially linking the goals of observed motor acts with their pictorial aspects.     

The role of ventral premotor cortex in action execution and action understanding

The human ventral premotor cortex overlaps, at least in part, with Broca's region in the dominant cerebral hemisphere, that is known to mediate the production of language and contributes to language comprehension. This region is constituted of Brodmann's areas 44 and 45 in the inferior frontal gyrus. We summarize the evidence that the motor related part of Broca's region is localized in the opercular portion of the inferior frontal cortex, mainly in area 44 of Brodmann. According to our own data, there seems to be a homology between Brodmann area 44 in humans and the monkey area F5. The non-language related motor functions of Broca's region comprise complex hand movements, associative sensorimotor learning and sensorimotor integration. Brodmann's area 44 is also a part of a specialized parieto-premotor network and interacts significantly with the neighbouring premotor areas. In the ventral premotor area F5 of monkeys, the so called mirror neurons have been found which discharge both when the animal performs a goal-directed hand action and when it observes another individual performing the same or a similar action. More recently, in the same area mirror neurons responding not only to the observation of mouth actions, but also to sounds characteristic to actions have been found. In humans, through an fMRI study, it has been shown that the observation of actions performed with the hand, the mouth and the foot leads to the activation of different sectors of Broca's area and premotor cortex, according to the effector involved in the observed action, following a somatotopic pattern which resembles the classical motor cortex homunculus. On the other hand the evidence is growing that human ventral premotor cortex, especially Brodmann's area 44, is involved in polymodal action processing. These results strongly support the existence of an execution-observation matching system (mirror neuron system). It has been proposed that this system is involved in polymodal action recognition and might represent a precursor of language processing. Experimental evidence in favour of this hypothesis both in the monkey and humans is shortly reviewed.     

Mirroring and the development of action understanding

The discovery of mirror neurons in the monkey motor cortex has inspired wide-ranging hypotheses about the potential relationship between action control and social cognition. In this paper, we consider the hypothesis that this relationship supports the early development of a critical aspect of social understanding, the ability to analyse others’ actions in terms of goals. Recent investigations of infant action understanding have revealed rich connections between motor development and the analysis of goals in others’ actions. In particular, infants’ own goal-directed actions influence their analysis of others’ goals. This evidence indicates that the cognitive systems that drive infants’ own actions contribute to their analysis of goals in others’ actions. These effects occur at a relatively abstract level of analysis both in terms of the structure infants perceive in others’ actions and relevant structure in infants’ own actions. Although the neural bases of these effects in infants are not yet well understood, current evidence indicates that connections between action production and action perception in infancy involve the interrelated neural systems at work in generating planned, intelligent action.     

Brain Response to a Humanoid Robot in Areas Implicated in the Perception of Human Emotional Gestures

Background

The humanoid robot WE4-RII was designed to express human emotions in order to improve human-robot interaction. We can read the emotions depicted in its gestures, yet might utilize different neural processes than those used for reading the emotions in human agents.

Methodology

Here, fMRI was used to assess how brain areas activated by the perception of human basic emotions (facial expression of Anger, Joy, Disgust) and silent speech respond to a humanoid robot impersonating the same emotions, while participants were instructed to attend either to the emotion or to the motion depicted.

Principal Findings

Increased responses to robot compared to human stimuli in the occipital and posterior temporal cortices suggest additional visual processing when perceiving a mechanical anthropomorphic agent. In contrast, activity in cortical areas endowed with mirror properties, like left Broca''s area for the perception of speech, and in the processing of emotions like the left anterior insula for the perception of disgust and the orbitofrontal cortex for the perception of anger, is reduced for robot stimuli, suggesting lesser resonance with the mechanical agent. Finally, instructions to explicitly attend to the emotion significantly increased response to robot, but not human facial expressions in the anterior part of the left inferior frontal gyrus, a neural marker of motor resonance.

Conclusions

Motor resonance towards a humanoid robot, but not a human, display of facial emotion is increased when attention is directed towards judging emotions.

Significance

Artificial agents can be used to assess how factors like anthropomorphism affect neural response to the perception of human actions.     

Infants’ Somatotopic Neural Responses to Seeing Human Actions: I’ve Got You under My Skin

Human infants rapidly learn new skills and customs via imitation, but the neural linkages between action perception and production are not well understood. Neuroscience studies in adults suggest that a key component of imitation–identifying the corresponding body part used in the acts of self and other–has an organized neural signature. In adults, perceiving someone using a specific body part (e.g., hand vs. foot) is associated with activation of the corresponding area of the sensory and/or motor strip in the observer’s brain–a phenomenon called neural somatotopy. Here we examine whether preverbal infants also exhibit somatotopic neural responses during the observation of others’ actions. 14-month-old infants were randomly assigned to watch an adult reach towards and touch an object using either her hand or her foot. The scalp electroencephalogram (EEG) was recorded and event-related changes in the sensorimotor mu rhythm were analyzed. Mu rhythm desynchronization was greater over hand areas of sensorimotor cortex during observation of hand actions and was greater over the foot area for observation of foot actions. This provides the first evidence that infants’ observation of someone else using a particular body part activates the corresponding areas of sensorimotor cortex. We hypothesize that this somatotopic organization in the developing brain supports imitation and cultural learning. The findings connect developmental cognitive neuroscience, adult neuroscience, action representation, and behavioral imitation.     

Moving just like you: motor interference depends on similar motility of agent and observer

Recent findings in neuroscience suggest an overlap between brain regions involved in the execution of movement and perception of another's movement. This so-called "action-perception coupling" is supposed to serve our ability to automatically infer the goals and intentions of others by internal simulation of their actions. A consequence of this coupling is motor interference (MI), the effect of movement observation on the trajectory of one's own movement. Previous studies emphasized that various features of the observed agent determine the degree of MI, but could not clarify how human-like an agent has to be for its movements to elicit MI and, more importantly, what 'human-like' means in the context of MI. Thus, we investigated in several experiments how different aspects of appearance and motility of the observed agent influence motor interference (MI). Participants performed arm movements in horizontal and vertical directions while observing videos of a human, a humanoid robot, or an industrial robot arm with either artificial (industrial) or human-like joint configurations. Our results show that, given a human-like joint configuration, MI was elicited by observing arm movements of both humanoid and industrial robots. However, if the joint configuration of the robot did not resemble that of the human arm, MI could longer be demonstrated. Our findings present evidence for the importance of human-like joint configuration rather than other human-like features for perception-action coupling when observing inanimate agents.     

Schema design and implementation of the grasp-related mirror neuron system

 Mirror neurons within a monkey's premotor area F5 fire not only when the monkey performs a certain class of actions but also when the monkey observes another monkey (or the experimenter) perform a similar action. It has thus been argued that these neurons are crucial for understanding of actions by others. We offer the hand-state hypothesis as a new explanation of the evolution of this capability: the basic functionality of the F5 mirror system is to elaborate the appropriate feedback – what we call the hand state– for opposition-space based control of manual grasping of an object. Given this functionality, the social role of the F5 mirror system in understanding the actions of others may be seen as an exaptation gained by generalizing from one's own hand to an other's hand. In other words, mirror neurons first evolved to augment the “canonical” F5 neurons (active during self-movement based on observation of an object) by providing visual feedback on “hand state,” relating the shape of the hand to the shape of the object. We then introduce the MNS1 (mirror neuron system 1) model of F5 and related brain regions. The existing Fagg–Arbib–Rizzolatti–Sakata model represents circuitry for visually guided grasping of objects, linking the anterior intraparietal area (AIP) with F5 canonical neurons. The MNS1 model extends the AIP visual pathway by also modeling pathways, directed toward F5 mirror neurons, which match arm–hand trajectories to the affordances and location of a potential target object. We present the basic schemas for the MNS1 model, then aggregate them into three “grand schemas”– visual analysis of hand state, reach and grasp, and the core mirror circuit – for each of which we present a useful implementation (a non-neural visual processing system, a multijoint 3-D kinematics simulator, and a learning neural network, respectively). With this implementation we show how the mirror system may learnto recognize actions already in the repertoire of the F5 canonical neurons. We show that the connectivity pattern of mirror neuron circuitry can be established through training, and that the resultant network can exhibit a range of novel, physiologically interesting behaviors during the process of action recognition. We train the system on the basis of final grasp but then observe the whole time course of mirror neuron activity, yielding predictions for neurophysiological experiments under conditions of spatial perturbation, altered kinematics, and ambiguous grasp execution which highlight the importance of the timingof mirror neuron activity. Received: 6 August 2001 / Accepted in revised form: 5 February 2002     

Seeing or Doing? Influence of Visual and Motor Familiarity in Action Observation

The human brain contains specialized circuits for observing and understanding actions. Previous studies have not distinguished whether this "mirror system" uses specialized motor representations or general processes of visual inference and knowledge to understand observed actions. We report the first neuroimaging study to distinguish between these alternatives. Purely motoric influences on perception have been shown behaviorally, but their neural bases are unknown. We used fMRI to reveal the neural bases of motor influences on action observation. We controlled for visual and knowledge effects by studying expert dancers. Some ballet moves are performed by only one gender. However, male and female dancers train together and have equal visual familiarity with all moves. Male and female dancers viewed videos of gender-specific male and female ballet moves. We found greater premotor, parietal, and cerebellar activity when dancers viewed moves from their own motor repertoire, compared to opposite-gender moves that they frequently saw but did not perform. Our results show that mirror circuits have a purely motor response over and above visual representations of action. We understand actions not only by visual recognition, but also motorically. In addition, we confirm that the cerebellum is part of the action observation network.