Applauding with Closed Hands: Neural Signature of Action-Sentence Compatibility Effects

Background

Behavioral studies have provided evidence for an action–sentence compatibility effect (ACE) that suggests a coupling of motor mechanisms and action-sentence comprehension. When both processes are concurrent, the action sentence primes the actual movement, and simultaneously, the action affects comprehension. The aim of the present study was to investigate brain markers of bidirectional impact of language comprehension and motor processes.

Methodology/Principal Findings

Participants listened to sentences describing an action that involved an open hand, a closed hand, or no manual action. Each participant was asked to press a button to indicate his/her understanding of the sentence. Each participant was assigned a hand-shape, either closed or open, which had to be used to activate the button. There were two groups (depending on the assigned hand-shape) and three categories (compatible, incompatible and neutral) defined according to the compatibility between the response and the sentence. ACEs were found in both groups. Brain markers of semantic processing exhibited an N400-like component around the Cz electrode position. This component distinguishes between compatible and incompatible, with a greater negative deflection for incompatible. Motor response elicited a motor potential (MP) and a re-afferent potential (RAP), which are both enhanced in the compatible condition.

Conclusions/Significance

The present findings provide the first ACE cortical measurements of semantic processing and the motor response. N400-like effects suggest that incompatibility with motor processes interferes in sentence comprehension in a semantic fashion. Modulation of motor potentials (MP and RAP) revealed a multimodal semantic facilitation of the motor response. Both results provide neural evidence of an action-sentence bidirectional relationship. Our results suggest that ACE is not an epiphenomenal post-sentence comprehension process. In contrast, motor-language integration occurring during the verb onset supports a genuine and ongoing brain motor-language interaction.     

Processing ordinality and quantity: the case of developmental dyscalculia

In contrast to quantity processing, up to date, the nature of ordinality has received little attention from researchers despite the fact that both quantity and ordinality are embodied in numerical information. Here we ask if there are two separate core systems that lie at the foundations of numerical cognition: (1) the traditionally and well accepted numerical magnitude system but also (2) core system for representing ordinal information. We report two novel experiments of ordinal processing that explored the relation between ordinal and numerical information processing in typically developing adults and adults with developmental dyscalculia (DD). Participants made "ordered" or "non-ordered" judgments about 3 groups of dots (non-symbolic numerical stimuli; in Experiment 1) and 3 numbers (symbolic task: Experiment 2). In contrast to previous findings and arguments about quantity deficit in DD participants, when quantity and ordinality are dissociated (as in the current tasks), DD participants exhibited a normal ratio effect in the non-symbolic ordinal task. They did not show, however, the ordinality effect. Ordinality effect in DD appeared only when area and density were randomized, but only in the descending direction. In the symbolic task, the ordinality effect was modulated by ratio and direction in both groups. These findings suggest that there might be two separate cognitive representations of ordinal and quantity information and that linguistic knowledge may facilitate estimation of ordinal information.     

View from the boundary

Re-implementing biological mechanisms on robots not only has technological application but can provide a unique perspective on the nature of sensory processing in animals. To make a robot work, we need to understand the function as part of an embodied, behaving system. I argue that this perspective suggests that the terms "representation" and "information processing" can be misleading when we seek to understand how neurobiological mechanisms carry out perceptual processes. This argument is presented here with reference to a robot model of cricket behavior, which has demonstrated competence comparable to that of the insect, but utilizes surprisingly simple central processing. Instead it depends on sensory interfaces that are well matched to the task, and on the link between environment, action, and perception.     

The relation between self-reported empathy and motor identification with imagined agents

Background

In a previous study, we found that when required to imagine another person performing an action, participants reported a higher correspondence between their own handedness and the hand used by the imagined person when the agent was seen from the back compared to when the agent was seen from the front. This result was explained as evidence of a greater involvement of motor areas in the back-view perspective, possibly indicating a greater proneness to put oneself in the agent''s shoes in such a condition. In turn, the proneness to put oneself in another''s shoes could also be considered as a cue of greater identification with the other, that is a form of empathy. If this is the case, the proportion of lateral matches vs mismatches should be different for subjects with high and low self-reported empathy. In the present study, we aimed at testing this hypothesis.

Methodology/Principal Findings

Participants were required to imagine a person performing a single manual action in a back view and to indicate the hand used by the imagined person during movement execution. Consistent with our hypothesis, the proportion of matching between the handedness of participants and the handedness of agents imagined was higher for participants scoring high in a self-report measure of empathy. Importantly, this relationship was specific for females.

Conclusions/Significance

At least for females, our data seem to corroborate the idea of a link between self-reported empathy and motor identification with imagined agents. This sex-specific result is consistent with neuroimaging studies indicating a stronger involvement of action representations during emotional and empathic processing in females than in males. In sum, our findings underline the possibility of employing behavioral research as a test-bed for theories deriving from functional studies suggesting a link between empathic processing and the activation of motor-related areas.     

Language comprehension in the balance: the robustness of the action-compatibility effect (ACE)

How does language comprehension interact with motor activity? We investigated the conditions under which comprehending an action sentence affects people''s balance. We performed two experiments to assess whether sentences describing forward or backward movement modulate the lateral movements made by subjects who made sensibility judgments about the sentences. In one experiment subjects were standing on a balance board and in the other they were seated on a balance board that was mounted on a chair. This allowed us to investigate whether the action compatibility effect (ACE) is robust and persists in the face of salient incompatibilities between sentence content and subject movement. Growth-curve analysis of the movement trajectories produced by the subjects in response to the sentences suggests that the ACE is indeed robust. Sentence content influenced movement trajectory despite salient inconsistencies between implied and actual movement. These results are interpreted in the context of the current discussion of embodied, or grounded, language comprehension and meaning representation.     

Body Space in Social Interactions: A Comparison of Reaching and Comfort Distance in Immersive Virtual Reality

Background

Do peripersonal space for acting on objects and interpersonal space for interacting with con-specifics share common mechanisms and reflect the social valence of stimuli? To answer this question, we investigated whether these spaces refer to a similar or different physical distance.

Methodology

Participants provided reachability-distance (for potential action) and comfort-distance (for social processing) judgments towards human and non-human virtual stimuli while standing still (passive) or walking toward stimuli (active).

Principal Findings

Comfort-distance was larger than other conditions when participants were passive, but reachability and comfort distances were similar when participants were active. Both spaces were modulated by the social valence of stimuli (reduction with virtual females vs males, expansion with cylinder vs robot) and the gender of participants.

Conclusions

These findings reveal that peripersonal reaching and interpersonal comfort spaces share a common motor nature and are sensitive, at different degrees, to social modulation. Therefore, social processing seems embodied and grounded in the body acting in space.     

First is best

We experience the world serially rather than simultaneously. A century of research on human and nonhuman animals has suggested that the first experience in a series of two or more is cognitively privileged. We report three experiments designed to test the effect of first position on implicit preference and choice using targets that range from individual humans and social groups to consumer goods. Experiment 1 demonstrated an implicit preference to buy goods from the first salesperson encountered and to join teams encountered first, even when the difference in encounter is mere seconds. In Experiment 2 the first of two consumer items presented in quick succession was more likely to be chosen. In Experiment 3 an alternative hypothesis that first position merely accentuates the valence of options was ruled out by demonstrating that first position enhances preference for the first even when it is evaluatively negative in meaning (a criminal). Together, these experiments demonstrate a "first is best" effect and we offer possible interpretations based on evolutionary mechanisms of this "bound" on rational behavior and suggest that automaticity of judgment may be a helpful principle in clarifying previous inconsistencies in the empirical record on the effects of order on preference and choice.     

Dynamic visuomotor transformation involved with remote flying of a plane utilizes the 'Mirror Neuron' system

Brain regions involved with processing dynamic visuomotor representational transformation are investigated using fMRI. The perceptual-motor task involved flying (or observing) a plane through a simulated Red Bull Air Race course in first person and third person chase perspective. The third person perspective is akin to remote operation of a vehicle. The ability for humans to remotely operate vehicles likely has its roots in neural processes related to imitation in which visuomotor transformation is necessary to interpret the action goals in an egocentric manner suitable for execution. In this experiment for 3(rd) person perspective the visuomotor transformation is dynamically changing in accordance to the orientation of the plane. It was predicted that 3(rd) person remote flying, over 1(st), would utilize brain regions composing the 'Mirror Neuron' system that is thought to be intimately involved with imitation for both execution and observation tasks. Consistent with this prediction differential brain activity was present for 3(rd) person over 1(st) person perspectives for both execution and observation tasks in left ventral premotor cortex, right dorsal premotor cortex, and inferior parietal lobule bilaterally (Mirror Neuron System) (Behaviorally: 1(st)>3(rd)). These regions additionally showed greater activity for flying (execution) over watching (observation) conditions. Even though visual and motor aspects of the tasks were controlled for, differential activity was also found in brain regions involved with tool use, motion perception, and body perspective including left cerebellum, temporo-occipital regions, lateral occipital cortex, medial temporal region, and extrastriate body area. This experiment successfully demonstrates that a complex perceptual motor real-world task can be utilized to investigate visuomotor processing. This approach (Aviation Cerebral Experimental Sciences ACES) focusing on direct application to lab and field is in contrast to standard methodology in which tasks and conditions are reduced to their simplest forms that are remote from daily life experience.     

Do not resonate with actions: sentence polarity modulates cortico-spinal excitability during action-related sentence reading

Background

Theories of embodied language suggest that the motor system is differentially called into action when processing motor-related versus abstract content words or sentences. It has been recently shown that processing negative polarity action-related sentences modulates neural activity of premotor and motor cortices.

Methods and Findings

We sought to determine whether reading negative polarity sentences brought about differential modulation of cortico-spinal motor excitability depending on processing hand-action related or abstract sentences. Facilitatory paired-pulses Transcranial Magnetic Stimulation (pp-TMS) was applied to the primary motor representation of the right-hand and the recorded amplitude of induced motor-evoked potentials (MEP) was used to index M1 activity during passive reading of either hand-action related or abstract content sentences presented in both negative and affirmative polarity. Results showed that the cortico-spinal excitability was affected by sentence polarity only in the hand-action related condition. Indeed, in keeping with previous TMS studies, reading positive polarity, hand action-related sentences suppressed cortico-spinal reactivity. This effect was absent when reading hand action-related negative polarity sentences. Moreover, no modulation of cortico-spinal reactivity was associated with either negative or positive polarity abstract sentences.

Conclusions

Our results indicate that grammatical cues prompting motor negation reduce the cortico-spinal suppression associated with affirmative action sentences reading and thus suggest that motor simulative processes underlying the embodiment may involve even syntactic features of language.     

Effects of TMS on Different Stages of Motor and Non-Motor Verb Processing in the Primary Motor Cortex

The embodied cognition hypothesis suggests that motor and premotor areas are automatically and necessarily involved in understanding action language, as word conceptual representations are embodied. This transcranial magnetic stimulation (TMS) study explores the role of the left primary motor cortex in action-verb processing. TMS-induced motor-evoked potentials from right-hand muscles were recorded as a measure of M1 activity, while participants were asked either to judge explicitly whether a verb was action-related (semantic task) or to decide on the number of syllables in a verb (syllabic task). TMS was applied in three different experiments at 170, 350 and 500 ms post-stimulus during both tasks to identify when the enhancement of M1 activity occurred during word processing. The delays between stimulus onset and magnetic stimulation were consistent with electrophysiological studies, suggesting that word recognition can be differentiated into early (within 200 ms) and late (within 400 ms) lexical-semantic stages, and post-conceptual stages. Reaction times and accuracy were recorded to measure the extent to which the participants'' linguistic performance was affected by the interference of TMS with M1 activity. No enhancement of M1 activity specific for action verbs was found at 170 and 350 ms post-stimulus, when lexical-semantic processes are presumed to occur (Experiments 1–2). When TMS was applied at 500 ms post-stimulus (Experiment 3), processing action verbs, compared with non-action verbs, increased the M1-activity in the semantic task and decreased it in the syllabic task. This effect was specific for hand-action verbs and was not observed for action-verbs related to other body parts. Neither accuracy nor RTs were affected by TMS. These findings suggest that the lexical-semantic processing of action verbs does not automatically activate the M1. This area seems to be rather involved in post-conceptual processing that follows the retrieval of motor representations, its activity being modulated (facilitated or inhibited), in a top-down manner, by the specific demand of the task.     

The Impact of Social Context and Language Comprehension on Behaviour: A Kinematic Investigation

We investigated whether and how comprehending sentences that describe a social context influences our motor behaviour. Our stimuli were sentences that referred to objects having different connotations (e.g., attractive/ugly vs. smooth/prickly) and that could be directed towards the self or towards “another person” target (e.g., “The object is ugly/smooth. Bring it to you/Give it to another person”). Participants judged whether each sentence was sensible or non-sensible by moving the mouse towards or away from their body. Mouse movements were analysed according to behavioral and kinematics parameters. In order to enhance the social meaning of the linguistic stimuli, participants performed the task either individually (Individual condition) or in a social setting, in co-presence with the experimenter. The experimenter could either act as a mere observer (Social condition) or as a confederate, interacting with participants in an off-line modality at the end of task execution (Joint condition). Results indicated that the different roles taken by the experimenter affected motor behaviour and are discussed within an embodied approach to language processing and joint actions.     

Motor simulation and the bodily self

Previous studies demonstrated the human ability to implicitly recognize their own body. When submitted to a visual matching task, participants showed the so-called self-advantage, that is, a better performance with self rather than others' body or body parts. Here, we investigated whether the body self-advantage relies upon a motor representation of one's body. Participants were submitted to a laterality judgment of self and others' hands (Experiment 1 and 3), which involves a sensory-motor mental simulation. Moreover, to investigate whether the self-advantage emerges also when an explicit self processing is required, the same participants were submitted to an explicit self-body recognition task (Experiment 2). Participants showed the self-advantage when performing the laterality judgment, but not when self-recognition was explicitly required. Thus, implicit and explicit recognition of the bodily self dissociate and only an implicit recognition of the bodily self, mapped in motor terms, allows the self-advantage to emerge.     

Taking Perspective: Personal Pronouns Affect Experiential Aspects of Literary Reading

Personal pronouns have been shown to influence cognitive perspective taking during comprehension. Studies using single sentences found that 3^rd person pronouns facilitate the construction of a mental model from an observer’s perspective, whereas 2^nd person pronouns support an actor’s perspective. The direction of the effect for 1^st person pronouns seems to depend on the situational context. In the present study, we investigated how personal pronouns influence discourse comprehension when people read fiction stories and if this has consequences for affective components like emotion during reading or appreciation of the story. We wanted to find out if personal pronouns affect immersion and arousal, as well as appreciation of fiction. In a natural reading paradigm, we measured electrodermal activity and story immersion, while participants read literary stories with 1^st and 3^rd person pronouns referring to the protagonist. In addition, participants rated and ranked the stories for appreciation. Our results show that stories with 1^st person pronouns lead to higher immersion. Two factors—transportation into the story world and mental imagery during reading—in particular showed higher scores for 1^st person as compared to 3^rd person pronoun stories. In contrast, arousal as measured by electrodermal activity seemed tentatively higher for 3^rd person pronoun stories. The two measures of appreciation were not affected by the pronoun manipulation. Our findings underscore the importance of perspective for language processing, and additionally show which aspects of the narrative experience are influenced by a change in perspective.     

The Embodiment of Success and Failure as Forward versus Backward Movements

People often speak of success (e.g., “advance”) and failure (e.g., “setback”) as if they were forward versus backward movements through space. Two experiments sought to examine whether grounded associations of this type influence motor behavior. In Experiment 1, participants categorized success versus failure words by moving a joystick forward or backward. Failure categorizations were faster when moving backward, whereas success categorizations were faster when moving forward. Experiment 2 removed the requirement to categorize stimuli and used a word rehearsal task instead. Even without Experiment 1’s response procedures, a similar cross-over interaction was obtained (e.g., failure memorizations sped backward movements relative to forward ones). The findings are novel yet consistent with theories of embodied cognition and self-regulation.     

Bodily selves in relation: embodied simulation as second-person perspective on intersubjectivity

This article addresses basic aspects of social cognition focusing on the pivotal role played by the lived body in the constitution of our experience of others. It is suggested that before studying intersubjectivity we should better qualify the notion of the self. A minimal notion of the self, the bodily self, defined in terms of its motor potentialities, is proposed. The discovery of mirror mechanisms for action, emotions and sensations led to the proposal of an embodied approach to intersubjectivity—embodied simulation (ES) theory. ES and the related notion of neural reuse provide a new empirically based perspective on intersubjectivity, viewed first and foremost as intercorporeality. ES challenges the notion that folk psychology is the sole account of interpersonal understanding. ES is discussed within a second-person perspective on mindreading.     

Background Speech Effects on Sentence Processing during Reading: An Eye Movement Study

Effects of background speech on reading were examined by playing aloud different types of background speech, while participants read long, syntactically complex and less complex sentences embedded in text. Readers’ eye movement patterns were used to study online sentence comprehension. Effects of background speech were primarily seen in rereading time. In Experiment 1, foreign-language background speech did not disrupt sentence processing. Experiment 2 demonstrated robust disruption in reading as a result of semantically and syntactically anomalous scrambled background speech preserving normal sentence-like intonation. Scrambled speech that was constructed from the text to-be read did not disrupt reading more than scrambled speech constructed from a different, semantically unrelated text. Experiment 3 showed that scrambled speech exacerbated the syntactic complexity effect more than coherent background speech, which also interfered with reading. Experiment 4 demonstrated that both semantically and syntactically anomalous speech produced no more disruption in reading than semantically anomalous but syntactically correct background speech. The pattern of results is best explained by a semantic account that stresses the importance of similarity in semantic processing, but not similarity in semantic content, between the reading task and background speech.     

Interference between sentence processing and probabilistic implicit sequence learning

Background

During sentence processing we decode the sequential combination of words, phrases or sentences according to previously learned rules. The computational mechanisms and neural correlates of these rules are still much debated. Other key issue is whether sentence processing solely relies on language-specific mechanisms or is it also governed by domain-general principles.

Methodology/Principal Findings

In the present study, we investigated the relationship between sentence processing and implicit sequence learning in a dual-task paradigm in which the primary task was a non-linguistic task (Alternating Serial Reaction Time Task for measuring probabilistic implicit sequence learning), while the secondary task were a sentence comprehension task relying on syntactic processing. We used two control conditions: a non-linguistic one (math condition) and a linguistic task (word processing task). Here we show that the sentence processing interfered with the probabilistic implicit sequence learning task, while the other two tasks did not produce a similar effect.

Conclusions/Significance

Our findings suggest that operations during sentence processing utilize resources underlying non-domain-specific probabilistic procedural learning. Furthermore, it provides a bridge between two competitive frameworks of language processing. It appears that procedural and statistical models of language are not mutually exclusive, particularly for sentence processing. These results show that the implicit procedural system is engaged in sentence processing, but on a mechanism level, language might still be based on statistical computations.     

Mirroring intentional forgetting in a shared-goal learning situation

Background

Intentional forgetting refers to the surprising phenomenon that we can forget previously successfully encoded memories if we are instructed to do so. Here, we show that participants cannot only intentionally forget episodic memories but they can also mirror the “forgetting performance” of an observed model.

Methodology/Principal Findings

In four experiments a participant observed a model who took part in a memory experiment. In Experiment 1 and 2 observers saw a movie about the experiment, whereas in Experiment 3 and 4 the observers and the models took part together in a real laboratory experiment. The observed memory experiment was a directed forgetting experiment where the models learned two lists of items and were instructed either to forget or to remember the first list. In Experiment 1 and 3 observers were instructed to simply observe the experiment (“simple observation” instruction). In Experiment 2 and 4, observers received instructions aimed to induce the same learning goal for the observers and the models (“observation with goal-sharing” instruction). A directed forgetting effect (the reliably lower recall of to-be-forgotten items) emerged only when models received the “observation with goal-sharing” instruction (P<.001 in Experiment 2, and P<.05 in Experiment 4), and it was absent when observers received the “simple observation” instruction (P>.1 in Experiment 1 and 3).

Conclusion

If people observe another person with the same intention to learn, and see that this person is instructed to forget previously studied information, then they will produce the same intentional forgetting effect as the person they observed. This seems to be a an important aspect of human learning: if we can understand the goal of an observed person and this is in line with our behavioural goals then our learning performance will mirror the learning performance of the model.     

Modulation of Arm Reaching Movements during Processing of Arm/Hand-Related Action Verbs with and without Emotional Connotation

The theory of embodied language states that language comprehension relies on an internal reenactment of the sensorimotor experience associated with the processed word or sentence. Most evidence in support of this hypothesis had been collected using linguistic material without any emotional connotation. For instance, it had been shown that processing of arm-related verbs, but not of those leg-related verbs, affects the planning and execution of reaching movements; however, at present it is unknown whether this effect is further modulated by verbs evoking an emotional experience. Showing such a modulation might shed light on a very debated issue, i.e. the way in which the emotional meaning of a word is processed. To this end, we assessed whether processing arm/hand-related verbs describing actions with negative connotations (e.g. to stab) affects reaching movements differently from arm/hand-related verbs describing actions with neutral connotation (e.g. to comb). We exploited a go/no-go paradigm in which healthy participants were required to perform arm-reaching movements toward a target when verbs expressing emotional hand actions, neutral hand actions or foot actions were shown, and to refrain from moving when no-effector-related verbs were presented. Reaction times and percentages of errors increased when the verb involved the same effector as used to give the response. However, we also found that the size of this interference decreased when the arm/hand-related verbs had a negative emotional connotation. Crucially, we show that such modulation only occurred when the verb semantics had to be retrieved. These results suggest that the comprehension of negatively valenced verbs might require the simultaneous reenactment of the neural circuitry associated with the processing of the emotion evoked by their meaning and of the neural circuitry associated with their motor features.