Action selection and action value in frontal-striatal circuits

The role that frontal-striatal circuits play in normal behavior remains unclear. Two of the leading hypotheses suggest that these circuits are important for action selection or reinforcement learning. To examine these hypotheses, we carried out an experiment in which monkeys had to select actions in two different task conditions. In the first (random) condition, actions were selected on the basis of perceptual inference. In the second (fixed) condition, the animals used reinforcement from previous trials to select actions. Examination of neural activity showed that the representation of the selected action was stronger in lateral prefrontal cortex (lPFC), and occurred earlier in the lPFC than it did in the dorsal striatum (dSTR). In contrast to this, the representation of action values, in both the random and fixed conditions, was stronger in the dSTR. Thus, the dSTR contains an enriched representation of action value, but it followed frontal cortex in action selection.     

A continuous-time neural model for sequential action

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.     

Beyond Motor Scheme: A Supramodal Distributed Representation in the Action-Observation Network

The representation of actions within the action-observation network is thought to rely on a distributed functional organization. Furthermore, recent findings indicate that the action-observation network encodes not merely the observed motor act, but rather a representation that is independent from a specific sensory modality or sensory experience. In the present study, we wished to determine to what extent this distributed and ‘more abstract’ representation of action is truly supramodal, i.e. shares a common coding across sensory modalities. To this aim, a pattern recognition approach was employed to analyze neural responses in sighted and congenitally blind subjects during visual and/or auditory presentation of hand-made actions. Multivoxel pattern analyses-based classifiers discriminated action from non-action stimuli across sensory conditions (visual and auditory) and experimental groups (blind and sighted). Moreover, these classifiers labeled as ‘action’ the pattern of neural responses evoked during actual motor execution. Interestingly, discriminative information for the action/non action classification was located in a bilateral, but left-prevalent, network that strongly overlaps with brain regions known to form the action-observation network and the human mirror system. The ability to identify action features with a multivoxel pattern analyses-based classifier in both sighted and blind individuals and independently from the sensory modality conveying the stimuli clearly supports the hypothesis of a supramodal, distributed functional representation of actions, mainly within the action-observation network.     

A word in the hand: action, gesture and mental representation in humans and non-human primates

The movements we make with our hands both reflect our mental processes and help to shape them. Our actions and gestures can affect our mental representations of actions and objects. In this paper, we explore the relationship between action, gesture and thought in both humans and non-human primates and discuss its role in the evolution of language. Human gesture (specifically representational gesture) may provide a unique link between action and mental representation. It is kinaesthetically close to action and is, at the same time, symbolic. Non-human primates use gesture frequently to communicate, and do so flexibly. However, their gestures mainly resemble incomplete actions and lack the representational elements that characterize much of human gesture. Differences in the mirror neuron system provide a potential explanation for non-human primates' lack of representational gestures; the monkey mirror system does not respond to representational gestures, while the human system does. In humans, gesture grounds mental representation in action, but there is no evidence for this link in other primates. We argue that gesture played an important role in the transition to symbolic thought and language in human evolution, following a cognitive leap that allowed gesture to incorporate representational elements.     

A comparison of variant theories of intact biochemical systems. II. Flux-oriented and metabolic control theories

In the past two decades, several theories, all ultimately based upon the same power-law formalism, have been proposed to relate the behavior of intact biochemical systems to the properties of their underlying determinants. Confusion concerning the relatedness of these alternatives has become acute because the implications of these theories have never been compared. In the preceding paper we characterized a specific system involving enzyme-enzyme interactions for reference in comparing alternative theories. We also analyzed the reference system by using an explicit variant that involves the S-system representation within biochemical systems theory (BST). We now analyze the same reference system according to two other variants within BST. First, we carry out the analysis by using an explicit variant that involves the generalized mass action representation, which includes the flux-oriented theory of Crabtree and Newsholme as a special case. Second, we carry out the analysis by using an implicit variant that involves the generalized mass action representation, which includes the metabolic control theory of Kacser and his colleagues as a special case. The explicit variants are found to provide a more complete characterization of the reference system than the implicit variants. Within each of these variant classes, the S-system representation is shown to be more mathematically tractable and accurate than the generalized mass action representation. The results allow one to make clear distinctions among the variant theories.     

Abnormalities in the awareness and control of action

Much of the functioning of the motor system occurs without awareness. Nevertheless, we are aware of some aspects of the current state of the system and we can prepare and make movements in the imagination. These mental representations of the actual and possible states of the system are based on two sources: sensory signals from skin and muscles, and the stream of motor commands that have been issued to the system. Damage to the neural substrates of the motor system can lead to abnormalities in the awareness of action as well as defects in the control of action. We provide a framework for understanding how these various abnormalities of awareness can arise. Patients with phantom limbs or with anosognosia experience the illusion that they can move their limbs. We suggest that these representations of movement are based on streams of motor commands rather than sensory signals. Patients with utilization behaviour or with delusions of control can no longer properly link their intentions to their actions. In these cases the impairment lies in the representation of intended movements. The location of the neural damage associated with these disorders suggests that representations of the current and predicted state of the motor system are in parietal cortex, while representations of intended actions are found in prefrontal and premotor cortex.     

Dissociating object directed and non-object directed action in the human mirror system; implications for theories of motor simulation

Mirror neurons are single cells found in macaque premotor and parietal cortices that are active during action execution and observation. In non-human primates, mirror neurons have only been found in relation to object-directed movements or communicative gestures, as non-object directed actions of the upper limb are not well characterized in non-human primates. Mirror neurons provide important evidence for motor simulation theories of cognition, sometimes referred to as the direct matching hypothesis, which propose that observed actions are mapped onto associated motor schemata in a direct and automatic manner. This study, for the first time, directly compares mirror responses, defined as the overlap between action execution and observation, during object directed and meaningless non-object directed actions. We present functional MRI data that demonstrate a clear dissociation between object directed and non-object directed actions within the human mirror system. A premotor and parietal network was preferentially active during object directed actions, whether observed or executed. Moreover, we report spatially correlated activity across multiple voxels for observation and execution of an object directed action. In contrast to predictions made by motor simulation theory, no similar activity was observed for non-object directed actions. These data demonstrate that object directed and meaningless non-object directed actions are subserved by different neuronal networks and that the human mirror response is significantly greater for object directed actions. These data have important implications for understanding the human mirror system and for simulation theories of motor cognition. Subsequent theories of motor simulation must account for these differences, possibly by acknowledging the role of experience in modulating the mirror response.     

Action representation in patients with bilateral vestibular impairments

During mental actions subjects feel themselves performing a movement without any corresponding motor output. Although broad information is available regarding the influence of central lesions on action representation, little is known about how peripheral damages affect mental events. In the current study, we investigated whether lack of vestibular information influences action representation. Twelve healthy adults and twelve patients with bilateral vestibular damage actually performed and mentally simulated walking and drawing. The locomotor paths implied one (first walking task) and four (second walking task) changes in the walking direction. In the drawing task, participants drew on a sheet of paper a path that was similar to that of the second walking task. We recorded and compared between the two groups the timing of actual and mental movements. We found significant temporal discrepancies between actual and mental walking movements in the group of patients. Conversely, drawing actual and drawing mental durations were similar. For the control group, an isochrony between mental and actual movements was observed for the three tasks. This result denotes an inconsistency between action representation and action execution following vestibular damage, which is specific to walking movements, and emphasizes the role of the vestibular system upon mental states of actions. This observation may have important clinical implications. During action planning vestibular patients may overestimate the capacity of their motor system (imaging faster, executing slower) with harmful consequences for their health.     

Infants’ Somatotopic Neural Responses to Seeing Human Actions: I’ve Got You under My Skin

Human infants rapidly learn new skills and customs via imitation, but the neural linkages between action perception and production are not well understood. Neuroscience studies in adults suggest that a key component of imitation–identifying the corresponding body part used in the acts of self and other–has an organized neural signature. In adults, perceiving someone using a specific body part (e.g., hand vs. foot) is associated with activation of the corresponding area of the sensory and/or motor strip in the observer’s brain–a phenomenon called neural somatotopy. Here we examine whether preverbal infants also exhibit somatotopic neural responses during the observation of others’ actions. 14-month-old infants were randomly assigned to watch an adult reach towards and touch an object using either her hand or her foot. The scalp electroencephalogram (EEG) was recorded and event-related changes in the sensorimotor mu rhythm were analyzed. Mu rhythm desynchronization was greater over hand areas of sensorimotor cortex during observation of hand actions and was greater over the foot area for observation of foot actions. This provides the first evidence that infants’ observation of someone else using a particular body part activates the corresponding areas of sensorimotor cortex. We hypothesize that this somatotopic organization in the developing brain supports imitation and cultural learning. The findings connect developmental cognitive neuroscience, adult neuroscience, action representation, and behavioral imitation.     

Action understanding and active inference

We have suggested that the mirror-neuron system might be usefully understood as implementing Bayes-optimal perception of actions emitted by oneself or others. To substantiate this claim, we present neuronal simulations that show the same representations can prescribe motor behavior and encode motor intentions during action?Cobservation. These simulations are based on the free-energy formulation of active inference, which is formally related to predictive coding. In this scheme, (generalised) states of the world are represented as trajectories. When these states include motor trajectories they implicitly entail intentions (future motor states). Optimizing the representation of these intentions enables predictive coding in a prospective sense. Crucially, the same generative models used to make predictions can be deployed to predict the actions of self or others by simply changing the bias or precision (i.e. attention) afforded to proprioceptive signals. We illustrate these points using simulations of handwriting to illustrate neuronally plausible generation and recognition of itinerant (wandering) motor trajectories. We then use the same simulations to produce synthetic electrophysiological responses to violations of intentional expectations. Our results affirm that a Bayes-optimal approach provides a principled framework, which accommodates current thinking about the mirror-neuron system. Furthermore, it endorses the general formulation of action as active inference.     

Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons

A mirror system is active both when an animal executes a class of actions (self-actions) and when it sees another execute an action of that class. Much attention has been given to the possible roles of mirror systems in responding to the actions of others but there has been little attention paid to their role in self-actions. In the companion article (Bonaiuto et al. Biol Cybern 96:9–38, 2007) we presented MNS2, an extension of the Mirror Neuron System model of the monkey mirror system trained to recognize the external appearance of its own actions as a basis for recognizing the actions of other animals when they perform similar actions. Here we further extend the study of the mirror system by introducing the novel hypotheses that a mirror system may additionally help in monitoring the success of a self-action and may also be activated by recognition of one’s own apparent actions as well as efference copy from one’s intended actions. The framework for this computational demonstration is a model of action sequencing, called augmented competitive queuing, in which action choice is based on the desirability of executable actions. We show how this “what did I just do?” function of mirror neurons can contribute to the learning of both executability and desirability which in certain cases supports rapid reorganization of motor programs in the face of disruptions.     

A Generic Model of Dyadic Social Relationships

We introduce a model of dyadic social interactions and establish its correspondence with relational models theory (RMT), a theory of human social relationships. RMT posits four elementary models of relationships governing human interactions, singly or in combination: Communal Sharing, Authority Ranking, Equality Matching, and Market Pricing. To these are added the limiting cases of asocial and null interactions, whereby people do not coordinate with reference to any shared principle. Our model is rooted in the observation that each individual in a dyadic interaction can do either the same thing as the other individual, a different thing or nothing at all. To represent these three possibilities, we consider two individuals that can each act in one out of three ways toward the other: perform a social action X or Y, or alternatively do nothing. We demonstrate that the relationships generated by this model aggregate into six exhaustive and disjoint categories. We propose that four of these categories match the four relational models, while the remaining two correspond to the asocial and null interactions defined in RMT. We generalize our results to the presence of N social actions. We infer that the four relational models form an exhaustive set of all possible dyadic relationships based on social coordination. Hence, we contribute to RMT by offering an answer to the question of why there could exist just four relational models. In addition, we discuss how to use our representation to analyze data sets of dyadic social interactions, and how social actions may be valued and matched by the agents.     

Impediments to the Success of Management Actions for Species Recovery

Finding cost-effective management strategies to recover species declining due to multiple threats is challenging, especially when there are limited resources. Recent studies offer insights into how costs and threats can influence the best choice of management actions. However, when implementing management actions in the real-world, a range of impediments to management success often exist that can be driven by social, technological and land-use factors. These impediments may limit the extent to which we can achieve recovery objectives and influence the optimal choice of management actions. Nonetheless, the implications of these impediments are not well understood, especially for recovery planning involving multiple actions. We used decision theory to assess the impact of these types of impediments for allocating resources among recovery actions to mitigate multiple threats. We applied this to a declining koala (Phascolarctos cinereus) population threatened by habitat loss, vehicle collisions, dog attacks and disease. We found that the unwillingness of dog owners to restrain their dogs at night (a social impediment), the effectiveness of wildlife crossings to reduce vehicle collisions (a technological impediment) and the unavailability of areas for restoration (a land-use impediment) significantly reduced the effectiveness of our actions. In the presence of these impediments, achieving successful recovery may be unlikely. Further, these impediments influenced the optimal choice of recovery actions, but the extent to which this was true depended on the target koala population growth rate. Given that species recovery is an important strategy for preserving biodiversity, it is critical that we consider how impediments to the success of recovery actions modify our choice of actions. In some cases, it may also be worth considering whether investing in reducing or removing impediments may be a cost-effective course of action.     

Group Membership Affects Spontaneous Mental Representation: Failure to Represent the Out-Group in a Joint Action Task

Predicting others’ actions is crucial to successful social interaction. Previous research on joint action, based on a reaction-time paradigm called the Joint Simon Task, suggests that successful joint action stems from the simultaneous representation of the self with the other. Performance on this task provides a read-out of the degree of intrusion from a partner that participants experience from acting jointly compared to acting alone, which in turn is a measure of the degree to which participants mentally represent their co-actors during the task. To investigate the role of perceived group membership in this type of joint action and its influence on the representation of others, we first subjected participants to a minimal group paradigm while manipulating differences in social competition. We then asked participants to do the Joint Simon Task in pairs with an in-group or out-group member. Only participants who acted with an “in-group” partner on the joint task showed altered reaction times compared to when acting alone, presumably a change caused by the simultaneous and automatic representation of their in-group partner. In contrast, participants who acted with an out-group partner were unaffected in their reactions when doing the joint task, showing no evidence of representation of their out-group partner. This effect was present in both the high-competition and low-competition conditions, indicating that the differential effects of group membership on representation during joint action were driven by perceived group membership and independent of the effects of social competition. We concluded that participants failed to represent out-group members as socially relevant agents not based on any personality or situational characteristics, but in reaction only to their status as “other”. In this way group membership appears to affect cognition on a very immediate and subconscious level.     

Natural selection and the conditions for existence: representational vs. conditional teleology in biological explanation

Human intentional action, including the design and use of artifacts, involves the prior mental representation of the goal (end) and the means to achieve that goal. This representation is part of the efficient cause of the action, and thus can be used to explain both the action and the achievement of the end. This is intentional teleological explanation. More generally, teleological explanation that depends on the real existence of a representation of the goal (and the means to achieve it) can be called representational teleological explanation. Such explanations in biology can involve both external representations (e.g., ideas in the mind of God) and internal representations (souls, vital powers, entelechies, developmental programs, etc.). However, another type of explanation of intentional action (or any other process) is possible. Given that an action achieving a result occurs, the action can be explained as fulfilling the necessary conditions (means) for that result (end), and, reciprocally, the result explained by the occurrence of those necessary conditions. This is conditional teleological explanation. For organisms, natural selection is often understood metaphorically as the designer, intentionally constructing them for certain ends. Unfortunately, this metaphor is often taken rather too literally, because it has been difficult to conceive of another way to relate natural selection to the process of evolution. I argue that combining a conditional teleological explanation of organisms and of evolution provides such an alternative. This conditional teleology can be grounded in existence or survival. Given that an organism exists, we can explain its existence by the occurrence of the necessary conditions for that existence. This principle of the 'conditions for existence' was introduced by Georges Cuvier in 1800, and provides a valid, conditional teleological method for explaining organismal structure and behavior. From an evolutionary perspective, the conditions for existence are the range of boundary conditions within which the evolutionary process must occur. Moreover, evolutionary change itself can be subjected to conditional teleological explanation, because natural selection theory is primarily a theory about the relation between the conditions for the existence of organisms and the conditions for the existence of traits in populations. I show that failure to distinguish representational from conditional teleological explanation has confused previous attempts to clarify the relation of teleology to biology.     

The intracrine hypothesis: an update

The intracellular actions of peptide hormones, growth factors, as well as of extracellular-signaling enzymes and DNA-binding proteins, either within target cells or within their cells of synthesis has been called intracrine action. Although these intracrine moieties are structurally diverse, they share certain characteristics of synthesis and function. This has given rise to the development of a theory of intracrine action which permits testable predictions to be made regarding the functioning of these peptides/proteins. Here the intracrine hypothesis is briefly described and then recent experimental findings which bear on predictions made earlier on the basis of the theory are discussed. These findings provide new support for the intracrine hypothesis.     

How frontoparietal brain regions mediate imitative and complementary actions: an FMRI study

The 'mirror neuron system' (MNS), located within inferior parietal lobe (IPL) and inferior frontal gyrus (IFG), creates an internal motor representation of the actions we see and has been implicated in imitation. Recently, the MNS has been implicated in non-identical responses: when the actions we must execute do not match those that we observe. However, in such conflicting situations non action-specific cognitive control networks also located in frontoparietal regions may be involved. In the present functional magnetic resonance imaging (fMRI) study participants made both similar and dissimilar actions within two action contexts: imitative and complementary. We aimed to determine whether activity within IPL/IFG depends on (i) responding under an imitative versus complementary context (ii) responding with similar versus dissimilar responses, and (iii) observing hand actions versus symbolic arrow cue stimuli. Activity within rIPL/rIFG regions was largest during observation of hand actions compared with arrow cues. Specifically, rIPL/rIFG were recruited only during the imitative context, when participants responded with similar actions. When responding to symbolic arrow cues, rIPL/rIFG activity increased during dissimilar responses, reflecting increased demands placed on general cognitive control mechanisms. These results suggest a specific role of rIPL/rIFG during imitation of hand actions, and also a general role of frontoparietal areas in mediating dissimilar responses to both hand actions and symbolic stimuli. We discuss our findings in relation to recent work that has examined the role of frontoparietal brain structures in joint-actions and inter-actor cooperation. We conclude that the specific brain regions identified here to show increased activation during action observation conditions are likely to form part of a mechanism specifically involved in matching observed actions directly with internal motor plans. Conversely, observation of arrow cues recruited part of a wider cognitive control network involved in the rapid remapping of stimulus-response associations.