Time-integrated position error accounts for sensorimotor behavior in time-constrained tasks

Several studies have shown that human motor behavior can be successfully described using optimal control theory, which describes behavior by optimizing the trade-off between the subject's effort and performance. This approach predicts that subjects reach the goal exactly at the final time. However, another strategy might be that subjects try to reach the target position well before the final time to avoid the risk of missing the target. To test this, we have investigated whether minimizing the control effort and maximizing the performance is sufficient to describe human motor behavior in time-constrained motor tasks. In addition to the standard model, we postulate a new model which includes an additional cost criterion which penalizes deviations between the position of the effector and the target throughout the trial, forcing arrival on target before the final time. To investigate which model gives the best fit to the data and to see whether that model is generic, we tested both models in two different tasks where subjects used a joystick to steer a ball on a screen to hit a target (first task) or one of two targets (second task) before a final time. Noise of different amplitudes was superimposed on the ball position to investigate the ability of the models to predict motor behavior for different levels of uncertainty. The results show that a cost function representing only a trade-off between effort and accuracy at the end time is insufficient to describe the observed behavior. The new model correctly predicts that subjects steer the ball to the target position well before the final time is reached, which is in agreement with the observed behavior. This result is consistent for all noise amplitudes and for both tasks.     

Risk-sensitivity and the mean-variance trade-off: decision making in sensorimotor control

Numerous psychophysical studies suggest that the sensorimotor system chooses actions that optimize the average cost associated with a movement. Recently, however, violations of this hypothesis have been reported in line with economic theories of decision-making that not only consider the mean payoff, but are also sensitive to risk, that is the variability of the payoff. Here, we examine the hypothesis that risk-sensitivity in sensorimotor control arises as a mean-variance trade-off in movement costs. We designed a motor task in which participants could choose between a sure motor action that resulted in a fixed amount of effort and a risky motor action that resulted in a variable amount of effort that could be either lower or higher than the fixed effort. By changing the mean effort of the risky action while experimentally fixing its variance, we determined indifference points at which participants chose equiprobably between the sure, fixed amount of effort option and the risky, variable effort option. Depending on whether participants accepted a variable effort with a mean that was higher, lower or equal to the fixed effort, they could be classified as risk-seeking, risk-averse or risk-neutral. Most subjects were risk-sensitive in our task consistent with a mean-variance trade-off in effort, thereby, underlining the importance of risk-sensitivity in computational models of sensorimotor control.     

Maximal Voluntary Force Strengthened by the Enhancement of Motor System State through Barely Visible Priming Words with Reward

The topic of unconscious influences on behaviour has long been explored as a means of understanding human performance and the neurobiological correlates of intention, motivation, and action. However, what is relatively unknown is whether subconsciously delivered priming stimuli, with or without rewards, can affect individuals’ maximum level of force produced with their best effort. We demonstrated using transcranial magnetic stimulation that barely visible priming of an action concept, when combined with a reward in the form of a consciously visible positive stimulus, could alter the state of the motor system. In accordance with this neurophysiological alteration, the prime-plus-reward stimuli significantly increased the hand-grip force level of maximum voluntary contraction with little conscious awareness. This is the first objective evidence that the barely conscious presence of a behavioral goal can influence the state of the motor system and arouse latent ability for human force exertion.     

Learning with slight forgetting optimizes sensorimotor transformation in redundant motor systems

Recent theoretical studies have proposed that the redundant motor system in humans achieves well-organized stereotypical movements by minimizing motor effort cost and motor error. However, it is unclear how this optimization process is implemented in the brain, presumably because conventional schemes have assumed a priori that the brain somehow constructs the optimal motor command, and largely ignored the underlying trial-by-trial learning process. In contrast, recent studies focusing on the trial-by-trial modification of motor commands based on error information suggested that forgetting (i.e., memory decay), which is usually considered as an inconvenient factor in motor learning, plays an important role in minimizing the motor effort cost. Here, we examine whether trial-by-trial error-feedback learning with slight forgetting could minimize the motor effort and error in a highly redundant neural network for sensorimotor transformation and whether it could predict the stereotypical activation patterns observed in primary motor cortex (M1) neurons. First, using a simple linear neural network model, we theoretically demonstrated that: 1) this algorithm consistently leads the neural network to converge at a unique optimal state; 2) the biomechanical properties of the musculoskeletal system necessarily determine the distribution of the preferred directions (PD; the direction in which the neuron is maximally active) of M1 neurons; and 3) the bias of the PDs is steadily formed during the minimization of the motor effort. Furthermore, using a non-linear network model with realistic musculoskeletal data, we demonstrated numerically that this algorithm could consistently reproduce the PD distribution observed in various motor tasks, including two-dimensional isometric torque production, two-dimensional reaching, and even three-dimensional reaching tasks. These results may suggest that slight forgetting in the sensorimotor transformation network is responsible for solving the redundancy problem in motor control.     

Do Humans Optimally Exploit Redundancy to Control Step Variability in Walking?

It is widely accepted that humans and animals minimize energetic cost while walking. While such principles predict average behavior, they do not explain the variability observed in walking. For robust performance, walking movements must adapt at each step, not just on average. Here, we propose an analytical framework that reconciles issues of optimality, redundancy, and stochasticity. For human treadmill walking, we defined a goal function to formulate a precise mathematical definition of one possible control strategy: maintain constant speed at each stride. We recorded stride times and stride lengths from healthy subjects walking at five speeds. The specified goal function yielded a decomposition of stride-to-stride variations into new gait variables explicitly related to achieving the hypothesized strategy. Subjects exhibited greatly decreased variability for goal-relevant gait fluctuations directly related to achieving this strategy, but far greater variability for goal-irrelevant fluctuations. More importantly, humans immediately corrected goal-relevant deviations at each successive stride, while allowing goal-irrelevant deviations to persist across multiple strides. To demonstrate that this was not the only strategy people could have used to successfully accomplish the task, we created three surrogate data sets. Each tested a specific alternative hypothesis that subjects used a different strategy that made no reference to the hypothesized goal function. Humans did not adopt any of these viable alternative strategies. Finally, we developed a sequence of stochastic control models of stride-to-stride variability for walking, based on the Minimum Intervention Principle. We demonstrate that healthy humans are not precisely “optimal,” but instead consistently slightly over-correct small deviations in walking speed at each stride. Our results reveal a new governing principle for regulating stride-to-stride fluctuations in human walking that acts independently of, but in parallel with, minimizing energetic cost. Thus, humans exploit task redundancies to achieve robust control while minimizing effort and allowing potentially beneficial motor variability.     

Evidence of a diurnal rhythm in implicit reward learning

Many aspects of hedonic behavior, including self-administration of natural and drug rewards, as well as human positive affect, follow a diurnal cycle that peaks during the species-specific active period. This variation has been linked to circadian modulation of the mesolimbic dopamine system, and is hypothesized to serve an adaptive function by driving an organism to engage with the environment during times where the opportunity for obtaining rewards is high. However, relatively little is known about whether more complex facets of hedonic behavior – in particular, reward learning – follow the same diurnal cycle. The current study aimed to address this gap by examining evidence for diurnal variation in reward learning on a well-validated probabilistic reward learning task (PRT). PRT data from a large normative sample (N = 516) of non-clinical individuals, recruited across eight studies, were examined for the current study. The PRT uses an asymmetrical reinforcement ratio to induce a behavioral response bias, and reward learning was operationalized as the strength of this response bias across blocks of the task. Results revealed significant diurnal variation in reward learning, however in contrast to patterns previously observed in other aspects of hedonic behavior, reward learning was lowest in the middle of the day. Although a diurnal pattern was also observed on a measure of more general task performance (discriminability), this did not account for the variation observed in reward learning. Taken together, these findings point to a distinct diurnal pattern in reward learning that differs from that observed in other aspects of hedonic behavior. The results of this study have important implications for our understanding of clinical disorders characterized by both circadian and reward learning disturbances, and future research is needed to confirm whether this diurnal variation has a truly circadian origin.     

A Model of Reward- and Effort-Based Optimal Decision Making and Motor Control

Costs (e.g. energetic expenditure) and benefits (e.g. food) are central determinants of behavior. In ecology and economics, they are combined to form a utility function which is maximized to guide choices. This principle is widely used in neuroscience as a normative model of decision and action, but current versions of this model fail to consider how decisions are actually converted into actions (i.e. the formation of trajectories). Here, we describe an approach where decision making and motor control are optimal, iterative processes derived from the maximization of the discounted, weighted difference between expected rewards and foreseeable motor efforts. The model accounts for decision making in cost/benefit situations, and detailed characteristics of control and goal tracking in realistic motor tasks. As a normative construction, the model is relevant to address the neural bases and pathological aspects of decision making and motor control.     

Perceived Cost and Intrinsic Motor Variability Modulate the Speed-Accuracy Trade-Off

Fitts’ Law describes the speed-accuracy trade-off of human movements, and it is an elegant strategy that compensates for random and uncontrollable noise in the motor system. The control strategy during targeted movements may also take into account the rewards or costs of any outcomes that may occur. The aim of this study was to test the hypothesis that movement time in Fitts’ Law emerges not only from the accuracy constraints of the task, but also depends on the perceived cost of error for missing the targets. Subjects were asked to touch targets on an iPad^® screen with different costs for missed targets. We manipulated the probability of error by comparing children with dystonia (who are characterized by increased intrinsic motor variability) to typically developing children. The results show a strong effect of the cost of error on the Fitts’ Law relationship characterized by an increase in movement time as cost increased. In addition, we observed a greater sensitivity to increased cost for children with dystonia, and this behavior appears to minimize the average cost. The findings support a proposed mathematical model that explains how movement time in a Fitts-like task is related to perceived risk.     

Perceptions of effort and heaviness during fatigue and during the size-weight illusion

Previous work has shown that force perception and the sense of motor effort are different attributes of sensorimotor function. This study explores the hypothesis that one reason force and effort perceptions are distinct is to inform an individual of impaired motor function when muscular force lags effort. This hypothesis predicts that effort and force perceptions will dissociate when motor function is impaired by fatigue but not during the size-weight illusion. All subjects reported a distinct increase in effort when lifting a standard test weight as fatigue developed. When fatigue was sufficiently marked so that they could barely lift the test weight, they rated their effort as similar to that required to lift a maximal weight in the unfatigued state. The perceived heaviness of the test weight also increased as fatigue developed, but this fatigue-weight illusion was smaller than the increase in effort for all subjects and displayed greater variability. In contrast, both the perceived weight of a small object and the effort required to lift it increased in parallel when small and large objects were lifted sequentially. The size-weight and size-effort illusions appear to be examples of a common phenomenon in which perceptual experience is rescaled to maintain acuity under different working conditions. The fatigue-weight illusion also has the effect of increasing perceptual acuity as the subject's weight lifting range decreases due to fatigue.     

Why Don't You Try Harder? An Investigation of Effort Production in Major Depression

Depression is mainly characterized as an emotional disorder, associated with reduced approach behavior. It remains unclear whether the difficulty in energising behavior relates to abnormal emotional states or to a flattened response to potential rewards, as suggested by several neuroimaging studies. Here, we aimed to demonstrate a specific incentive motivation deficit in major depression, independent of patients' emotional state. We employed a behavioral paradigm designed to measure physical effort in response to both emotional modulation and incentive motivation. Patients did exert more effort following emotionally arousing pictures (whether positive or negative) but not for higher monetary incentives, contrary to healthy controls. These results show that emotional and motivational sources of effort production are dissociable in pathological conditions. In addition, patients' ratings of perceived effort increased for high incentives, whereas controls' ratings were decreased. Thus, depressed patients objectively behave as if they do not want to gain larger rewards, but subjectively feel that they try harder. We suggest that incentive motivation impairment is a core deficit of major depression, which may render everyday tasks abnormally effortful for patients.     

Emergence of a Stable Cortical Map for Neuroprosthetic Control

Cortical control of neuroprosthetic devices is known to require neuronal adaptations. It remains unclear whether a stable cortical representation for prosthetic function can be stored and recalled in a manner that mimics our natural recall of motor skills. Especially in light of the mixed evidence for a stationary neuron-behavior relationship in cortical motor areas, understanding this relationship during long-term neuroprosthetic control can elucidate principles of neural plasticity as well as improve prosthetic function. Here, we paired stable recordings from ensembles of primary motor cortex neurons in macaque monkeys with a constant decoder that transforms neural activity to prosthetic movements. Proficient control was closely linked to the emergence of a surprisingly stable pattern of ensemble activity, indicating that the motor cortex can consolidate a neural representation for prosthetic control in the presence of a constant decoder. The importance of such a cortical map was evident in that small perturbations to either the size of the neural ensemble or to the decoder could reversibly disrupt function. Moreover, once a cortical map became consolidated, a second map could be learned and stored. Thus, long-term use of a neuroprosthetic device is associated with the formation of a cortical map for prosthetic function that is stable across time, readily recalled, resistant to interference, and resembles a putative memory engram.     

Trust and Reciprocity: Are Effort and Money Equivalent?

Trust and reciprocity facilitate cooperation and are relevant to virtually all human interactions. They are typically studied using trust games: one subject gives (entrusts) money to another subject, which may return some of the proceeds (reciprocate). Currently, however, it is unclear whether trust and reciprocity in monetary transactions are similar in other settings, such as physical effort. Trust and reciprocity of physical effort are important as many everyday decisions imply an exchange of physical effort, and such exchange is central to labor relations. Here we studied a trust game based on physical effort and compared the results with those of a computationally equivalent monetary trust game. We found no significant difference between effort and money conditions in both the amount trusted and the quantity reciprocated. Moreover, there is a high positive correlation in subjects' behavior across conditions. This suggests that trust and reciprocity may be character traits: subjects that are trustful/trustworthy in monetary settings behave similarly during exchanges of physical effort. Our results validate the use of trust games to study exchanges in physical effort and to characterize inter-subject differences in trust and reciprocity, and also suggest a new behavioral paradigm to study these differences.     

Optimization of muscle activity for task-level goals predicts complex changes in limb forces across biomechanical contexts

Optimality principles have been proposed as a general framework for understanding motor control in animals and humans largely based on their ability to predict general features movement in idealized motor tasks. However, generalizing these concepts past proof-of-principle to understand the neuromechanical transformation from task-level control to detailed execution-level muscle activity and forces during behaviorally-relevant motor tasks has proved difficult. In an unrestrained balance task in cats, we demonstrate that achieving task-level constraints center of mass forces and moments while minimizing control effort predicts detailed patterns of muscle activity and ground reaction forces in an anatomically-realistic musculoskeletal model. Whereas optimization is typically used to resolve redundancy at a single level of the motor hierarchy, we simultaneously resolved redundancy across both muscles and limbs and directly compared predictions to experimental measures across multiple perturbation directions that elicit different intra- and interlimb coordination patterns. Further, although some candidate task-level variables and cost functions generated indistinguishable predictions in a single biomechanical context, we identified a common optimization framework that could predict up to 48 experimental conditions per animal (n = 3) across both perturbation directions and different biomechanical contexts created by altering animals' postural configuration. Predictions were further improved by imposing experimentally-derived muscle synergy constraints, suggesting additional task variables or costs that may be relevant to the neural control of balance. These results suggested that reduced-dimension neural control mechanisms such as muscle synergies can achieve similar kinetics to the optimal solution, but with increased control effort (≈2×) compared to individual muscle control. Our results are consistent with the idea that hierarchical, task-level neural control mechanisms previously associated with voluntary tasks may also be used in automatic brainstem-mediated pathways for balance.     

The Effect of Incentives and Meta-incentives on the Evolution of Cooperation

Although positive incentives for cooperators and/or negative incentives for free-riders in social dilemmas play an important role in maintaining cooperation, there is still the outstanding issue of who should pay the cost of incentives. The second-order free-rider problem, in which players who do not provide the incentives dominate in a game, is a well-known academic challenge. In order to meet this challenge, we devise and analyze a meta-incentive game that integrates positive incentives (rewards) and negative incentives (punishments) with second-order incentives, which are incentives for other players’ incentives. The critical assumption of our model is that players who tend to provide incentives to other players for their cooperative or non-cooperative behavior also tend to provide incentives to their incentive behaviors. In this paper, we solve the replicator dynamics for a simple version of the game and analytically categorize the game types into four groups. We find that the second-order free-rider problem is completely resolved without any third-order or higher (meta) incentive under the assumption. To do so, a second-order costly incentive, which is given individually (peer-to-peer) after playing donation games, is needed. The paper concludes that (1) second-order incentives for first-order reward are necessary for cooperative regimes, (2) a system without first-order rewards cannot maintain a cooperative regime, (3) a system with first-order rewards and no incentives for rewards is the worst because it never reaches cooperation, and (4) a system with rewards for incentives is more likely to be a cooperative regime than a system with punishments for incentives when the cost-effect ratio of incentives is sufficiently large. This solution is general and strong in the sense that the game does not need any centralized institution or proactive system for incentives.     

The Sense of Effort and two Models of Single-Joint Motor Control

Two sets of experiments were carried out. In the first set, human subjects were asked to make the same effort with the elbow flexors at different joint angles under isometric conditions. In some experiments, the subjects were standing with the arm in a vertical (parasagittal) plane; in others, they were seated with the arm in a horizontal (transverse) plane. When muscular torque at a given effort level (ordinate) was plotted as a function of elbow joint angle (abscissa), the resulting isoeffort torque-angle profiles tended to be flat or negatively sloping over a range from 45° to 135°, and they were often nonmonotonic. Increases in effort up to near-maximal levels caused the isoeffort torque-angle profiles to shift upward with little alteration in shape. In the second set of experiments, seated subjects with the arm horizontal resisted baseline torques produced by a motor that acted to extend the elbow joint. Unexpected increases and decreases in torque were superimposed on the baseline torque. The subjects either were instructed to intervene and return the elbow to the initial (90°) position, or were told, “Do not intervene voluntarily; let the motor move your arm.” Effort was reported both under baseline conditions and after the changes in torque. It was found that changes in effort were a function of the changes in torque opposed by the elbow flexors, and were similar whether the subject had repositioned the arm or allowed it to be moved by the motor. In the latter case, the arm came to rest after displacements that were a function of the size and direction of the torque change. For individual subjects, the largest angular displacements ranged from ° 10° to °20° for changes in torque of ° 10 N.m. There was no evidence for any angular dependence of the effort judgments at a given torque over this angular range. Depending on whether effort is primarily an efferent perception proportional to voluntary motor activity or also has a significant afferent (involuntary) component, different models of motor control are supported by these data.     

Reasoning, learning, and creativity: frontal lobe function and human decision-making

The frontal lobes subserve decision-making and executive control--that is, the selection and coordination of goal-directed behaviors. Current models of frontal executive function, however, do not explain human decision-making in everyday environments featuring uncertain, changing, and especially open-ended situations. Here, we propose a computational model of human executive function that clarifies this issue. Using behavioral experiments, we show that unlike others, the proposed model predicts human decisions and their variations across individuals in naturalistic situations. The model reveals that for driving action, the human frontal function monitors up to three/four concurrent behavioral strategies and infers online their ability to predict action outcomes: whenever one appears more reliable than unreliable, this strategy is chosen to guide the selection and learning of actions that maximize rewards. Otherwise, a new behavioral strategy is tentatively formed, partly from those stored in long-term memory, then probed, and if competitive confirmed to subsequently drive action. Thus, the human executive function has a monitoring capacity limited to three or four behavioral strategies. This limitation is compensated by the binary structure of executive control that in ambiguous and unknown situations promotes the exploration and creation of new behavioral strategies. The results support a model of human frontal function that integrates reasoning, learning, and creative abilities in the service of decision-making and adaptive behavior.     

Risk-Sensitive Optimal Feedback Control Accounts for Sensorimotor Behavior under Uncertainty

Many aspects of human motor behavior can be understood using optimality principles such as optimal feedback control. However, these proposed optimal control models are risk-neutral; that is, they are indifferent to the variability of the movement cost. Here, we propose the use of a risk-sensitive optimal controller that incorporates movement cost variance either as an added cost (risk-averse controller) or as an added value (risk-seeking controller) to model human motor behavior in the face of uncertainty. We use a sensorimotor task to test the hypothesis that subjects are risk-sensitive. Subjects controlled a virtual ball undergoing Brownian motion towards a target. Subjects were required to minimize an explicit cost, in points, that was a combination of the final positional error of the ball and the integrated control cost. By testing subjects on different levels of Brownian motion noise and relative weighting of the position and control cost, we could distinguish between risk-sensitive and risk-neutral control. We show that subjects change their movement strategy pessimistically in the face of increased uncertainty in accord with the predictions of a risk-averse optimal controller. Our results suggest that risk-sensitivity is a fundamental attribute that needs to be incorporated into optimal feedback control models.     

Religious people discount the future less

The propensity for religious belief and behavior is a universal feature of human societies, but religious practice often imposes substantial costs upon its practitioners. This suggests that during human cultural evolution, the costs associated with religiosity might have been traded off for psychological or social benefits that redounded to fitness on average. One possible benefit of religious belief and behavior, which virtually every world religion extols, is delay of gratification—that is, the ability to forego small rewards available immediately in the interest of obtaining larger rewards that are available only after a time delay. In this study, we found that religious commitment was associated with a tendency to forgo immediate rewards in order to gain larger, future rewards. We also found that this relationship was partially mediated by future time orientation, which is a subjective sense that the future is very close in time and is approaching rapidly. Although the effect sizes of these associations were relatively small in magnitude, they were obtained even when controlling for sex and the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism).     

Trial-to-Trial Reoptimization of Motor Behavior Due to Changes in Task Demands Is Limited

Each task requires a specific motor behavior that is tuned to task demands. For instance, writing requires a lot of accuracy while clapping does not. It is known that the brain adjusts the motor behavior to different task demands as predicted by optimal control theory. In this study, the mechanism of this reoptimization process is investigated by varying the accuracy demands of a reaching task. In this task, the width of the reaching target (0.5 or 8 cm) was varied either on a trial-to-trial basis (random schedule) or in blocks (blocked schedule). On some trials, the hand of the subjects was clamped to a rectilinear trajectory that ended 2 cm on the left or right of the target center. The rejection of this perturbation largely varied with target width in the blocked schedule but not in the random schedule. That is, subjects exhibited different motor behavior in the different schedules despite identical accuracy demands. Therefore, while reoptimization has been considered immediate and automatic, the differences in motor behavior observed across schedules suggest that the reoptimization of the motor behavior is neither happening on a trial-by-trial basis nor obligatory. The absence of trial-to-trial mechanisms, the inability of the brain to adapt to two conflicting task demands and the existence of a switching cost are discussed as possible sources of the non-optimality of motor behavior during the random schedule.     

A new research strategy for integrating studies of paleoclimate,paleoenvironment, and paleoanthropology

Paleoanthropologists (scientists studying human origins) universally recognize the evolutionary significance of ancient climates and environments for understanding human origins. 1 - 6 Even those scientists working in recent phases of human evolution, when modern humans evolved, agree that hunter‐gatherer adaptations are tied to the way that climate and environment shape the food and technological resource base. 7 - 10 The result is a long tradition of paleoanthropologists engaging with climate and environmental scientists in an effort to understand if and how hominin bio‐behavioral evolution responded to climate and environmental change. Despite this unusual consonance, the anticipated rewards of this synergy are unrealized and, in our opinion, will not reach potential until there are some fundamental changes in the way the research model is constructed. Discovering the relation between climate and environmental change to human origins must be grounded in a theoretical framework and a causal understanding of the connection between climate, environment, resource patterning, behavior, and morphology, then move beyond the strict correlative research that continues to dominate the field.