The role of the basal ganglia in habit formation

Many organisms, especially humans, are characterized by their capacity for intentional, goal-directed actions. However, similar behaviours often proceed automatically, as habitual responses to antecedent stimuli. How are goal-directed actions transformed into habitual responses? Recent work combining modern behavioural assays and neurobiological analysis of the basal ganglia has begun to yield insights into the neural basis of habit formation.     

Impairments in Goal-Directed Actions Predict Treatment Response to Cognitive-Behavioral Therapy in Social Anxiety Disorder

Social anxiety disorder is characterized by excessive fear and habitual avoidance of social situations. Decision-making models suggest that patients with anxiety disorders may fail to exhibit goal-directed control over actions. We therefore investigated whether such biases may also be associated with social anxiety and to examine the relationship between such behavior with outcomes from cognitive-behavioral therapy. Patients diagnosed with social anxiety and controls completed an instrumental learning task in which two actions were performed to earn food outcomes. After outcome devaluation, where one outcome was consumed to satiety, participants were re-tested in extinction. Results indicated that, as expected, controls were goal-directed, selectively reducing responding on the action that previously delivered the devalued outcome. Patients with social anxiety, however, exhibited no difference in responding on either action. This loss of a devaluation effect was associated with greater symptom severity and poorer response to therapy. These findings indicate that variations in goal-directed control in social anxiety may represent both a behavioral endophenotype and may be used to predict individuals who will respond to learning-based therapies.     

Actions,Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized

Behavioral evidence suggests that instrumental conditioning is governed by two forms of action control: a goal-directed and a habit learning process. Model-based reinforcement learning (RL) has been argued to underlie the goal-directed process; however, the way in which it interacts with habits and the structure of the habitual process has remained unclear. According to a flat architecture, the habitual process corresponds to model-free RL, and its interaction with the goal-directed process is coordinated by an external arbitration mechanism. Alternatively, the interaction between these systems has recently been argued to be hierarchical, such that the formation of action sequences underlies habit learning and a goal-directed process selects between goal-directed actions and habitual sequences of actions to reach the goal. Here we used a two-stage decision-making task to test predictions from these accounts. The hierarchical account predicts that, because they are tied to each other as an action sequence, selecting a habitual action in the first stage will be followed by a habitual action in the second stage, whereas the flat account predicts that the statuses of the first and second stage actions are independent of each other. We found, based on subjects'' choices and reaction times, that human subjects combined single actions to build action sequences and that the formation of such action sequences was sufficient to explain habitual actions. Furthermore, based on Bayesian model comparison, a family of hierarchical RL models, assuming a hierarchical interaction between habit and goal-directed processes, provided a better fit of the subjects'' behavior than a family of flat models. Although these findings do not rule out all possible model-free accounts of instrumental conditioning, they do show such accounts are not necessary to explain habitual actions and provide a new basis for understanding how goal-directed and habitual action control interact.     

Deep brain stimulation for Parkinson's disease

Deep brain stimulation at high frequency was first used in 1997 to replace thalamotomy in treating the characteristic tremor of Parkinson's disease, and has subsequently been applied to the pallidum and the subthalamic nucleus. The subthalamic nucleus is a key node in the functional control of motor activity in the basal ganglia. Its inhibition suppresses symptoms in animal models of Parkinson's disease, and high frequency chronic stimulation does the same in human patients. Acute and long-term results after deep brain stimulation show a dramatic and stable improvement of a patient's clinical condition, which mimics the effects of levodopa treatment. The mechanism of action may involve a functional disruption of the abnormal neural messages associated with the disease. Long-term changes, neural plasticity and neural protection might be induced in the network. Similar effects of stimulation and lesioning have led to the extension of this technique for other targets and diseases.     

Basal ganglia oscillations and pathophysiology of movement disorders

Low frequency rest tremor is one of the cardinal signs of Parkinson's disease and some of its animal models. Current physiological studies and models of the basal ganglia differ as to which aspects of neuronal activity are crucial to the pathophysiology of Parkinson's disease. There is evidence that neural oscillations and synchronization play a central role in the generation of the disease. However, parkinsonian tremor is not strictly correlated with the synchronous oscillations in the basal ganglia networks. Rather, abnormal basal ganglia output enforces abnormal thalamo-cortical processing leading to akinesia, the main negative symptom of Parkinson's disease. Parkinsonian tremor has probably evolved as a downstream compensatory mechanism.     

The endogenous opioid system in neurological disorders of the basal ganglia

The endogenous opioid peptides have for some time been implicated in the regulation of motor behavior in animals. Recently, however, there is increased evidence to suggest a role for these peptides in the control of human motor functions as well as in the pathophysiology of abnormal movement disorders. Degeneration of opioid peptide-containing neurons in the basal ganglia has been demonstrated in Parkinson's disease and Huntington's chorea, but the clinical significance of these findings is largely unknown. On the other hand, there is evidence that excessive opioid activity may be important in the pathophysiology of some movement disorders such as tardive dyskinesia, progressive supra-nuclear palsy, and a subgroup of Tourette's patients. These findings indicate that diseases of the basal ganglia are possibly associated with alterations in opioid peptide activity, and that these alterations may be useful in designing experimental therapeutic strategies in these conditions.     

Striatal information signaling and integration in globus pallidus: timing matters

Advances in research on globus pallidus (GP) suggest that this 'long thought to be' relay in the 'indirect pathway' plays a unique and critical role in basal ganglia function. The traditional idea of parallel processing within the basal ganglia is also challenged by recent findings. It is now clear that axons of GP neurons form large, perisomatic baskets around target neurons in all major basal ganglia nuclei, thereby exerting a profound influence on the output of the entire basal ganglia. GP neurons are autonomously active both in vivo and in vitro. It is believed that temporal information carried along the corticostriatopallidal pathway is critical for proper motor execution. The importance of appropriately controlled discharge of GP neurons is highlighted by psychomotor disorders such as Parkinson's disease, in which alterations in the pattern and synchrony of discharge in GP neurons are thought to contribute to motor symptoms. Several lines of evidence suggest that the aberrant activity of GP neurons following dopamine depletion is caused by alteration in the synaptic input from both striatum and subthalamic nucleus. In normal subjects, the capability of striatal input in translating cortical input into precisely timed responses in GP neurons is mediated by (1) the expression of postsynaptic GABA(A) receptor composed of subunits with fast kinetic properties; (2) an effective GABA reuptake system in terminating the action of synaptically released GABA, and (3) the existence of dendritic HCN channels that actively abbreviate the time course of the inhibitory postsynaptic potentials and reset rhythmic discharge. Despite the rapid pace in uncovering the elements that shape the activity along the striatopallidosubthalamic pathway, the origin of rhythmic, synchronized bursting of GP neurons seen in parkinsonism has not been fully established experimentally. Further elucidation of the factors that control the information transfer in the striatopallidal synapses is thus critical to our understanding of basal ganglia function and establishing treatment for Parkinson's disease and other basal ganglia disorders.     

Increased dopamine peroxidation in postmortem Parkinsonian brain

We have previously reported the presence, in human midbrain, of an enzymatic activity which catalyzes the formation of dopaminochrome from dopamine (DA) and hydrogen peroxide. Here, we report, for the first time, an increased DA peroxidizing activity in the midbrain and basal ganglia of autoptic Parkinsonian brains. The crude activity was determined spectrophotometrically in extracts of paraffin-embedded slices obtained from autopsied brain. No addition of substrate was necessary since endogenous substrates such as DA and hydrogen peroxide were present in the samples. In Parkinson's patients' midbrain, this activity was substantially increased compared to normal midbrain. Moreover, the DA peroxidizing activity, which was absent in basal ganglia of normal people, was detectable in all our Parkinson's patients. These observations suggest that a peroxidizing pathway of DA may be present in human brain. The increased peroxidizing activity in Parkinson's patients generates the toxic compound dopaminochrome which may play a role in the pathogenesis of this disease.     

Motor skill learning, retention, and control deficits in Parkinson's disease

Parkinson's disease, which affects the basal ganglia, is known to lead to various impairments of motor control. Since the basal ganglia have also been shown to be involved in learning processes, motor learning has frequently been investigated in this group of patients. However, results are still inconsistent, mainly due to skill levels and time scales of testing. To bridge across the time scale problem, the present study examined de novo skill learning over a long series of practice sessions that comprised early and late learning stages as well as retention. 19 non-demented, medicated, mild to moderate patients with Parkinson's disease and 19 healthy age and gender matched participants practiced a novel throwing task over five days in a virtual environment where timing of release was a critical element. Six patients and seven control participants came to an additional long-term retention testing after seven to nine months. Changes in task performance were analyzed by a method that differentiates between three components of motor learning prominent in different stages of learning: Tolerance, Noise and Covariation. In addition, kinematic analysis related the influence of skill levels as affected by the specific motor control deficits in Parkinson patients to the process of learning. As a result, patients showed similar learning in early and late stages compared to the control subjects. Differences occurred in short-term retention tests; patients' performance constantly decreased after breaks arising from poorer release timing. However, patients were able to overcome the initial timing problems within the course of each practice session and could further improve their throwing performance. Thus, results demonstrate the intact ability to learn a novel motor skill in non-demented, medicated patients with Parkinson's disease and indicate confounding effects of motor control deficits on retention performance.     

Localization and physiological roles of metabotropic glutamate receptors in the direct and indirect pathways of the basal ganglia

Summary.  Our current understanding of the circuitry of the basal ganglia, and the pathophysiology of Parkinson's disease has led to major breakthroughs in the treatment of this debilitating movement disorder. Unfortunately, there are significant problems with the currently available pharmacological therapies that focus on dopamine replacement or dopaminergic agonists. Because of this, much effort has been focused on developing novel targets for the treatment of Parkinson's disease. The metabotropic glutamate receptors are a family of G-protein coupled receptors activated by glutamate. These receptors are differentially distributed throughout the basal ganglia in a manner suggesting that they may provide novel targets for the treatment of movement disorders. In this review we summarize anatomical and physiological data from our work and the work of other laboratories describing the distribution and physiological roles of metabotropic glutamate receptors in the basal ganglia with emphasis on possible therapeutic targets. Received July 2, 2001 Accepted August, 6, 2001 Published online June 26, 2002     

GluR1 glutamate receptor subunit is regulated differentially in the primate basal ganglia following nigrostriatal dopamine denervation

Nigrostriatal dopaminergic denervation is associated with complex changes in the functional and neurochemical anatomy of the basal ganglia. The excitatory neurotransmitter glutamate mediates neural signaling at crucial points of this circuitry, and glutamate receptors are differentially distributed in the basal ganglia. Available evidence suggests that the glutamatergic corticostriatal and subthalamofugal pathways become overactive after nigrostriatal dopamine depletion. In this study, we have analyzed the regulation of the GluR1 subunit of the a-amino-3-hydroxy-5-methyl-4-isoxazole propionate glutamate receptor in the basal ganglia of primates following 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine-induced dopamine denervation. The dopamine denervation resulted in distinct alterations in GluR1 distribution: (1) GluR1 protein expression was markedly increased in caudate and putamen, and this was most pronounced in the striosomes; (2) GluR1 protein was altered minimally in subthalamic nucleus; (3) expression of GluR1 was down-regulated in the globus pallidus by 63% and in the substantia nigra by 57%. The down-regulation of GluR1 expression in the output nuclei of the basal ganglia, the internal segment of the globus pallidus and the substantia nigra pars reticulata, may be a compensation for the overactive glutamatergic input from subthalamic nucleus, which arises after striatal dopamine denervation. Our results indicate that the glutamatergic system undergoes regulatory changes in response to altered basal ganglia activity in a primate model of Parkinson's disease. Targeted manipulation of the glutamatergic system may be a viable approach to the symptomatic treatment of Parkinson's disease.     

New developments in understanding the etiology of Parkinson's disease and in its treatment

Important recent advances have been made in understanding the etiology and pathogenesis of Parkinson's disease, as well as in developing novel treatments. Two newly identified genes, α-synuclein and parkin, have been linked to parkinsonism. In addition, disturbances to the normal basal ganglia circuits in Parkinson's patients are being described at both anatomical and physiological levels. These developments provide a strong scientific basis for novel medical and surgical strategies to treat the profound motor disturbances in patients with Parkinson's disease.     

Evidence for Habitual and Goal-Directed Behavior Following Devaluation of Cocaine: A Multifaceted Interpretation of Relapse

Background

Cocaine addiction is characterized as a chronically relapsing disorder. It is believed that cues present during self-administration become learned and increase the probability that relapse will occur when they are confronted during abstinence. However, the way in which relapse-inducing cues are interpreted by the user has remained elusive. Recent theories of addiction posit that relapse-inducing cues cause relapse habitually or automatically, bypassing processing information related to the consequences of relapse. Alternatively, other theories hypothesize that relapse-inducing cues produce an expectation of the drug''s consequences, designated as goal-directed relapse. Discrete discriminative stimuli signaling the availability of cocaine produce robust cue-induced responding after thirty days of abstinence. However, it is not known whether cue-induced responding is a goal-directed action or habit.

Methodology/Principal Findings

We tested whether cue-induced responding is a goal-directed action or habit by explicitly pairing or unpairing cocaine with LiCl-induced sickness (n = 7/group), thereby decreasing or not altering the value of cocaine, respectively. Following thirty days of abstinence, no difference in responding between groups was found when animals were reintroduced to the self-administration environment alone, indicating habitual behavior. However, upon discriminative stimulus presentations, cocaine-sickness paired animals exhibited decreased cue-induced responding relative to unpaired controls, indicating goal-directed behavior. In spite of the difference between groups revealed during abstinent testing, no differences were found between groups when animals were under the influence of cocaine.

Conclusions/Significance

Unexpectedly, both habitual and goal-directed responding occurred during abstinent testing. Furthermore, habitual or goal-directed responding may have been induced by cues that differed in their correlation with the cocaine infusion. Non-discriminative stimulus cues were weak correlates of the infusion, which failed to evoke a representation of the value of cocaine and led to habitual behavior. However, the discriminative stimulus–nearly perfectly correlated with the infusion–likely evoked a representation of the value of the infusion and led to goal-directed behavior. These data indicate that abstinent cue-induced responding is multifaceted, dynamically engendering habitual or goal-directed behavior. Moreover, since goal-directed behavior terminated habitual behavior during testing, therapeutic approaches aimed at reducing the perceived value of cocaine in addicted individuals may reduce the capacity of cues to induce relapse.     

Abnormal oscillatory synchronisation in the motor system leads to impaired movement

Converging data suggest that abnormal synchronised oscillatory activity in the basal ganglia may contribute to bradykinesia in patients with Parkinson's disease. This synchrony preferentially occurs over 10-30 Hz, the so-called beta band. Correlative evidence has been supplemented by experiments in which direct stimulation of the basal ganglia in the beta band slows movement. Yet questions remain regarding the small scale of the latter effects and whether synchrony is an early or even obligatory feature of parkinsonism. Nevertheless, the principle that abnormally synchronised activity in the beta band can disrupt the function finds a precedent in the syndrome of cortical myoclonus. Here, pathologically synchronised discharges of pyramidal neurons are transmitted to the healthy spinal cord. The result is the synchronous discharge of motor units leading to rhythmic jerking.     

Metabotropic glutamate receptors in the basal ganglia motor circuit

In recent years there have been tremendous advances in our understanding of the circuitry of the basal ganglia and our ability to predict the behavioural effects of specific cellular changes in this circuit on voluntary movement. These advances, combined with a new understanding of the rich distribution and diverse physiological roles of metabotropic glutamate receptors in the basal ganglia, indicate that these receptors might have a key role in motor control and raise the exciting possibility that they might provide therapeutic targets for the treatment of Parkinson's disease and related disorders.     

Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis.

Amyotrophic lateral sclerosis (ALS) is a disorder marked by loss of motoneurons. We hypothesized that subjects with ALS would have an altered gait rhythm, with an increase in both the magnitude of the stride-to-stride fluctuations and perturbations in the fluctuation dynamics. To test for this locomotor instability, we quantitatively compared the gait rhythm of subjects with ALS with that of normal controls and with that of subjects with Parkinson's disease (PD) and Huntington's disease (HD), pathologies of the basal ganglia. Subjects walked for 5 min at their usual pace wearing an ankle-worn recorder that enabled determination of the duration of each stride and of stride-to-stride fluctuations. We found that the gait of patients with ALS is less steady and more temporally disorganized compared with that of healthy controls. In addition, advanced ALS, HD, and PD were associated with certain common, as well as apparently distinct, features of altered stride dynamics. Thus stride-to-stride control of gait rhythm is apparently compromised with ALS. Moreover, a matrix of markers based on gait dynamics may be useful in characterizing certain pathologies of motor control and, possibly, in quantitatively monitoring disease progression and evaluating therapeutic interventions.     

Action,time and the basal ganglia

The ability to control the speed of movement is compromised in neurological disorders involving the basal ganglia, a set of subcortical cerebral nuclei that receive prominent dopaminergic projections from the midbrain. For example, bradykinesia, slowness of movement, is a major symptom of Parkinson''s disease, whereas rapid tics are observed in patients with Tourette syndrome. Recent experimental work has also implicated dopamine (DA) and the basal ganglia in action timing. Here, I advance the hypothesis that the basal ganglia control the rate of change in kinaesthetic perceptual variables. In particular, the sensorimotor cortico-basal ganglia network implements a feedback circuit for the control of movement velocity. By modulating activity in this network, DA can change the gain of velocity reference signals. The lack of DA thus reduces the output of the velocity control system which specifies the rate of change in body configurations, slowing the transition from one body configuration to another.     

PET imaging of the basal ganglia

The present chapter reviews PET imaging in basal ganglia disorders; Parkinson's disease is used as a model of these disorders because the neurochemical pathobiology of this disease is well known and great advances in the imaging area have been achieved. Other basal ganglia disorders including Tourette's syndrome, dystonia, Huntington's chorea and Wilson's disease are also dealt with. With PET and SPECT techniques, the whole integrative dopaminergic network of neurons can be studied, which plays an important role in differential diagnostics. Furthermore, pharmacological effects of medication can be visualized and the role of stereotaxic neurosurgery can be evaluated. Finally, functional imaging gives clues about the prognosis and rehabilitation aspects of the basal ganglia disorders.     

Cellular principles underlying normal and pathological activity in the subthalamic nucleus

The motor symptoms of Parkinson's disease are associated with abnormal, correlated, low frequency, rhythmic burst activity in the subthalamic nucleus and connected nuclei. Research into the mechanisms controlling the pattern of subthalamic activity has intensified because therapies that manipulate the pattern of subthalamic activity, such as deep brain stimulation and levodopa administration, improve motor function in Parkinson's disease. Recent findings suggest that dopamine denervation of the striatum and extrastriatal basal ganglia profoundly alters the transmission and integration of glutamatergic cortical and GABAergic pallidal inputs to subthalamic neurons, leading to pathological activity that resonates throughout the basal ganglia and wider motor system.     

Mean-field modeling of the basal ganglia-thalamocortical system. II: Dynamics of parkinsonian oscillations

Neuronal correlates of Parkinson's disease (PD) include a shift to lower frequencies in the electroencephalogram (EEG) and enhanced synchronized oscillations at 3-7 and 7-30 Hz in the basal ganglia, thalamus, and cortex. This study describes the dynamics of a recent physiologically based mean-field model of the basal ganglia-thalamocortical system, and shows how it accounts for many key electrophysiological correlates of PD. Its detailed functional connectivity comprises partially segregated direct and indirect pathways through two populations of striatal neurons, a hyperdirect pathway involving a corticosubthalamic projection, thalamostriatal feedback, and local inhibition in striatum and external pallidum (GPe). In a companion paper, realistic steady-state firing rates were obtained for the healthy state, and after dopamine loss modeled by weaker direct and stronger indirect pathways, reduced intrapallidal inhibition, lower firing thresholds of the GPe and subthalamic nucleus (STN), a stronger projection from striatum to GPe, and weaker cortical interactions. Here it is shown that oscillations around 5 and 20 Hz can arise with a strong indirect pathway, which also causes increased synchronization throughout the basal ganglia. Furthermore, increased theta power with progressive nigrostriatal degeneration is correlated with reduced alpha power and peak frequency, in agreement with empirical results. Unlike the hyperdirect pathway, the indirect pathway sustains oscillations with phase relationships that coincide with those found experimentally. Alterations in the responses of basal ganglia to transient stimuli accord with experimental observations. Reduced cortical gains due to both nigrostriatal and mesocortical dopamine loss lead to slower changes in cortical activity and may be related to bradykinesia. Finally, increased EEG power found in some studies may be partly explained by a lower effective GPe firing threshold, reduced GPe-GPe inhibition, and/or weaker intracortical connections in parkinsonian patients. Strict separation of the direct and indirect pathways is not necessary to obtain these results.