Autonomic correlates of the formation of a functional self-stimulation system

Capability of intracerebral electrostimulation to serve as an unconditioned reinforcing stimulus in classical conditioning was studied in rabbits. Changes of such vegetative characteristics as respiration frequency and ECG were taken as criterion of conditioned response (CR) elaboration. In preliminary experiments, optimal parameters of stimulation maintaining the highest level of instrumental self-stimulation behaviour were found for each of the animals. Isolated presentation of the unconditioned reinforcing stimulus led to the increase of respiratory rate. Such kind of stimulation induced tachicardia in 5 animals, bradicardia in 3 ones, and in the remaining 6 rabbits a biphasic reaction was observed with initial tachicardia changing for bradicardia. Reactions were taken as CRs if they were similar to those to the unconditioned stimulus and appeared at the moment of omitted reinforcement. After 10 pairings of conditioned sound stimulus with positive reinforcement, CR changes of the two vegetative parameters were observed in 21,4 per cent of cases. After 40 pairings CRs were observed in 87,5 per cent of trials for cardiac and in 78,5 per cent cases for respiratory components. The results obtained confirm the idea of validity and efficiency of intracerebral stimulation of self-stimulation zones as a factor of positive reinforcement.     

Klassische Konditionierung agonistischen Verhaltens beim Syrischen Goldhamster (Mesocricetus auratus Waterhouse)

Nine different reactions of unconditioned agonistic behavior of golden hamsters (Mesocricetus auratus Waterhouse) which could be elicited by an air blow (UCS) were classically conditioned at a sound of a xylophone (tone g), a previously neutral stimulus. Tone g (CS) was paired with UCS for a different number of trials/day. In all animals exposed to over 20 pairings/day components of the agonistic behavior were brought under the control of Cs, but it was not possible to predict which one of the agonistic reactions would appear. Most of the animals generalized and also showed conditioned reactions (CR) when presenting tone c' instead of tone g. When utilizing a differential conditioning procedure the animals learned to discriminate between the different tones. Extinction of CR at tone g was reached after an individually different number of trials.     

A Neural Computational Model of Incentive Salience

Incentive salience is a motivational property with ‘magnet-like’ qualities. When attributed to reward-predicting stimuli (cues), incentive salience triggers a pulse of ‘wanting’ and an individual is pulled toward the cues and reward. A key computational question is how incentive salience is generated during a cue re-encounter, which combines both learning and the state of limbic brain mechanisms. Learning processes, such as temporal-difference models, provide one way for stimuli to acquire cached predictive values of rewards. However, empirical data show that subsequent incentive values are also modulated on the fly by dynamic fluctuation in physiological states, altering cached values in ways requiring additional motivation mechanisms. Dynamic modulation of incentive salience for a Pavlovian conditioned stimulus (CS or cue) occurs during certain states, without necessarily requiring (re)learning about the cue. In some cases, dynamic modulation of cue value occurs during states that are quite novel, never having been experienced before, and even prior to experience of the associated unconditioned reward in the new state. Such cases can include novel drug-induced mesolimbic activation and addictive incentive-sensitization, as well as natural appetite states such as salt appetite. Dynamic enhancement specifically raises the incentive salience of an appropriate CS, without necessarily changing that of other CSs. Here we suggest a new computational model that modulates incentive salience by integrating changing physiological states with prior learning. We support the model with behavioral and neurobiological data from empirical tests that demonstrate dynamic elevations in cue-triggered motivation (involving natural salt appetite, and drug-induced intoxication and sensitization). Our data call for a dynamic model of incentive salience, such as presented here. Computational models can adequately capture fluctuations in cue-triggered ‘wanting’ only by incorporating modulation of previously learned values by natural appetite and addiction-related states.     

Reinstatement of short-latency responses after asymptotic Pavlovian conditioning training by the presentation of an extraneous stimulus

The purpose of this study was to examine whether the progressive disappearance of short-latency conditioned responses, or inhibition of delay, observed in Pavlovian conditioning with long inter-stimulus intervals, could be reverted by the presentation of a novel stimulus. In one experiment, two groups of rabbits received extensive training with a short (250 ms) or a long (1500 ms) tone that overlapped and terminated with a periorbital shock unconditioned stimulus. After training, the presentation of an extraneous stimulus prior to tone onset produced a reinstatement of short latency CRs in the group trained with the long CS, but did not affect CR latency in the group trained with the short CS. This finding is consistent with Pavlov's (1927) view that conditioning with long conditioned stimuli involves the acquisition of response tendencies in the early portion of the stimulus that are subsequently suppressed by the development of an inhibitory process.     

Modelling Individual Differences in the Form of Pavlovian Conditioned Approach Responses: A Dual Learning Systems Approach with Factored Representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.     

The role of glutamate signaling in incentive salience: second‐by‐second glutamate recordings in awake Sprague‐Dawley rats

The attribution of incentive salience to reward‐predictive stimuli has been shown to be associated with substance abuse‐like behavior such as increased drug taking. Evidence suggests that glutamate neurotransmission and sequential N‐methyl‐D‐aspartate (NMDA) activation are involved in the attribution of incentive salience. Here, we further explore the role of second‐by‐second glutamate neurotransmission in the attribution of incentive salience to reward‐predictive stimuli by measuring sign‐tracking behavior during a Pavlovian conditioned approach procedure using ceramic‐based microelectrode arrays configured for sensitive measures of extracellular glutamate in awake behaving Sprague‐Dawley rats. Specifically, we show that there is an increase in extracellular glutamate levels in the prelimbic cortex (PrL) and the nucleus accumbens core (NAcC) during sign‐tracking behavior to a food‐predictive conditioned stimulus (CS+) compared to the presentation of a non‐predictive conditioned stimulus (CS?). Furthermore, the results indicate greater increases in extracellular glutamate levels in the PrL compared to NAcC in response to the CS+, including differences in glutamate release and signal decay. Taken together, the present research suggests that there is differential glutamate signaling in the NAcC and PrL during sign‐tracking behavior to a food‐predictive CS+.

     

Pavlovian conditioning: pigeon nictitating membrane

A new Pavlovian conditioning preparation was developed using the nictitating membrane of the restrained pigeon. Either visual or auditory stimuli served as conditioned stimuli (CSs) with an unconditioned stimulus (US) of a puff of air to the cornea. Movement of the nictitating membrane constituted the conditioned and unconditioned responses (CR and UR). Conditioning was studied with the Kamin blocking procedure. In agreement with findings from other conditioning preparations, responding to the redundant stimulus was attenuated relative to a stimulus that received the same number of CS-US pairings in a compound-conditioning procedure. Although response attenuation occurred, substantial individual variation was observed within the blocking procedure, a finding with some precedent in the experimental literature. Theoretical analysis and neural-network simulations indicate that inter-subject variation in response attenuation may result from differences in the extent to which contextual stimuli contribute to the functional CS.     

Functional differentiation of the theta-rhythm of the hippocampus in the cat

On the model of discriminating alimentary conditioned reflex with spatial separation of the sources of conditioned (CR, light flashes) and unconditioned (feeding trough) stimuli the theta-rhythm of the dorsal hippocampus was evaluated in cats. Two types of the theta-rhythm were observed in the spectrum of the hippocampal electrical activity: low-amplitude, consisting of a slow growing theta-waves (type I) and high-amplitude consisting of rapidly increasing theta-waves (type II). The type I theta-rhythm is sensitive to noradrenaline and correlates with behavioural forms directed immediately to the realization of alimentary motivation, while the type II is sensitive to serotonin and correlates with behavioural forms directed to the source of the conditioned stimulus. Enhancement of the type II theta-rhythm takes place during a delay of the expected conditioned stimulus. It is suggested that type I theta-rhythm reflects a level of activity of the brain structures connected with unconditioned mechanisms, with realization of biological motivations, while type II theta-rhythm is connected with conditioned reinforcing stimuli; it correlates with various forms of conditioned orienting reflexes and reflects the work of the "nervous model of reinforcing stimulus".     

Aspects of compound-conditioning to gustatory stimuli in the housefly Musca domestica L.

Houseflies (Musca domestica L.) were trained in a compound-conditioning paradigm where the conditioning stimuli were water and 1% sodium chloride solution. The unconditioned stimulus was 16% sucrose solution. A high degree of conditioning was produced. Control experiments for pseudoconditioning and sensitization revealed that the response of the flies to the procedure were due to an associative process. Experiments with double-water conditioned stimuli and with an interval between stimulus presentations indicate that the marked response to the first conditioned stimulus may be due to stimulus generalization in which the water, rather than the salt component of both stimuli served as the learning cue.     

Implicit Memory in Monkeys: Development of a Delay Eyeblink Conditioning System with Parallel Electromyographic and High-Speed Video Measurements

Delay eyeblink conditioning, a cerebellum-dependent learning paradigm, has been applied to various mammalian species but not yet to monkeys. We therefore developed an accurate measuring system that we believe is the first system suitable for delay eyeblink conditioning in a monkey species (Macaca mulatta). Monkey eyeblinking was simultaneously monitored by orbicularis oculi electromyographic (OO-EMG) measurements and a high-speed camera-based tracking system built around a 1-kHz CMOS image sensor. A 1-kHz tone was the conditioned stimulus (CS), while an air puff (0.02 MPa) was the unconditioned stimulus. EMG analysis showed that the monkeys exhibited a conditioned response (CR) incidence of more than 60% of trials during the 5-day acquisition phase and an extinguished CR during the 2-day extinction phase. The camera system yielded similar results. Hence, we conclude that both methods are effective in evaluating monkey eyeblink conditioning. This system incorporating two different measuring principles enabled us to elucidate the relationship between the actual presence of eyelid closure and OO-EMG activity. An interesting finding permitted by the new system was that the monkeys frequently exhibited obvious CRs even when they produced visible facial signs of drowsiness or microsleep. Indeed, the probability of observing a CR in a given trial was not influenced by whether the monkeys closed their eyelids just before CS onset, suggesting that this memory could be expressed independently of wakefulness. This work presents a novel system for cognitive assessment in monkeys that will be useful for elucidating the neural mechanisms of implicit learning in nonhuman primates.     

Nicotine-evoked conditioned responding is dependent on concentration of sucrose unconditioned stimulus

Previous studies have shown that the interoceptive nicotine conditional stimulus (CS) functions similarly to exteroceptive CSs such as lights or environments. For instance, the appetitive conditioned response (CR) evoked when nicotine is repeatedly paired with sucrose presentations (the unconditioned stimulus; US) is sensitive to changes in training dose (CS salience) and the contiguity between the CS effects and sucrose. The current study was conducted to extend this research by examining the possible role of US intensity in CR acquisition and maintenance. Rats were trained using one of four sucrose concentrations: 0, 4, 16, or 32% (w/v). On nicotine sessions (0.4 mg base/kg), rats received 36 deliveries (4 s each) of their assigned concentration intermittently throughout the session; sucrose was withheld on saline sessions. In all groups, an appetitive goal-tracking CR was acquired at a similar rate. However, the asymptotic CR level varied with sucrose concentration. The magnitude of the CR was increased in rats trained with higher sucrose US concentrations. These findings are consistent with previous Pavlovian conditioning research, and extend the conditions under which the nicotine state functions as an interoceptive conditional stimulus.     

Concomitant Pavlovian conditioning of heart rate and leg flexion responses in the rat

Pavlovian conditioning was studied in male Fischer 344 rats using tones as the conditioned stimulus (CS) and footshock as the unconditioned stimulus (UCS). Different groups of animals received (a) contiguous CS-UCS pairings with a 0.5 sec CS, (b) contiguous CS-UCS pairings with a 4.0 sec CS, or (c) a random sequence of noncontiguous tones and shocks using either a 0.5 sec or a 4.0 sec CS. Heart rate (HR) and leg flexion (LF) responses were recorded. Leg flexion conditioning occurred only in the 0.5 sec contiguous group. Decelerative HR CRs occurred only in the 4.0 sec contiguous group. Accelerative HR changes occurred in the other two groups but were significantly greater in the 0.5 sec contiguous group. These results are similar to but not identical to those obtained during eyeblink or nictitating membrane conditioning in rabbits, and suggest that the topography of the Pavlovian HR CR is dependent on the simultaneous occurrence of other classically conditioned responses.     

A model of ventilatory instability induced in the unrestrained rat.

A classic conditioning paradigm was used to examine the hypothesis that perturbations during sleep in the neonate rat can have a lasting impact on breathing. During the first 4 wk of life, stimuli were presented to rats during behaviorally defined sleep. In a conditioned hypoxic (CH) group, brief periods of hypoxic gas were used as the unconditioned stimulus. Tactile and auditory stimuli were used as the conditioned stimuli. In a conditioned control (CC) group, air was used as the unconditioned stimulus. A third group of unconditioned control (UC) rats was not exposed to the conditioning paradigm. Animals were provided routine care for 3.5 mo; ventilation was then assessed using plethysmography. Conditioning during neonatal life produced increased ventilatory irregularities and apnea during behaviorally defined sleep in adult rats. Both CH and CC rats showed a significantly greater number of apneic events compared with UC rats. Over a 2-h sleep period, CH rats exhibited a total of 105.1 +/- 9.4 (SE) apneic events, CC rats 69.4 +/- 4.2 events, and UC rats 42.1 +/- 3.1 events [F(2,18) = 25.568; P < 0.0001]. These findings suggest that experiences in the first few weeks of life will alter ventilatory patterning in the adult animal.     

External inhibition in a goldfish (Carassius auratus) classical conditioning situation

Goldfish were classically conditioned with a light as the CS and shock as the US. The UR was a decrease in respiration. After 15 or 60 conditioning trials the fish were tested with novel stimuli (clicks) during the CS-US interval. High and moderate intensity novel stimuli produced a significant decrease in CRs (external inhibition) for fish with 60 conditioning trials (5.5 or 10.5 sec CS-US interval), but not fish with 15 conditioning trials. Low intensity novel stimuli produced no evidence for disinhibition (an increase in CRs). Control groups (e.g., groups with random presentations of the CS and US) showed that the external inhibition for fish with 60 conditioning trials was inhibition of a true CR.     

Task Specialization and Odor Effects on Proboscis Extension Conditioning in Bumblebees (Bombus huntii)

Workers in a social insect colony have distinct experiences that may affect their performance in a learning task. In this study using free-foraging and flight-cage bumblebee Bombus huntii colonies, the strength of olfactory proboscis extension conditioning (PEC) was affected by a bee’s task specialization as a nurse or forager and the stimulus odor. Nurses (n?=?26) learned to respond to the odors 1-hexanol and lavender, but foragers (n?=?25) had inhibited conditioning to both odors. More nurses (73 %) than foragers (48 %) had at least one conditioned response (CR), and nurses displayed significantly more CRs than foragers. As expected, a pseudoconditioned control group (n?=?9) showed very few CRs. Among bees that were given a second day of trials, nurses—but not foragers—showed modest improvement. Such strong inhibition of PEC has not been described in honeybee or bumblebee foragers. The stimulus odor also affected conditioning strength in nurses. Lavender, a familiar odor, elicited earlier and more CRs than 1-hexanol. We propose that learning floral odors in the context of foraging may inhibit PEC in bumblebee foragers, whereas exposure to odors in the honey stores may prime subsequent learning in nurses.     

Inhibitory effects of external stimulation on neuronal spike response in the cat association cortex during conditioning

The effects of external stimulus presentation on neuronal spike response induced in area 7 by conditioned and unconditioned stimuli were investigated in waking cats. Pointing reaction arising in response to the action of external stimuli led to the phasing out of spike response to unconditioned cutaneous stimulation, as well as depression or complete suppression of response in area 7 neurons to conditioned stimulus presentation, despite occurrence of conditioned reflex motion. External stimuli no longer exerted an inhibitory effect on the neuronal spike response induced by protracted use of conditioned and unconditioned stimuli. The fact that the conditioned lever-pressing reflex could occur either following or without neuronal spike response in area 7 would raise the question of whether neurons belonging to this area always participate directly in the triggering and occurrence of conditioned reflex motion.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev. Translated from Neirofiziogiya, Vol. 22, No. 2, pp. 147–155, March–April, 1990.     

The role of the lateral and medial hypothalamus in reproducing a motor reaction as the signal during the acquisition of classical conditioned reflexes]

In experiments on three dogs there was shown that testing electrostimulation of the lateral hypothalamus reproduced the motor reaction which is a signal stimulus at elaboration of classic alimentary conditioned reflexes (CRs) and did not reproduce it at elaboration of classic defensive CR. Testing electrostimulation of medial parts of the hypothalamus reproduced, as LH electrostimulation the "signal" motor reaction, but in less percentage of cases, during elaboration of classic alimentary CRs and did not reproduce it at elaboration of classic defensive CRs. The reproduction of the signal motor reaction at LH electrostimulation is connected with activation of backward conditioned connection from motivation structures of the hypothalamus to representation of the signal stimulus in the motor cortex.     

Updating versus Exposure to Prevent Consolidation of Conditioned Fear

Targeting the consolidation of fear memories following trauma may offer a promising method for preventing the development of flashbacks and other unwanted re-experiencing symptoms that characterise Posttraumatic Stress Disorder (PTSD). Research has demonstrated that performing visuo-spatial tasks after analogue trauma can block the consolidation of fear memory and reduce the frequency of flashbacks. However, no research has yet used verbal techniques to alter memories during the consolidation window. This is surprising given that the most effective treatments for PTSD are verbally-based with exposure therapy and trauma-focused cognitive behavioural therapy gaining the most evidence of efficacy. Psychological therapies aim to reduce the conditioned fear response, which is in keeping with the preliminary finding that an increased propensity for fear conditioning may be a vulnerability factor for PTSD. Our research had two aims. We investigated the degree to which individual differences in fear conditioning predict the development of PTSD symptoms. We also compared the preventative effects of two clinically informed psychological techniques administered during the consolidation window: exposure to the trauma memory and updating the meaning of the trauma. 115 healthy participants underwent a fear conditioning paradigm in which traumatic film stimuli (unconditioned stimuli) were paired with neutral stimuli (conditioned stimuli). Participants were randomly allocated to an updating, exposure or control group to compare the effects on the conditioned fear response and on PTSD symptomatology. The results showed that stronger conditioned responses at acquisition significantly predicted the development of PTSD symptoms. The updating group, who verbally devalued the unconditioned stimulus within the consolidation window, experienced significantly lower levels of PTSD symptoms during follow-up than the exposure and control groups. These findings are consistent with clinical interventions for chronic PTSD and have important implications for identifying those at risk as well as for designing novel early interventions to prevent the development of PTSD.     

豚鼠齿状回颗粒细胞在追踪性眨眼条件反应巩固过程中的活动

海马在追踪性眨眼条件反应的巩固过程中发挥重要作用,但解剖学上与其紧密联系的齿状回在此过程中的作用尚不清楚. 实验拟观察齿状回颗粒细胞在追踪性眨眼条件反应巩固过程中的放电活动,阐明齿状回在此海马依赖任务中所发挥的作用. 条件反射组动物 (n=8) 首先接受 200 ms 声音条件刺激,间隔 600 ms 后,再被给予 200 ms 吹气非条件刺激,多次重复配对,建立追踪性眨眼条件反应. 对照组动物 (n=8) 接受非配对出现的上述两种刺激. 采用在体单细胞外记录技术,研究习得条件反应豚鼠的齿状回颗粒细胞在条件反应巩固过程中的放电活动. 结果显示:a. 通过 14 天的训练,条件反射组动物均建立了追踪性眨眼条件反应,而非配对组动物则没有建立该条件反应;b. 齿状回颗粒细胞在追踪性眨眼条件反应的巩固过程中表现出不同的活动模式,如在声音条件刺激、间隔期或吹气非条件刺激出现后活动的增强. 这些结果提示:齿状回可能参与巩固追踪性眨眼条件反应所需的神经环路,其颗粒细胞在追踪性眨眼条件反应巩固过程中可能编码不同的信息.     

An Analysis of Variability in the Feeding Motor Program of the Honey Bee; the Role of Learning in Releasing a Modal Action Pattern

Sequences of behavior are highly predictable (stereotyped) during some segments and less predictable during transitions between those segments. Statistical characterization of behavior must involve observation of the behavior under different stimulus conditions, which includes how stimuli associated with behavioral releasers come to trigger behavior during learning. For example, the feeding motor program (FMP) of the honey bee Apis mellifera (Hymenoptera: Apidae) during proboscis extension can be divided into three response phases (Rehder 1987). We conditioned honey bees in an olfactory conditioning paradigm with one or several rewarded trials during which an odorant was paired with the sugar-water unconditioned stimulus (US); the latter elicits proboscis extension and feeding in properly motivated bees. By recording electromyogram activity from one of the muscles that move the proboscis during feeding, we quantified the bees' responses during an unrewarded test with either the conditioned odorant, a different (novel) odorant, or the sugar-water US. Various parameters of the response phases of the FMP varied in a consistent manner across these experimental treatments, with certain stimuli eliciting stronger, more consistent responses. The different response phases followed one another in time with some variability and statistical uncertainty. For example, the length of an individual licking movement with the glossa was relatively invariant, and may indicate that this parameter can be used to differentiate the FMP into more basic, independent units. Our work shows how learned information may release action patterns in ways slightly different from traditional sign-stimuli releasers.