A pitfall for the expectancy theory of human eyelid conditioning

Two simple eyeblink conditioning experiments with random intermittent reinforcement schedules were performed. In Experiment 1, subjects had to rate their expectancy for an unconditioned stimulus (US) on a seven-level scale prior to each trial. As anticipated, expectancy for US increased with a successive conditioned stimulus (CS) alone, and decreased with successive CS-US pairings. However, Experiments 1 and 2 showed that the frequency of eyeblink conditioned responses (CRs) evolved in a direction opposite to that of expectancy changes: CRs increased, whereas expectancy for US decreased, and vice versa. The possible effect of sensitization on eyeblink response was ruled out by the lack of a run effect in an unpaired control group in Experiment 2. These results tend to disconfirm the expectancy theory of conditioning. Although they were explicitly predicted by the conventional "strength" theory of conditioning, an alternative interpretation is proposed within a cognitive framework.     

Reinstatement of short-latency responses after asymptotic Pavlovian conditioning training by the presentation of an extraneous stimulus

The purpose of this study was to examine whether the progressive disappearance of short-latency conditioned responses, or inhibition of delay, observed in Pavlovian conditioning with long inter-stimulus intervals, could be reverted by the presentation of a novel stimulus. In one experiment, two groups of rabbits received extensive training with a short (250 ms) or a long (1500 ms) tone that overlapped and terminated with a periorbital shock unconditioned stimulus. After training, the presentation of an extraneous stimulus prior to tone onset produced a reinstatement of short latency CRs in the group trained with the long CS, but did not affect CR latency in the group trained with the short CS. This finding is consistent with Pavlov's (1927) view that conditioning with long conditioned stimuli involves the acquisition of response tendencies in the early portion of the stimulus that are subsequently suppressed by the development of an inhibitory process.     

Properties of olfactory conditioning in the housefly, Musca domestica

The housefly, Musca domestica, was conditioned to odours using the proboscis extension response to labellar stimulation with sucrose solution as an unconditioned response and the properties of conditioning were investigated. Among trials including forward pairing of the conditioned stimulus (CS) with the unconditioned stimulus (US), backward pairing and isolated presentations of CS and US, only forward pairing is effective on the acquisition of conditioning. Backward pairing combined with forward pairing does not influence the effectiveness of the forward pairing. CS given overlapping with a US presentation permits only weak conditioning. The acquisition of conditioning decreases with increase of the CS-US interval. In the differential conditioning situation to two odours, discriminative responses are observed. In the flies conditioned with one antenna, the conditioned response is elicited not only by stimulation of the antenna used for conditioning but also by stimulation of the antenna not used for conditioning, although the response using the former is higher than with the latter. The ability to be conditioned is reduced immediately after fastening on a clay bed and increases with time. Ability can also be improved by transection of the ventral nerve cord.     

A time contraction effect of acute tail-pinch stress on the associative learning of rats

The effect of tail-pinch stress interpolated between the saccharin conditioned stimulus (CS) and the illness-inducing unconditioned stimulus (US) during long-trace taste-aversion conditioning was examined in young- and old adult rats with a two-cylinder (saccharin versus water) test. A 2 x 2 x 4 factorial ANOVA was performed on percent-preference-for-saccharin data, with age (young, old), stress condition (stressed, non-stressed), and CS-US interval (22.5-, 45-, 90-, and 180-min) being the factors under consideration. The ANOVA yielded only significant main effects of stress condition and CS-US interval. These findings indicate that stress weakens the CS-US association as evidenced by a higher percent preference for saccharin in the stressed rats than in non-stressed rats at all CS-US intervals. A comparison of the stressed and non-stressed conditioned rats with pseudo-conditioned controls showed that the non-stressed rats formed strong aversions up to the 45-min CS-US interval whereas the stressed rats showed no conditioning beyond the 22.5 min CS-US interval, indicating that stress decreases the effective CS-US interval. Results were interpreted in terms of time-contraction and an internal biological countdown timer hypothesized to govern processes involved in associative learning over long delays.     

Role of reuniens nucleus projections to the medial prefrontal cortex and to the hippocampal pyramidal CA1 area in associative learning

We studied the interactions between short- and long-term plastic changes taking place during the acquisition of a classical eyeblink conditioning and following high-frequency stimulation (HFS) of the reuniens nucleus in behaving mice. Synaptic changes in strength were studied at the reuniens-medial prefrontal cortex (mPFC) and the reuniens-CA1 synapses. Input/output curves and a paired-pulse study enabled determining the functional capabilities of the two synapses and the optimal intensities to be applied at the reuniens nucleus during classical eyeblink conditioning and for HFS applied to the reuniens nucleus. Animals were conditioned using a trace paradigm, with a tone as conditioned stimulus (CS) and an electric shock to the trigeminal nerve as unconditioned stimulus (US). A single pulse was presented to the reuniens nucleus to evoke field EPSPs (fEPSPs) in mPFC and CA1 areas during the CS-US interval. No significant changes in synaptic strength were observed at the reuniens-mPFC and reuniens-CA1 synapses during the acquisition of eyelid conditioned responses (CRs). Two successive HFS sessions carried out during the first two conditioning days decreased the percentage of CRs, without evoking any long-term potentiation (LTP) at the recording sites. HFS of the reuniens nucleus also prevented the proper acquisition of an object discrimination task. A subsequent study revealed that HFS of the reuniens nucleus evoked a significant decrease of paired-pulse facilitation. In conclusion, reuniens nucleus projections to prefrontal and hippocampal circuits seem to participate in the acquisition of associative learning through a mechanism that does not required the development of LTP.     

Learning and anticipatory behaviour in a “sit-and-wait” predator: The Atlantic halibut

We studied the learning capacities and anticipatory behaviour in a “sit-and-wait” predatory fish, the Atlantic halibut, Hippoglossus hippoglossus. In Experiment 1 two groups of halibut received series of light flashes (conditioned stimulus, CS) that started before delivery of food (unconditioned stimulus, US) and persisted until after food delivery, i.e. delay conditioning. Control groups received unpaired CS and US presentations. The anticipatory behaviour of delay conditioned halibut consisted mainly of take-offs towards the surface shortly after onset of the CS. In Experiment 2 six groups of halibut were trained in three trace conditioning procedures: Two groups with 20 s, two groups with 60 s and two groups with 120 s trace interval. Learning was evident in the 20 and 60 s trace groups and in one of the 120 s trace groups. In contrast to delay conditioning the anticipatory behaviour of trace conditioned halibut was characterized by subtle movements near the tank floor with orientation towards the CS. The cautious responses of halibut after trace conditioning differed markedly from what is observed in other fish species and are suggested to reflect a “sit-and-wait” foraging strategy that requires the predator to remain undetected until the prey is within lunging range.     

Exposure to cold: aversive Pavlovian conditioning in individual Drosophila melanogaster

Abstract. Temperature changes can be especially threatening for ectotherms, such as Drosophila melanogaster (Diptera: Drosophilidea Meigen, 1830 ), and in this study we tested whether flies can associate olfactory stimuli with a sudden drop in temperature. Such Pavlovian conditioning would allow them to make appropriate behavioural and/or physiological responses in the future. We found that exposing individual flies to one of two odours in the presence of a sudden drop in temperature resulted in Pavlovian conditioning with flies subsequently avoiding the odour paired with cold. The characteristics of Pavlovian conditioning in flies were comparable to those observed for mammalian species. Specifically, the strength of conditioning increased with increasing intensity of the cold and decreased as the time interval between the olfactory stimulus (CS) and cold (US) was lengthened. Finally, the order in which CS and US were presented affected the strength of conditioning. Learning was observed when the CS preceded US and when the US immediately preceded the CS, but not when the CS preceded the US by 30 s or more. These results provide further evidence for learning in individual flies, and confirm that Pavlovian conditioning is a general mechanism used by organisms to obtain information about their environment.     

Stimulus representation in SOP: II. An application to inhibition of delay

The componential extension of SOP accounts for conditioned response (CR) timing in Pavlovian conditioning by assuming that learning accrues with relative independence to stimulus elements that are differentially occasioned during the duration of the conditioned stimulus (CS). SOP, using a competitive learning rule and the assumption that temporal learning emerges via resolution of what is equivalent to an "AX+BX-" discrimination, predicts a progressive increase in the latency of the CR over training, or what Pavlov refer to as "inhibition of delay." Other componential models, which use noncompetitive learning rules, do not predict inhibition of delay. Either type of model makes the prediction indicated, independently of the length of the CS-unconditioned stimulus (US) interval. We report two experiments that demonstrated inhibition of delay when rabbits were trained with relatively long, but not with short, CS-US intervals. To account for this divergence, we assumed that the SOP stimulus trace involves two kinds of elements, some with a temporally distributed pattern of activity over the duration of the CS duration, and some with a randomly distributed pattern. This stimulus representation, not only allows for inhibition of delay with long but not short CS-US intervals, but in combination with SOP's performance rule deduces CR's with "Weber variability."     

The reflection of the operations of comparing afferent flow with internal pretuning in the neuronal impulse activity of the frontal cortex in the dog

From 192 neurons, recorded in the frontal cortex of dogs trained for differential classical conditioning of salivary reflex, 24 cells were identified related to the detection of matching-mismatching conditioned and unconditioned stimuli (CS and US) to internal set. Impulse reactions of the 1-st group of neurones (11 cells) to US disappeared, when any deviation from standard pattern of CS-US pairing occurred or when US characteristics were suddenly changed. These reactions developed again at the 3-5th repeated presentation of the stimuli delivered in "new" regime. Neurones of the 2-nd group (13 cells) were activated in response to CS only when two sequential CSs had different meaning. The 1-st group of neurones is considered as detectors of matching, and the 2-nd group--as detectors of mismatching the stimulus to its neural model, formed by mechanism of reinforcement.     

Modelling Individual Differences in the Form of Pavlovian Conditioned Approach Responses: A Dual Learning Systems Approach with Factored Representations

Reinforcement Learning has greatly influenced models of conditioning, providing powerful explanations of acquired behaviour and underlying physiological observations. However, in recent autoshaping experiments in rats, variation in the form of Pavlovian conditioned responses (CRs) and associated dopamine activity, have questioned the classical hypothesis that phasic dopamine activity corresponds to a reward prediction error-like signal arising from a classical Model-Free system, necessary for Pavlovian conditioning. Over the course of Pavlovian conditioning using food as the unconditioned stimulus (US), some rats (sign-trackers) come to approach and engage the conditioned stimulus (CS) itself – a lever – more and more avidly, whereas other rats (goal-trackers) learn to approach the location of food delivery upon CS presentation. Importantly, although both sign-trackers and goal-trackers learn the CS-US association equally well, only in sign-trackers does phasic dopamine activity show classical reward prediction error-like bursts. Furthermore, neither the acquisition nor the expression of a goal-tracking CR is dopamine-dependent. Here we present a computational model that can account for such individual variations. We show that a combination of a Model-Based system and a revised Model-Free system can account for the development of distinct CRs in rats. Moreover, we show that revising a classical Model-Free system to individually process stimuli by using factored representations can explain why classical dopaminergic patterns may be observed for some rats and not for others depending on the CR they develop. In addition, the model can account for other behavioural and pharmacological results obtained using the same, or similar, autoshaping procedures. Finally, the model makes it possible to draw a set of experimental predictions that may be verified in a modified experimental protocol. We suggest that further investigation of factored representations in computational neuroscience studies may be useful.     

Altering the synchrony of stimulus trace processes: tests of a neural-network model

A previously described neural-network model (Desmond 1991; Desmond and Moore 1988; Moore et al. 1989) predicts that both CS-onset-evoked and CS-offset-evoked stimulus trace processes acquire associative strength during classical conditioning, and that CR waveforms can be altered by manipulating the time at which the processes are activated. In a trace conditioning paradigm, where CS offset precedes US onset, the model predicts that onset and offset traces act in synchrony to generate unimodal CR waveforms. However, if the CS duration is subsequently lengthened on CS-alone probe trials, the model predicts that onset and offset traces will asynchronously contribute to CR output and bimodal CRs will be generated. In a delay conditioning paradigm, in which US onset occurs prior to CS offset, the model predicts that only the onset process will gain associative strength, and hence, only unimodal CRs will occur. Using the rabbit conditioned nictitating membrane response preparation, we found experimental support for these predictions.This research was supported by National Science Foundation grant BNS 88-10624 and Air Force Office of Scientific Research grant 89-0391.     

豚鼠齿状回颗粒细胞在追踪性眨眼条件反应巩固过程中的活动

海马在追踪性眨眼条件反应的巩固过程中发挥重要作用,但解剖学上与其紧密联系的齿状回在此过程中的作用尚不清楚. 实验拟观察齿状回颗粒细胞在追踪性眨眼条件反应巩固过程中的放电活动,阐明齿状回在此海马依赖任务中所发挥的作用. 条件反射组动物 (n=8) 首先接受 200 ms 声音条件刺激,间隔 600 ms 后,再被给予 200 ms 吹气非条件刺激,多次重复配对,建立追踪性眨眼条件反应. 对照组动物 (n=8) 接受非配对出现的上述两种刺激. 采用在体单细胞外记录技术,研究习得条件反应豚鼠的齿状回颗粒细胞在条件反应巩固过程中的放电活动. 结果显示:a. 通过 14 天的训练,条件反射组动物均建立了追踪性眨眼条件反应,而非配对组动物则没有建立该条件反应;b. 齿状回颗粒细胞在追踪性眨眼条件反应的巩固过程中表现出不同的活动模式,如在声音条件刺激、间隔期或吹气非条件刺激出现后活动的增强. 这些结果提示:齿状回可能参与巩固追踪性眨眼条件反应所需的神经环路,其颗粒细胞在追踪性眨眼条件反应巩固过程中可能编码不同的信息.     

Training-Dependent Associative Learning Induced Neocortical Structural Plasticity: A Trace Eyeblink Conditioning Analysis

Studies utilizing general learning and memory tasks have suggested the importance of neocortical structural plasticity for memory consolidation. However, these learning tasks typically result in learning of multiple different tasks over several days of training, making it difficult to determine the synaptic time course mediating each learning event. The current study used trace-eyeblink conditioning to determine the time course for neocortical spine modification during learning. With eyeblink conditioning, subjects are presented with a neutral, conditioned stimulus (CS) paired with a salient, unconditioned stimulus (US) to elicit an unconditioned response (UR). With multiple CS-US pairings, subjects learn to associate the CS with the US and exhibit a conditioned response (CR) when presented with the CS. Trace conditioning is when there is a stimulus free interval between the CS and the US. Utilizing trace-eyeblink conditioning with whisker stimulation as the CS (whisker-trace-eyeblink: WTEB), previous findings have shown that primary somatosensory (barrel) cortex is required for both acquisition and retention of the trace-association. Additionally, prior findings demonstrated that WTEB acquisition results in an expansion of the cytochrome oxidase whisker representation and synaptic modification in layer IV of barrel cortex. To further explore these findings and determine the time course for neocortical learning-induced spine modification, the present study utilized WTEB conditioning to examine Golgi-Cox stained neurons in layer IV of barrel cortex. Findings from this study demonstrated a training-dependent spine proliferation in layer IV of barrel cortex during trace associative learning. Furthermore, findings from this study showing that filopodia-like spines exhibited a similar pattern to the overall spine density further suggests that reorganization of synaptic contacts set the foundation for learning-induced neocortical modifications through the different neocortical layers.     

Conditioned modification of viscerosensory evoked potentials during sleep and wakefulness in cats

The effects of classic conditioning on the viscerosensory evoked potentials (EPs) were studied in twenty cats during wakefulness (W), slow-wave-sleep (SWS) and paradoxical sleep (PS). Four types of the experiment were performed on four groups of animals. Weak, non-painful stimulation of the small intestine or of the left splanchnic nerve was used as conditional stimulus (CS) in all experiments. A painful or non-painful shock on the left radial nerve served as unconditional stimulus (US) which followed the CS with a delay of 500 ms. In the first and second series of experiments, the CS was paired with non-painful or painful CS during W. In the third and fourth types of experiment, weak US was used and conditioning was done during SWS or PS. The evoked responses were recorded from the primary (SI) and secondary (SII) somatosensory and associative (AS) cortex, the thalamus (VPL), hypothalamus (HPT) and dorsal hippocampus (HPC). In each experiment, the stimulus pairings resulted in a complex electrographic conditional response (CR) which included an amplitude increase of the late components of EP's (early CR) and the development of a wave of 500 ms latency (delayed CR). In the second experiment, however, a behavioural CR (limb flexion) also appeared. All these CRs proved to be extinguishable. The recall of CR established during W was successful in SWS. The traces of CS-US pairings during SWS could, however, be elicited only in SWS. Both establishment and recall of CR were unsuccessful during PS. The possible mechanism of the effects originating from an interaction of conditioning and sleep on the viscerosensory inputs of the brain are discussed.     

Pavlovian conditioning: pigeon nictitating membrane

A new Pavlovian conditioning preparation was developed using the nictitating membrane of the restrained pigeon. Either visual or auditory stimuli served as conditioned stimuli (CSs) with an unconditioned stimulus (US) of a puff of air to the cornea. Movement of the nictitating membrane constituted the conditioned and unconditioned responses (CR and UR). Conditioning was studied with the Kamin blocking procedure. In agreement with findings from other conditioning preparations, responding to the redundant stimulus was attenuated relative to a stimulus that received the same number of CS-US pairings in a compound-conditioning procedure. Although response attenuation occurred, substantial individual variation was observed within the blocking procedure, a finding with some precedent in the experimental literature. Theoretical analysis and neural-network simulations indicate that inter-subject variation in response attenuation may result from differences in the extent to which contextual stimuli contribute to the functional CS.     

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.     

Memorization of contextual and CS conditioned fear response (freezing) in a one-trial acquisition paradigm.

In fear-conditioned Wistar rats freezing was induced by the delivery of a series of footshocks paired to tones (CS) in a specific conditioning chamber (context). CS and contextual fear were acquired in the same single conditioning session without preexposition to the conditioning chamber (day 1). Different groups of animals were conditioned employing three increasing US (footshock) intensities (0.25, 0.5, 0.75 mA). During the retention sessions context and CS conditioned freezing (fear response) were measured using a paradigm that fulfilled the following conditions: i) CS freezing retention was measured in a context different from the conditioning one; ii) CS and context freezing were measured at increased delays after the training session (days 3 and 4, 14 and 15, 28 and 29). The results show that there are significant differences between CS and context freezing retention, which are clearly related to delay after the initial session and to US intensity. In particular: 1) conditioned freezing to a discrete tone is better retained than conditioned freezing to context (irrespective of US intensity); 2) context freezing is directly related to US intensity much more than to tone freezing; 3) context freezing is easier to extinguish than tone freezing. The results are discussed in relation to previous ones and to their relevance to freezing genesis neural correlates.     

A comparison of the effects of vicariously instigated classical conditioning and direct classical conditioning procedures.

Two groups of Ss received either two or 16 paired classical conditioning trails beyond the peak CR. A third group received the same stimuli as the 16 postpeak condition but in an unpaired and random order. The stimuli in all three groups were delivered directly to S. Subsequently, all three groups, including a fourth which was not given any prior direct classical conditioning, were exposed to vicariously instigated classical conditioning. This consisted of having S observe someone (model) employed by E who received the same CS as was delivered during direct conditioning. The CS was paired with the feigned arm movement of the model, simulating a reaction to shock. This vicarious classical conditioning procedure when compared to direct classical conditioning resulted in smaller GSR magnitudes for both the CRs and UCRs. Previous experience with direct classical conditional seems to have had an attenuating effect on GSR magnitude during the vicarious situation. A postexperimental questionnaire tended to support the results, and the relationship between the present study and current classical conditioning theory is discussed.     

Neural-network simulations of two context-dependence phenomena

This paper describes simulations of two context-dependence phenomena in Pavlovian conditioning, using a neural-network model that draws on knowledge from neuroscience and makes no distinction between operant and respondent learning mechanisms. One phenomenon is context specificity or the context-shift effect, the decrease of conditioned responding (CR) when the conditioned stimulus (CS) is tested in a context different from the one in which it had been paired with the unconditioned stimulus (US). The other effect is renewal, the recovery of CR in the training context after extinction in another context. For specificity (simulation 1), two neural networks were first given 200 CS-US pairings in a context. Then, the CS was tested either in the training context or a new context. Output activations in the new context were substantially lower. For renewal (simulation 2), two networks were first given 200 CS-US pairings in a context, then 100 extinction trials in either the same context or a new one, and then tested back in the training context. Output activations during the test phase were substantially higher after extinction in a new context. The results are interpreted in terms of the dynamics of activations and weights.     

Conditioning from an information processing perspective

The framework provided by Claude Shannon's [Bell Syst. Technol. J. 27 (1948) 623] theory of information leads to a quantitatively oriented reconceptualization of the processes that mediate conditioning. The focus shifts from processes set in motion by individual events to processes sensitive to the information carried by the flow of events. The conception of what properties of the conditioned and unconditioned stimuli are important shifts from the tangible properties to the intangible properties of number, duration, frequency and contingency. In this view, a stimulus becomes a CS if its onset substantially reduces the subject's uncertainty about the time of occurrence of the next US. One way to represent the subject's knowledge of that time of occurrence is by the cumulative probability function, which has two limiting forms: (1) The state of maximal uncertainty (minimal knowledge) is represented by the inverse exponential function for the random rate condition, in which the US is equally likely at any moment. (2) The limit to the subject's attainable certainty is represented by the cumulative normal function, whose momentary expectation is the CS-US latency minus the time elapsed since CS onset. Its standard deviation is the Weber fraction times the CS-US latency.