Activity of neurons in the pedunculopontine nucleus during defensive instrumental conditioning

The activity of 109 neurons in the compact and diffuse parts of the pedunculopontine tegmental nucleus (PPTg) was recorded in freely moving rabbits during the acquisition and actualization of the defensive instrumental conditioned reflex. It was found that 47% of the recorded neurons responded to a conditioned stimulus (CS). This finding suggests the involvement of the PPTg in the instrumental conditioning. A significant prevalence of the excitatory conditioned responses to the CS suggests the predominantly activating influence of the PPTg on its projection structures during conditioning. The neuronal responses to the CS were classified into several basic patterns reflecting stimulus effects, the structure of the behavioral act, and the nature of the reinforcement. They indicated the involvement of the PPTg in attention, motor learning, and reinforcement. A significant decrease in the reactivity of the CS of the PPTg neurons as a result of learning specialization was shown. The revealed differences in the associative and reactive properties (with respect to CS) between the neurons of the compact and diffuse PPTg parts testify to the functional heterogeneity of this structure and suggest the leading role of the cholinergic compact part of the PPTg in the instrumental defensive conditioning. Thus, the obtained evidence suggests the involvement of the PPTg in the mechanisms of attention and acquisition of the active defensive motor conditioning.     

The individual organization of frontal-hippocampal networks during realization of different behavioral tasks

Four cats were subjected to appetitive instrumental conditioning with light as a conditioned stimulus by the method of "active choice" of the reinforcement quality: short-delay conditioned bar-press responses were followed by bread-meat mixture and the delayed responses--by meat. The animals differed in behavior strategy: four animals preferred bar-pressing with long delay (so called "self-control" group); two animal preferred bar-pressing with short-delay (so called "impulsive" group). Then all the animals were learned to short-delay (1 s) instrumental conditioned reflex to light (CS+) reinforced by meat. The multiunit activity in the frontal cortex and the hippocampus (CA3) was recorded through chronically implanted nichrome-wire semimicroelectrodes. The interactions among the neighboring neurons in the frontal cortex and hippocampus (within the local neuronal networks) and between the neurons of the frontal cortex and hippocampus (distributed neuronal networks of frontal-hippocampal and hippocampal-frontal directions) were evaluated by means of statistical crosscorrelation analysis of the spike trains. Crosscorrelation interneuronal connections in the delay range 0-100 ms were explored. It was shown that the functional organization of the frontal and hippocampal neuronal networks differed in choice behavior and was similar during realization of short-delayed conditioned reflex. We suggest that the local and distributed neural networks of the frontal cortex and hippocampus take part in the realization of cognitive behavior, in particularly in the processes of the decision making.     

Interaction between neurons of the frontal cortex and hippocampus during the realization of choice of food reinforcement quality in cats

Six cats were subjected to the procedure of appetitive instrumental conditioning (with light as a conditioned stimuls) by the method of the "active choice" of reinforcement quality. Short-delay conditioned bar-press responses were rewarded with bread-meat mixture, and the delayed responses were reinforced by meat. The animals differed in behavior strategy: four animals preferred the bar-pressing with a long delay (the so-called "self-control" group), and two cats preferred the bar-pressing with a short delay (the so-called "impulsive" group). Multiunit activity in the frontal cortex and hippocampus (CA3) was recorded via chronically implanted nichrome wire semimicroelectrodes. An interaction between the neighboring neurons in the frontal cortex and hippocampus (within local neural networks) and between the neurons of the frontal cortex and hippocampus (distributed neural networks in frontal-hippocampal and hippocampal-frontal directions) was evaluated by means of statistical crosscorrelation analysis of spike trains. Crosscorrelations between neuronal spike trains in the delay range of 0-100 ms were explored. It was shown that the number of crosscorrelations between the neuronal discharges both in the local and distributed networks was significantly higher in the "self-control" cats. It was suggested that the local and distributed neural networks of the frontal cortex and hippocampus are involved in the system of brain structures which determine the behavioral strategy of animals in the "self-control" group.     

Light-induced activation of distinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae

During classical conditioning, a positive or negative value is assigned to a previously neutral stimulus, thereby changing its significance for behavior. If an odor is associated with a negative stimulus, it can become repulsive. Conversely, an odor associated with a reward can become attractive. By using Drosophila larvae as a model system with minimal brain complexity, we address the question of which neurons attribute these values to odor stimuli. In insects, dopaminergic neurons are required for aversive learning, whereas octopaminergic neurons are necessary and sufficient for appetitive learning. However, it remains unclear whether two independent neuronal populations are sufficient to mediate such antagonistic values. We report the use of transgenically expressed channelrhodopsin-2, a light-activated cation channel, as a tool for optophysiological stimulation of genetically defined neuronal populations in Drosophila larvae. We demonstrate that distinct neuronal populations can be activated simply by illuminating the animals with blue light. Light-induced activation of dopaminergic neurons paired with an odor stimulus induces aversive memory formation, whereas activation of octopaminergic/tyraminergic neurons induces appetitive memory formation. These findings demonstrate that antagonistic modulatory subsystems are sufficient to substitute for aversive and appetitive reinforcement during classical conditioning.     

Unit responses of the secondary somatosensory cortex during defensive conditioning in cats

Unit responses in the secondary somatosensory cortex during the formation and extinction of a defensive conditioned reflex to acoustic stimulation were investigated in chronic experiments on cats. In 21 of 28 neurons tested during defensive conditioning the firing pattern changed in accordance with the character of responses to electric shock reinforcement. Two types of conditioned-reflex unit responses were distinguished: excitatory and inhibitory. Most neurons responding to the conditioned stimulus by activation did so during the first 50 msec, which was 80–100 msec before the conditioned motor response. Considerable variability of the unit responses was observed during conditioning. By the time of stabilization of the conditioned-reflex connections the unit response to the conditioned stimulus was stable in form. The pattern of extinction of the conditioned unit activity was expressed as a decrease in the discharge frequency in responses of excitatory type and disinhibition of activity in the case of inhibitory responses.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev, Translated from Neirofiziologiya, Vol. 9, No. 3, pp. 232–238, May–June, 1977.     

Excitatory backward conditioning in an appetitive conditioned reinforcement preparation with rats

Four experiments were conducted to examine appetitive backward conditioning in a conditioned reinforcement preparation. In all experiments, off-line classical conditioning was conducted following lever-press training on two levers. Presentations of a sucrose solution by a liquid dipper served as an unconditioned stimulus (US) and two auditory stimuli served as conditioned stimuli (CSs); one was paired with the US in either a forward (Experiment 1a) or a backward (Experiments 1b, 2, and 3) relationship, and the other served as a control CS, which was not paired with the US. In testing, each lever-press response produced a presentation of one of the CSs instead of appetitive reinforcers. The response to a lever was facilitated, compared to the response to another lever, when the response produced the backward CS presentation as well as when it produced the forward CS presentation; that is, the backward CS served as an excitatory conditioned reinforcer.     

Dynamics of cortical unit responses during defensive conditioning to sound

Several phases were distinguished in single-unit responses in areas 3 and 4 during defensive conditioning to acoustic stimulation: an initial response, short inhibition of the spike discharge, early and late after-discharges, and changes arising after the end of acoustic stimulation. The initial spike response appeared or intensified (if present already) in the first period of defensive conditioning parallel with an increase in spontaneous unit activity. After-discharges appeared later. The conditioned-reflex movement usually began 100–400 msec after stimulation began. This latent period of the first movement was the same whether for a real conditioned reflex or an after-discharge. Comparison of the latent periods of conditioned movements with the phases of the unit responses showed that the conditioned responses of the cortical neuron were primarily modified after-discharges of neurons evoked by a conditioned stimulus. Differential unit responses to acoustic stimulation, also based on after-discharges, were formed just as actively as positive. The basic role of reinforcement during conditioning is not to increase the excitability of the neurons, which is important in connection with their acquisition of polysensory properties, but to modify the after-discharges of the neurons.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev. Translated from Neirofiziologiya, Vol. 10, No. 4, pp. 339–347, July–August, 1978.     

Increased regularity of activity of cortical neurons in learning due to disinhibitory effect of reinforcement

This paper reviews the author's studies on neurophysiologic mechanisms of conditioned reflex learning. Electroencephalograms, evoked potentials, activity of neocortical and hippocampal neurons and the rabbits' behavior in the course of elaboration of defensive and inhibitory conditioned reflexes to light flashes have been recorded. Electric shock (ECS) applied to the paw served as reinforcement. The study demonstrated three types of reinforcement effect on the activity of cortical neurons: activating, disinhibitory, and inhibitory. EEG activation due to reinforcement is accompanied by a change in phasic cortical neuronal activity from chaotic or irregular, typical of rest or inhibition, to regular tonic discharges (in neocortex and hippocampus) and group discharges in the stress rhythm, 5-7 Hz in the hippocampus. Following a number of conditioning trials, the effect of reinforcement is simulated by the effect of a conditioned stimulus. With EEG activation and increased regularity in impulses, facilitation of motor reactions is observed.     

Neuronal activity in the nucleus basalis magnocellularis during performance of an instrumental task

Spike activity of 95 neurons in the rabbit basal forebrain forebrain magnocellular nucleus was recorded during spontaneous behavior and instrumental conditioned performance. Almost half of the neurons (48.4%) displayed a significant (p > 0.05) negative correlation between their spontaneous discharge rate and the power of the delta rhythm in the EEG of the frontal cortex; most of these cells can be classified as projection cholinergic neurons. During instrumental conditioned performance, neurons of this subgroup responded with excitation to the conditioned stimulus, whereas presumably noncholinergic nonprojection neurons responded to the conditioned stimulus with inhibition. Excitatory response of cells in the basal forebrain magnocellular nucleus was significantly more intense as compared to trials without the conditioned reaction. On the whole, our data testify that the basal forebrain magnocellular nucleus maintains the level of arousal and attention required for the instrumental conditioned performance.     

Neuronal response in the cat parietal association cortex to conditioned and non-conditioned acoustic stimuli

Neuronal response in the cat association cortex (area 5) to conditioned and non-conditioned acoustic stimulation was investigated. Numbers of neurons responding to a conditioned acoustic stimulus according to the traditional reflex pattern were twice as high. Numbers of inhibitory neuronal responses to the stimulus increased when instrumental reflex occurred. Neurons were found which only reacted to a conditioned acoustic stimulus in the absence of conditioned reflex movement occurring with instrumental food reflex. Although findings do not exclude the possibility of this cortical area contributing to the analysis of sensory signals and evaluation of their biological significance, it might be supposed that its main functional property lies in its involvement in the process of initiating behavioral response to a conditioned response.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev. Translated from Neirofiziologiya, Vol. 20, No. 5, pp. 637–645, September–October, 1988.     

Somatosensory cortical unit responses during avoidance conditioning to sound

Unit responses of the primary somatosensory cortical-projection area were recorded in cats with established conditioned avoidance to sound during the conditioned response. Five types of changes in unit activity were distinguished during the conditioned reflex. The responses were 400–1000 msec in duration; their time of onset coincided with that of the expected reinforcing stimulus. As a rule the change in unit activity was preceded by changes in the electromyogram of the corresponding limb muscles. Among neurons responding to the conditioning stimulus, the proportion with a short latent period of response to electrical stimulation of the skin was less than during investigation of ordinary unit responses to the same stimulus.     

CS-US delay does not impair appetitive conditioning in Chasmagnathus

Habituation and appetitive conditioning have been already described in the crab Chasmagnathus. The purpose of this work is to study whether associative learning can be obtained despite a long conditioned stimulus-unconditioned stimulus interval. Results of the first experiment show that the weakening of temporal contiguity does not prevent appetitive conditioning to occur while after a long 4-h delay, conditioning wanes completely. A second experiment was conducted, after one and three days of training respectively, confirming the above results. Though initially neutral the context trace may be still available immediately after training and for the period of two but not after 4:00 h, demonstrating a forward limit for the conditioning window. After 3 days of training, a further decrease in the exploratory activity suggested that a longer training could increase the relative weight of habituation. Conditioning and habituation seem to work as opponent processes in the crab CHASMAGNATHUS GRANULATUS: if habituation training in the box is followed by the administration of reinforcement after a short period of time, appetitive conditioning will take place. However, as this interval is increased, habituation prevails. A persistent effect of the exposure to a given environment that may underlie trace conditioning in this crab is discussed in adaptive terms.     

Temporal difference models and reward-related learning in the human brain

Temporal difference learning has been proposed as a model for Pavlovian conditioning, in which an animal learns to predict delivery of reward following presentation of a conditioned stimulus (CS). A key component of this model is a prediction error signal, which, before learning, responds at the time of presentation of reward but, after learning, shifts its response to the time of onset of the CS. In order to test for regions manifesting this signal profile, subjects were scanned using event-related fMRI while undergoing appetitive conditioning with a pleasant taste reward. Regression analyses revealed that responses in ventral striatum and orbitofrontal cortex were significantly correlated with this error signal, suggesting that, during appetitive conditioning, computations described by temporal difference learning are expressed in the human brain.     

Reward,motivation, and reinforcement learning

There is substantial evidence that dopamine is involved in reward learning and appetitive conditioning. However, the major reinforcement learning-based theoretical models of classical conditioning (crudely, prediction learning) are actually based on rules designed to explain instrumental conditioning (action learning). Extensive anatomical, pharmacological, and psychological data, particularly concerning the impact of motivational manipulations, show that these models are unreasonable. We review the data and consider the involvement of a rich collection of different neural systems in various aspects of these forms of conditioning. Dopamine plays a pivotal, but complicated, role.     

Responses of motor cortex neurons of the cat to auditory stimulation and their role in the instrumental food reflex

The responses of motor cortex neurons in the cat to the presentation of a single auditory click and a series of 10 clicks presented with 1,000/sec frequency were studied under conditions of chronic experiments before and after the development of an instrumental food reflex. After reflex development a single presentation of a positive conditioned stimulus (single click) markedly influenced for 7 sec the appearance of instrumental movements. At the same time, the immediate responses of motor cortex neurons to presentation of the conditioned auditory stimulus had no impact on the appearance in the motor cortex of discharges leading to the realization of instrumental movements. Consequently, motor cortex neurons do not require activation from afferent sensory inputs for the generation of such discharges. The immediate neuronal responses to conditioned stimulation did not inhibit the realization of the instrumental reflex. It is proposed that they are associated with the realization of motor function in the unconditioned defensive response evoked by the presentation of an auditory stimulus. The presence or absence of responses to auditory conditioned stimulation was dependent upon the signal meaning of the stimulus, its physical parameters, and the degree of excitability of the animal.A. A. Bogomolets Institute of Physiology, Academy of Sciences of the Ukrainian SSR, Kiev. Translated from Neirofiziologiya, Vol. 17, No. 4, pp. 539–550, July–August, 1985.     

Discriminative behavior and Pavlovian conditioning in the mollusc Pleurobranchaea

The buccal motor system in the sea slug Pleurobranchaea californica is multifunctional; similar sets of neurons and muscles generate different behaviors through similar electrophysiological motor patterns. Such multifunctional systems compromise the traditional practice of identifying a motor pattern and then using that pattern to indicate the behavior in reduced preparations. We address this issue in a series of experiments leading to the comparison of differential Pavlovian conditioning in whole animals with the conditioned behavior of the same animals during electrophysiological recording. Because differential conditioning requires two conditioned stimuli (CSs), we show here that each of two CSs activated the conditioned response from animals after they received the stimulus (CS+) paired with an unconditioned stimulus (UCS). Conditioning sessions consisted of 5 training trials with a 2-h intertrial interval. In one study, experimental animals received a 60-s CS+, derived from beer (Sbr), paired with a 50-s electrical shock UCS whose onset occurred 10 s after the CS+ onset; control animals received the Sbr and UCS explicitly unpaired. In a second study, animals received similar procedures as in the first but with a CS+ consisting of squid homogenate (Ssq). Tests with both CSs showed that animals did not discriminate between Sbr and Ssq before beginning conditioning, but did so afterward. Experimental animals exhibited robust food aversion (withdrawal and suppressed feeding) to the CS+, but retained strong appetitive responses to the CS they did not receive in training; response thresholds to the CS+ changed as much as 1000-fold by comparison to the preconditioning values. Control animals exhibited similar though significantly smaller behavioral changes as the experimental animals. Both stimuli evoked associatively learned responses, but Sbr produced greater experimental-control differences than Ssq did. Two accompanying papers show the results of using both CSs in differential conditioning, and describe the behavioral/electrophysiological comparisons.     

The interrelations between neurons of the amygdala and hypothalamus during conditioning with selection of food reinforcement quality in cats

Three cats were subjected to appetitive instrumental conditioning to light by the method of the "active choice" of the reinforcement quality. The short-delayed conditioned bar-pressings were reinforced by bread-meat mixture and the delayed response by meat. The animals differed in behavior strategy: two animals preferred bar-pressing with long delay (the so-called "self-control" group) and one animal preferred bar-pressing with short delay (the so-called "impulsive" group). The multiunit activity of the basolateral amygdala and nucleus lateralis of the hypothalamus was recorded through chronically implanted nichrome wire semimicroelecrodes. The interactions between the neighboring neurons in the lateral hypothalamus and basolateral amygdala (within the local neuronal network) and between the neurons of the basolateral amygdala and lateral hypothalamus (distributed neuronal networks in the direction amygdala--hypothalamus and vice versa) were evaluated by means of statistical crosscorrelation analysis of spike trains. The crosscorrelational interneuronal connections in the delay range of 0-100 ms were examined. It was shown that the number of crosscorrelations between the discharges on neurons both in the local networks of basolateral amygdala and distributed networks was significantly higher in "impulsive" cats. In both groups of animals, the percentage of crosscorrelations between neighbouring neurons in the local networks of the lateral hypothalamus was similar. We suggest that the local networks of the basolateral amygdala and amygdalar-hypothalamic distributed neuronal networks are involved in the system of brain structures which determine the individual features of animal behavior.     

NMDA-dependent and NMDA-independent neuronal processes in the cat motor cortex,disinhibited by bicuculline,during forepaw placement conditioning

Neuronal activity associated with a conditioned forepaw placing reaction was recorded in the cat's motor cortex locally disinhibited by bicuculline spontaneously diffused from the recording pipette. Electrical stimulation of the parieral cortex (area 5) with 3-5 pulses was used as a conditioned stimulus. In both naive and trained cats, adding of APV (NMDA receptor blocker) led to disappearance of the late (30-120 ms) secondary excitatory responses from the pattern of the neuronal reaction to the parietal stimulation recorded in the motor cortex. At the same time, the APV administration did not change the excitatory reactions (recorded, predominantly, in the deep cortical layers) time-locked to the execution of the conditioned movement. The conditioning resulted in a statistically significant increase in the amplitude and duration of the late secondary responses as well as in a shortening of their latency. In some cases (after a long period of training), the late secondary responses to the conditioned stimulus transformed into paroxysmal epileptiform bursts. A hypothesis is discussed that the increase in synaptic strength of the backward horizontal collaterals of layer-II/III pyramidal neurons is responsible for the learning-related changes in the neuronal reactions in the disinhibited motor cortex.     

Genotypic Influence on Aversive Conditioning in Honeybees,Using a Novel Thermal Reinforcement Procedure

In Pavlovian conditioning, animals learn to associate initially neutral stimuli with positive or negative outcomes, leading to appetitive and aversive learning respectively. The honeybee (Apis mellifera) is a prominent invertebrate model for studying both versions of olfactory learning and for unraveling the influence of genotype. As a queen bee mates with about 15 males, her worker offspring belong to as many, genetically-different patrilines. While the genetic dependency of appetitive learning is well established in bees, it is not the case for aversive learning, as a robust protocol was only developed recently. In the original conditioning of the sting extension response (SER), bees learn to associate an odor (conditioned stimulus - CS) with an electric shock (unconditioned stimulus - US). This US is however not a natural stimulus for bees, which may represent a potential caveat for dissecting the genetics underlying aversive learning. We thus first tested heat as a potential new US for SER conditioning. We show that thermal stimulation of several sensory structures on the bee’s body triggers the SER, in a temperature-dependent manner. Moreover, heat applied to the antennae, mouthparts or legs is an efficient US for SER conditioning. Then, using microsatellite analysis, we analyzed heat sensitivity and aversive learning performances in ten worker patrilines issued from a naturally inseminated queen. We demonstrate a strong influence of genotype on aversive learning, possibly indicating the existence of a genetic determinism of this capacity. Such determinism could be instrumental for efficient task partitioning within the hive.     

Neuronal responses in area 7 of the cat cortex during defensive conditioning

Responses of neurons in area 7 of the parietal association cortex during and after formation of a defensive conditioned reflex to sound were recorded in waking cats. Changes in spike responses of the neurons as a result of the onset of conditioned reflex limb movements were observed in 68% of neurons. Spike responses of neurons formed as a result of learning appeared only if conditioned-reflex limb movements appeared, and they were not observed if, for some reason or other, movements were absent after presentation of the positive conditioned stimulus or on extinction of the reflex. Responses of 46% neurons to conditioned stimulation preceded the conditioned-reflex motor responses by 50–450 msec. The remaining responding neurons were recruited into the response after the beginning of movement. Characteristic spike responses of neurons to the conditioned stimulus appeared 500–900 msec before the beginning of movement and, in the case of appearance of special, "prolonged" motor responses of limb withdrawal, evoked by subsequent reinforcing stimulation.