Facets of Pavlovian and operant extinction

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.     

奖励性操作式条件反射任务在大鼠学习记忆研究中的应用

目的将奖励性操作式条件反射方法应用于学习记忆研究。方法首次采用奖励性操作式条件反射方法,设置单次操作训练、连续多次操作训练、信号辨识与消退四种检测模式,研究Wistar大鼠和SHR大鼠学习记忆能力。结果奖赏训练中,SHR组鼻触次数显著增多（vs.Wistar＆Donepezil,P〈0.05）;单次操作训练中,三组动物对操作反应的获得无显著差异;连续多次操作学习任务中,SHR组错误操作次数增多（vs.Wistar,P〈0.05）,奖赏获得次数减少（vs.Donepezil,P〈0.001）,反应准确率显著降低（vs.Wistar＆Donepezil,P〈0.05）;信号辨识任务中SHR组错误反应次数显著增多（vs.Wistar＆Donepezil,P〈0.05）,而反应准确率降低,组间差异有显著性（vs.Wistar＆Donepezil,P〈0.05）;消退实验中,三组动物差异无显著性。神经递质检测发现,SHR组大鼠谷氨酸[（1.0639±0.07086）mg/g]、乙酰胆碱[（2.7760±0.2609）μg/g]与五羟色胺[（1.2200±0.1137）μg/g]含量均显著低于Wistar组大鼠（P〈0.05）,多奈哌齐组大鼠乙酰胆碱含量显著增加[（3.9344±0.2747）μg/g]（vs.Wistar,P〈0.05;vs.SHR,P〈0.001）。结论奖励性操作式条件反射方法可作为一种正性增强条件反射检测新方法应用于大鼠学习记忆研究。     

Operant conditioning of antennal muscle activity in the honey bee (Apis mellifera L.)

Antennal movements of the honey bee can be conditioned operantly under laboratory conditions. Using this behavioural paradigm we have developed a preparation in which the activity of a single antennal muscle has been operantly conditioned. This muscle, the fast flagellum flexor muscle, is innervated by an identified motoneuron whose action potentials correlate 1:1 with the muscle potentials. The activity of the fast flagellum flexor muscle was recorded extracellularly from the scapus of the antenna. The animal was rewarded with a drop of sucrose solution whenever the muscle activity exceeded a defined reward threshold. The reward threshold was one standard deviation above the mean spontaneous frequency prior to conditioning. After ten conditioning trials, the frequency of the muscle potentials had increased significantly compared to the spontaneous frequency. The conditioned changes of frequency were observed for 30 min after conditioning. No significant changes of the frequency were found in the yoke control group. The firing pattern of the muscle potentials did not change significantly after conditioning or feeding. Fixing the antennal joints reduces or abolishes associative operant conditioning. The conditioned changes of the frequency of muscle potentials in the freely moving antenna are directly comparable to the behavioural changes during operant conditioning. Accepted: 29 March 2000     

Operant conditioning of scratching in lemurs

An adult femaleLemur catta and an adult femaleEulemur fulvus were given edible rewards for scratching. Both subjects learned to scratch in order to obtain the rewards, showed diminished rates of scratching during periods of extinction, and learned to scratch preferentially with one foot when required. TheLemur catta subject was more responsive to the changing experimental conditions than theEulemur fulvus. The conditionability of scratching in primates does not appear to be directly related to the widespread occurrence of scratching in simian social contexts.     

Spontaneous decisions and operant conditioning in fruit flies

Already in the 1930s Skinner, Konorskiand colleagues debated the commonalities, differences and interactions among the processes underlying what was then known as “conditioned reflexes type I and II”, but which is today more well-known as classical (Pavlovian) and operant (instrumental) conditioning. Subsequent decades of research have confirmed that the interactions between the various learning systems engaged during operant conditioning are complex and difficult to disentangle. Today, modern neurobiological tools allow us to dissect the biological processes underlying operant conditioning and study their interactions. These processes include initiating spontaneous behavioral variability, world-learning and self-learning. The data suggest that behavioral variability is generated actively by the brain, rather than as a by-product of a complex, noisy input-output system. The function of this variability, in part, is to detect how the environment responds to such actions. World-learning denotes the biological process by which value is assigned to environmental stimuli. Self-learning is the biological process which assigns value to a specific action or movement. In an operant learning situation using visual stimuli for flies, world-learning inhibits self-learning via a prominent neuropil region, the mushroom-bodies. Only extended training can overcome this inhibition and lead to habit formation by engaging the self-learning mechanism. Self-learning transforms spontaneous, flexible actions into stereotyped, habitual responses.     

Directionality of antennal sweeps elicited by water jet stimulation of the tailfan in the crayfish Procambarus clarkii

Summary Directionality and intensity dependence of antennal sweeps elicited by water jet stimulation of the tailfan in tethered, reversibly blinded adult and juvenile crayfish (Procambarus clarkii) were analyzed.Resting crayfish keep their antennae at about 50° symmetrically to the longitudinal body axis (Figs. 2 bottom, and 3).In adults, tailfan stimulation elicits synchronous backward sweeps of both antennae, which increase for more caudal stimulus directions (Figs. 2–4 and 5A). Directions differing by 30°–60° are significantly distinguished (Fig. 4). The mean sweep of the ipsilateral antenna significantly overrides that of the contralateral antenna for rostrolateral stimulation at 40–200 mm/s stimulus velocity and lateral to caudolateral stimulation at 40 mm/s and thus lateralization of the stimulus is revealed (Figs. 2 top, 4 and 5A). Mean antennal sweeps at a given stimulus direction and distance increase with increasing stimulus velocity (40–250 mm/s, Fig. 5A).In juveniles, the directional dependence of antennal sweeps is reduced compared to that of adults, while a similar intensity dependence is found (Fig. 5B).The pronounced directionality of the antennal response in adult crayfish vanishes and response latencies increase after reversibly covering the tailfan with a small bag or the telson with waterproof paste (Figs. 6 and 7). Thus, tailfan and especially telson mechanoreceptors play an important role in the localization of water movements elicited by predators or prey behind the crayfish.     

The functions of antennal mechanoreceptors and antennal joints in tactile discrimination of the honeybee (<Emphasis Type="Italic">Apis mellifera</Emphasis> L.)

Honeybees learn and discriminate excellently between different surface structures and different forms of objects, which they scan with their antennae. The sensory plate on the antennal tip plays a key role in the perception of mechanosensory and gustatory information. It is densely covered with small tactile hairs and carries a few large taste hairs. Both types of sensilla contain a mechanoreceptor, which is involved in the antennal scanning of an object. Our experiments test the roles of the mechanoreceptors on the antennal tip in tactile antennal learning and discrimination. Joints between head capsule and scapus and between scapus and pedicellus enable the bee to perform three-dimensional movements when they scan an object. The role of these joints in tactile antennal learning and discrimination is studied in separate experiments. The mechanoreceptors on the antennal tip were decisive for surface discrimination, but not for tactile acquisition or discrimination of shapes. When the scapus–pedicellus joint or the headcapsule–scapus joint was fixed on both antennae, tactile learning was still apparent but surface discrimination was abolished. Fixing both scapi to the head capsule reduced tactile acquisition.     

Central projections of olfactory receptor neurons from single antennal and palpal sensilla in mosquitoes

In insects, olfactory receptor neurons (ORNs) are located in cuticular sensilla, that are present on the antennae and on the maxillary palps. Their axons project into spherical neuropil, the glomeruli, which are characteristic structures in the primary olfactory center throughout the animal kingdom. ORNs in insects often respond specifically to single odor compounds. The projection patterns of these neurons within the primary olfactory center, the antennal lobe, are, however, largely unknown.We developed a method to stain central projections of intact receptor neurons known to respond to host odor compounds in the malaria mosquito, Anopheles gambiae. Terminal arborizations from ORNs from antennal sensilla had only a few branches apparently restricted to a single glomerulus. Axonal arborizations of the different neurons originating from the same sensillum did not overlap.ORNs originating from maxillary palp sensilla all projected into a dorso-medial area in both the ipsi- and contralateral antennal lobe, which received in no case axon terminals from antennal receptor neurons. Staining of maxillary palp receptor neurons in a second mosquito species (Aedes aegypti) revealed unilateral arborizations in an area at a similar position as in An. gambiae.     

Maturation of odor representation in the honeybee antennal lobe

The antennal lobe (AL) is the first center for processing odors in the insect brain, as is the olfactory bulb (OB) in vertebrates. Both the AL and the OB have a characteristic glomerular structure; odors sensed by olfactory receptor neurons are represented by patterns of glomerular activity. Little is known about when and how an odor begins to be perceived in a developing brain. We address this question by using calcium imaging to monitor odor-evoked neural activity in the ALs of bees of different ages. We find that odor-evoked neural activity already occurs in the ALs of bees as young as 1 or 2 days. In young bees, the responses to odors are relatively weak and restricted to a small number of glomeruli. However, different odors already evoke responses in different combinations of glomeruli. In mature bees, the responses are stronger and are evident in more glomeruli, but continue to have distinct odor-dependent signatures. Our findings indicate that the specific glomerular patterns for odors are conserved during the development, and that odor representations are fully developed in the AL during the first 2 weeks following emergence.     

Clicker increases resistance to extinction but does not decrease training time of a simple operant task in domestic dogs (Canis familiaris)

Despite its popularity among pet owners and professional trainers, we are not aware of any studies that have investigated the efficacy of clicker training in canines. To this end, we taught 35 basenjis to nose-touch an orange traffic cone. Upon meeting pre-determined criteria, dogs progressed through: (1) training trials, wherein correct responses were followed immediately with a click plus food (clicker group) or food alone (control group); (2) strengthening trials, wherein dogs received the same reinforcement protocol as in training trials, except nose-touching behaviour was variably reinforced; and (3) extinction trials, wherein food was withheld from both groups, but dogs in the clicker group received a click alone for nose-touches. We found that the clicker and control groups did not differ with regard to the number of trials or the time required to meet training or strengthening criteria (P > 0.05 for all). However, the clicker group required significantly more trials (log₁₀ transformed means ± S.E. = 1.6 ± 0.03 trials versus 1.4 ± 0.03 trials, P < 0.001) and more time (log₁₀ transformed means ± S.E. = 2.85 ± 0.03 s versus 2.73 ± 0.03 s, P = 0.008) to reach extinction criterion. Additionally, younger dogs required fewer training (, P = 0.001) and strengthening (, P = 0.029) trials and less training (, P = 0.005) and strengthening (, P = 0.013) time to meet criteria than did older dogs. However, no age effect was found on extinction for either the number or duration of trials (P > 0.05 for both), implying that persistence in previously reinforced behaviour did not influence the age sensitivity found in task acquisition. Overall, these results suggest that, whereas the clicker may prolong behaviour without primary reinforcement, it does not reduce the training time of a simple operant task in dogs when primary reinforcement is briefly delayed. We speculate that the clicker may be most useful in maintaining established behaviours when primary reinforcement is unavailable or when its delivery is impractical. Additionally, we found that basenji dogs may become progressively impaired with age in the acquisition of stimulus-reward contingencies.     

Operant conditioning of rat navigation using electrical stimulation for directional cues and rewards

Operant conditioning is often used to train a desired behavior in an animal. The contingency between a specific behavior and a reward is required for successful training. Here, we compared the effectiveness of two different mazes for training turning behaviors in response to directional cues in Sprague-Dawley rats. Forty-three rats were implanted with electrodes into the medial forebrain bundle and the left and right somatosensory cortices for reward and cues. Among them, thirteen rats discriminated between the left and right somatosensory stimulations to obtain rewards. They were trained to learn ipsilateral turning response to the stimulation of the left or right somatosensory cortex in either the T-maze (Group T) or the maze (Group W). Performance was measured by the navigation speed in the mazes. Performances of rats in Group T were enhanced faster than those in Group W. A significant correlation between performances during training and performance in final testing was observed in Group T starting with the fifth training session while such a correlation was not observed in Group W until the tenth training session. The training mazes did not however affect the performances in the final test. These results suggest that a simple maze is better than a complicated maze for training animals to learn directions and direct cortical stimulation can be used as a cue for direction training.     

Serotonin-immunoreactive neurons in the antennal lobes and suboesophageal ganglion of the honeybee

Summary We have used immunohistochemical methods to investigate the morphology of identified, presumptive serotonergic neurons in the antennal lobes and suboesophageal ganglion of the worker honeybee. A large interneuron (deutocerebral giant, DCG) is described that interconnects the deutocerebral antennal and dorsal lobes with the suboesophageal ganglion and descends into the ventral nerve chord. This neuron is accompanied by a second serotonin-immunoreactive interneuron with projections into the protocerebrum. Two pairs of bilateral immunoreactive serial homologues were identified in each of the three suboesophageal neuromeres and were also found in the thoracic ganglia. With the exception of the frontal commissure, no immunoreactive processes could be found in the peripheral nerves of the brain and the suboesophageal ganglion. The morphological studies on the serial homologues were extended by intracellular injections of Lucifer Yellow combined with immunofluorescence.     

A monoamine in the gustatory cell of the frog's taste organ

Summary Fluorescence histochemistry reveals that in the frog's taste organ a yellow fluorescence is regularly observed at the most basal region of the sensory epithelium. The fluorescence has a strong intensity, but it fades rapidly upon the UV-irradiation. The peak of the emission spectrum is at 520 m. Following reserpine treatment the yellow fluorescence is markedly reduced, but not depleted completely. From these characteristics the monoamine fluorescence is regarded as representing 5-HT (serotonin).The ultrastructural study on sensory epithelia shows that the terminal portions of gustatory cell processes are localized at the basal region. These portions are filled with dense cored vesicles (700–1000 Å in diameter) and frequently opposed with nerve fibers penetrating into the epithelium. The gustatory cell processes are also interposed between the terminal portions or nerve fibers. The cytoplasm of the gustatory cell process is characterized by many mitochondria, fine filaments and glycogen particles, but contains few cored vesicles. The distribution of terminal portions of gustatory cell processes seems to correspond fairly well to that of the monoamine fluorescence observed discontinuously along the basal lamina. Accordingly it is concluded that the fluorigenic monoamine is localized in the cored vesicles of the gustatory cell.These results were reported in a preliminary form to the October, 1974 meeting of the Japan Society of Histochemistry and Cytochemistry.The authors gratefully acknowledge the support and helpful advice of Prof. Dr. T. Kanaseki.     

Effect of fipronil on side-specific antennal tactile learning in the honeybee

In the honeybee, the conditioning of the proboscis extension response using tactile antennal stimulations is well suited for studying the side-specificity of learning including the possible bilateral transfer of memory traces in the brain, and the role of inhibitory networks. A tactile stimulus was presented to one antenna in association with a sucrose reward to the proboscis. The other antenna was either not stimulated (A+/0 training), stimulated with a non-reinforced tactile stimulus B (A+/B− training) or stimulated with B reinforced with sucrose to the proboscis (A+/B+ training). Memory tests performed 3 and 24 h after training showed in all situations that a tactile stimulus learnt on one side was only retrieved ipsilaterally, indicating no bilateral transfer of information. In all these groups, we investigated the effect of the phenylpyrazole insecticide fipronil by applying a sublethal dose (0.5 ng/bee) on the thorax 15 min before training. This treatment decreased acquisition success and the subsequent memory performances were lowered but the distribution of responses to the tactile stimuli between sides was not affected. These results underline the role of the inhibitory networks targeted by fipronil on tactile learning and memory processes.     

Sugar cell responses to lactose and sucrose in labellar and tarsal taste hairs of Musca domestica

Receptor cell responses in the largest labellar (LL) and tarsal (D) taste hairs of the housefly Musca domestica were investigated electrophysiologically using the tip-recording technique. In LL hairs, test series with lactose in concentrations of 12.5–400 mmol · l⁻¹ yielded a threshold concentration around 12 mmol · l⁻¹ and a calculated concentration eliciting half-maximal response of around 40 mmol · l⁻¹, the maximal response varying between 18 and 30 impulses/300 ms. D hairs are more sensitive towards lactose, indicated by a slightly lower threshold and a by 60% higher response to 400 mmol · l⁻¹ lactose. The high variation in the relative stimulating effectiveness of lactose and sucrose and experiments with sugar mixtures imply that these sugars bind to different receptor sites without noticeable cross affinity. A comparison of the concentration response characteristics for sucrose and lactose in LL and D hairs suggests that sucrose can combine with more than one site type, expressed in different proportions in both hair types. Results obtained with p-nitrophenyl-β-galactoside as stimulus indicate that a β-galactoside link is not sufficient for a substance to interact specifically with the lactose binding site. The exceptional lactose sensitivity of the sugar cell in M. domestica is discussed in the context of food acquirement and digestion. Accepted: 14 November 1997     

The temporal pattern of responding in conditioned bar-press suppression: the role of the context switch and training mode

The present study examined the temporal pattern of responding in a conditioned bar-press suppression task in rats. Rats were exposed to either a 30-s or a 120-s conditioned stimulus (CS) followed by a footshock. Training took place either while the rats were lever-pressing for water (online), or with the lever removed from the box (offline). They were then exposed to the CS while they were lever-pressing for water, either in the training context or in a different context. Bar-press suppression during the CS was constant across the duration of the CS during training, but was restricted to the initial portion of the CS at the time of testing, especially when subjects were tested in a different context. Those results replicate the reactive (as opposed to anticipatory) pattern observed in a lick suppression procedure by Jozefowiez et al. (2011) and indicate that a change in context at the time of testing might be critical for its expression.     

Chemosensory control of pollen ingestion in the hoverfly Eristalis tenax by labellar taste hairs

The labellar gustatory system of the dronefly Eristalis tenax L. (Syrphidae; Diptera) that enables the fly to discriminate between pollen and nectar is investigated, and the triggering of pollen ingestion is examined. In behavioural preference tests, exhaustively extracted pollen of the sunflower Helianthus annuus is consumed in smaller amounts than untreated pollen, indicating that water-soluble substances are important for acceptance. Dry pollen is preferred over moist pollen in which the grains stick together, suggesting that mechanical properties of the pollen also play a role in its sensory assessment. Electrophysiological studies of the labellar taste hairs reveal that aqueous extracts of pollen (2% w/v) stimulate the salt receptor cell, but not the sugar receptor cell. The response of the water receptor cell remains the same as to pure water (or standard electrolyte, 10 mmol · l⁻¹ KCl). Of the 20 amino acids tested, the salt cell is sensitive only to proline in a submillimolar range. Behavioural experiments support the electrophysiological findings. When KCl is applied at concentrations eliciting salt-cell spike frequencies equal to those produced by pollen extract (which is often accepted), the water receptor cell is inhibited and a pronounced rejection behaviour occurs. This rejection of concentrated salt solution in Eristalis is therefore mainly mediated by the inhibition of the water cell. Accepted: 27 November 1999     

Distribution and specific central projections of mechanoreceptors in the thorax and proximal leg joints of locusts

Summary Tactile hairs on the locust thorax can be divided into two classes by their external morphology and their central projection pattern: Short hairs, 10–100 m in length, which are assembled in distinct plates and rows, and long hairs, 100–800 m in length, which are distributed all over the body and are organized in large fields or aligned along the ridges of the appendages.The sensory fibers of the first class arborize in the lateral dorsal neuropile of thoracic ganglia and then extend further into the ipsilateral half of the corresponding ganglion in three main bundles from which fine rami of fibers end in the intermediate neuropile. In all three thoracic ganglia the projection pattern of homologous hair plates is similar.The sensory fibers of the second class exclusively terminate in special median ventral neuropiles, the ventral association center (VAC) and ventralmost ventral association center (VVAC). In addition fibers from meso- and metathoracic hairs, located close to the longitudinal midline of the animal, may terminate in the contralateral VAC and with one branch project to the next anterior ganglion through the ipsilateral connective. In contrast, fibers from prothoracic hairs were not found to leave their ganglion.With support by the DFG Neurale Mechanismen des VerhaltensSome of the studies were started at Universität Bielefeld, Fakultät für Biologie II (Abtlg. Prof. Dr. P. Görner)