Batesian, quasi-Batesian or Müllerian mimicry? Theory and data in mimicry Research

In this paper I argue that the nature of mimetic relationships remains contentious because there are insufficient data to enable full evaluation of theoretical models. There is, however, a growing appreciation of the need to draw together empirical studies to provide foundations for theoretical work. I review some recent data that considers the responses of predators to changing numbers of defended prey items and the nature of mimicry along a palatability spectrum. A simple model of predator behaviour is constructed which combines assumptions from Pavlovian learning studies with traditional ‘number dependent’ learning models. This model has two important properties. First it shows that Pavlovian assumptions can be represented in a simple model which generates interesting predictions. Second it indicates some areas that still need detailed empirical study – most importantly perhaps is the way that predators respond to prey with different levels of edibility. This revised version was published online in July 2006 with corrections to the Cover Date.     

Facets of Pavlovian and operant extinction

Research on extinction is of fundamental importance in both Pavlovian and operant approaches to the experimental analysis of learning. Although these approaches are often motivated by different empirical and theoretical questions, extinction has emerged as a research area in which common themes unite the two approaches. In this review, we focus on some common considerations in the analysis of Pavlovian and operant extinction. These include methodological challenges and interpretational issues in analyzing behavior during and after extinction. We consider the different roles that theory has played in the development of research on extinction in these preparations and conclude with some attention to applications of extinction.     

Assessing Pavlov's impact on the American conditioning enterprise

In this article, the visibility of Pavlov and of Watson in American psychology are compared, and the periods of their respective influence are specified with greater precision than is afforded by merely impressionistic methods. The author also critically examines the possibility that the early history of the American classic-conditioning enterprise involved a succession of two phases: a Watsonian/speculative phase and a Pavlovian/empirical phase. In conclusion, the author assesses the possibility that the publication of Pavlov's Conditioned Reflexes (1927) "stimulated" scholarly work on Pavlovian conditioning, and finds this proposition lacking empirical support.     

Pavlovian backward conditioned inhibition in humans: summation and retardation tests

Two experiments using human participants investigated whether a Pavlovian backward inhibitory treatment (nonreinforced trials in phase 1 followed by reinforced trials in phase 2; i.e., AX- followed by A+) produces a stimulus which can pass summation and retardation tests for inhibition. The rationale for conducting these experiments was that previous demonstrations of Pavlovian backward inhibition informed participants about the nature of the outcome before starting the experiment. According to some theoretical views, this is a potential confound. In the present experiments we used a predictive task in which participants had no knowledge about the outcome until phase 2, when reinforcement occurred. The results of Experiment 1 (summation test) and Experiment 2 (retardation test) provide a clear demonstration of backward conditioned inhibition.     

I. P. Pavlov and the development of neuroscience

Ivan Petrovitch Pavlov significantly changed and developed our knowledge of the brain functions and of the behaviour by his fundamental experimental and theoretical work on the physiology and pathophysiology of the higher nervous activity. He was one of the scientists who prepared the development of neuroscience in our century. During the Pavlovian Conference, 1950 in Moscow Stalin and the Communist Party tried to dogmatize his and his pupil's fundamental theories. But his pupils continued to develop Pavlovian ideas in open discussions with representatives of other schools in a very creative way, opening the doors for a system approach to understand the integrative functional systems of brain and behavior. Pavlov emphasized the high plasticity of the central nervous system. He investigated the complex functional systems within the brain and between the organism and its environment, and the designed models for pathology of the higher nervous activity. During his last years, Pavlov freed himself from the strong deterministic view and characterized the organism and its environment as a self-organizing system.     

Determinants of cue interactions

In a Pavlovian conditioning situation, there are many training variables that may affect responding to a conditioned stimulus (CS), such as contiguity, contingency, and the presence of other CSs. This review describes recent experiments that show that some manipulations that usually decrease responding to a CS may have the opposite effect when they are combined with other normally pernicious manipulations. A theoretical framework that explains these so-called counteraction effects is provided. The apparent boundary conditions on the effects and limitations of the theory are discussed.     

An experimental psychophysiological approach to human bradycardiac reflexes

Bradycardic reflexes in man are both of scientific and clinical interest. Using the methods of experimental psychophysiology, control over relevant independent variables permits the study of fine-grained temporal physiologic response topographies, and of psychological factors that may modify the reflex. In addition, information can also be sought through interdisciplinary collaborations with experimental physiologists in order to shed light on the mechanism of the reflexes. These general features of the approach are illustrated by presenting data on two bradycardic reflex preparations: the laboratory dive analog, and the 90-degree negative tilt. The dive-analog studies have shown that a) the dive-reflex proper is a late-occurring bradycardia accompanied by a late-occurring vasoconstriction; and b) for the elicitation of this reflex, both breath-holding and face immersion are necessary. In addition, the physiologic manipulation of temperature affects the reflex in an inverse way over the range of 10 degrees to 40 degrees C, while the sense of control (a psychological variable) attenuates the reflex. The negative-tilt preparation produces a bradycardic response that is ideal as a Pavlovian unconditional response. Some Pavlovian conditioning arrangements, especially an "imaginational" form, do produce significant conditional bradycardic responding, and this has both potential clinical (e.g., biofeedback-related) and theoretical (e.g., S-R vs. S-S accounts of Pavlovian conditioning) applications. The paper ends with a comment on the cognitive paradigm shift in psychology. Although this shift is of importance, it is suggested that it is also important to "remember the response."     

Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding

Hard-wired, Pavlovian, responses elicited by predictions of rewards and punishments exert significant benevolent and malevolent influences over instrumentally-appropriate actions. These influences come in two main groups, defined along anatomical, pharmacological, behavioural and functional lines. Investigations of the influences have so far concentrated on the groups as a whole; here we take the critical step of looking inside each group, using a detailed reinforcement learning model to distinguish effects to do with value, specific actions, and general activation or inhibition. We show a high degree of sophistication in Pavlovian influences, with appetitive Pavlovian stimuli specifically promoting approach and inhibiting withdrawal, and aversive Pavlovian stimuli promoting withdrawal and inhibiting approach. These influences account for differences in the instrumental performance of approach and withdrawal behaviours. Finally, although losses are as informative as gains, we find that subjects neglect losses in their instrumental learning. Our findings argue for a view of the Pavlovian system as a constraint or prior, facilitating learning by alleviating computational costs that come with increased flexibility.     

Neural signatures of arbitration between Pavlovian and instrumental action selection

Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations “misbehave” when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability.     

A comparison of the effects of discriminative and Pavlovian inhibitors and excitors on instrumental responding

In two experiments, the effects of Pavlovian or discriminative conditioned inhibitors on operant responding were investigated in rats. Experiment 1 found that a Pavlovian conditioned inhibitor for food suppressed food-reinforced lever pressing more than a non-differentially trained control stimulus did. Experiment 2 demonstrated that an operant discriminative inhibitor produced greater suppression of lever pressing than a Pavlovian conditioned inhibitor. Experiment 2 also found that compounding an operant discriminative stimulus (SD) for food-reinforced responding with another SD for food-reinforced responding resulted in more additive summation than when an SD was compounded with a Pavlovian conditioned excitor for food. The results of these experiments support two-factor theories that postulate that incentive and response discriminative processes summate algebraically when the processes are inhibitory or excitatory.     

A comparative analysis of the conditional reflex discoveries of Pavlov and Twitmyer, and the birth of a paradigm

Research on conditional reflex (CR) in Pavlov's Physiological Laboratory has preceded Twitmyer's work on conditioning at the University of Pennsylvania by 3 or 4 years. The events in Pavlov's laboratory lead toward the postulation of a new paradigm that rejected the Cartesians conceptualization of the reflex as a mechanistic response to stimuli by replacing it with the Darwinian notion of the organism's adaptation to the environmental conditions. The Pavlovian paradigm rejected the Wundtian method in favor of the objective, conditional reflex method.     

Synaptic mechanisms of associative memory in the amygdala

Do associative learning and synaptic long-term potentiation (LTP) depend on the same cellular mechanisms? Recent work in the amygdala reveals that LTP and Pavlovian fear conditioning induce similar changes in postsynaptic AMPA-type glutamate receptors and that occluding these changes by viral-mediated overexpression of a dominant-negative GluR1 construct attenuates both LTP and fear memory in rats. Novel forms of presynaptic plasticity in the lateral nucleus may also contribute to fear memory formation, bolstering the connection between synaptic plasticity mechanisms and associative learning and memory.     

Relapse processes after the extinction of instrumental learning: renewal, resurgence, and reacquisition

It is widely recognized that extinction (the procedure in which a Pavlovian conditioned stimulus or an instrumental action is repeatedly presented without its reinforcer) weakens behavior without erasing the original learning. Most of the experiments that support this claim have focused on several "relapse" effects that occur after Pavlovian extinction, which collectively suggest that the original learning is saved through extinction. However, although such effects do occur after instrumental extinction, they have not been explored there in as much detail. This article reviews recent research in our laboratory that has investigated three relapse effects that occur after the extinction of instrumental (operant) learning. In renewal, responding returns after extinction when the behavior is tested in a different context; in resurgence, responding recovers when a second response that has been reinforced during extinction of the first is itself put on extinction; and in rapid reacquisition, extinguished responding returns rapidly when the response is reinforced again. The results provide new insights into extinction and relapse, and are consistent with principles that have been developed to explain extinction and relapse as they occur after Pavlovian conditioning. Extinction of instrumental learning, like Pavlovian learning, involves new learning that is relatively dependent on the context for expression.     

Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients

The negative symptoms of schizophrenia (SZ) are associated with a pattern of reinforcement learning (RL) deficits likely related to degraded representations of reward values. However, the RL tasks used to date have required active responses to both reward and punishing stimuli. Pavlovian biases have been shown to affect performance on these tasks through invigoration of action to reward and inhibition of action to punishment, and may be partially responsible for the effects found in patients. Forty-five patients with schizophrenia and 30 demographically-matched controls completed a four-stimulus reinforcement learning task that crossed action (“Go” or “NoGo”) and the valence of the optimal outcome (reward or punishment-avoidance), such that all combinations of action and outcome valence were tested. Behaviour was modelled using a six-parameter RL model and EEG was simultaneously recorded. Patients demonstrated a reduction in Pavlovian performance bias that was evident in a reduced Go bias across the full group. In a subset of patients administered clozapine, the reduction in Pavlovian bias was enhanced. The reduction in Pavlovian bias in SZ patients was accompanied by feedback processing differences at the time of the P3a component. The reduced Pavlovian bias in patients is suggested to be due to reduced fidelity in the communication between striatal regions and frontal cortex. It may also partially account for previous findings of poorer “Go-learning” in schizophrenia where “Go” responses or Pavlovian consistent responses are required for optimal performance. An attenuated P3a component dynamic in patients is consistent with a view that deficits in operant learning are due to impairments in adaptively using feedback to update representations of stimulus value.     

The Emotional Brain, Fear, and the Amygdala

1. Considerable progress has been made over the past 20 years in relating specific circuits of the brain to emotional functions. Much of this work has involved studies of Pavlovian or classical fear conditioning, a behavioral procedure that is used to couple meaningless environmental stimuli to emotional (defense) response networks.2. The major conclusion from studies of fear conditioning is that the amygdala plays critical role in linking external stimuli to defense responses.3. Before describing research on the role of the amygdala in fear conditioning, though, it will be helpful to briefly examine the historical events that preceded modern research on conditioned fear.     

Quantifying individual variation in the propensity to attribute incentive salience to reward cues

If reward-associated cues acquire the properties of incentive stimuli they can come to powerfully control behavior, and potentially promote maladaptive behavior. Pavlovian incentive stimuli are defined as stimuli that have three fundamental properties: they are attractive, they are themselves desired, and they can spur instrumental actions. We have found, however, that there is considerable individual variation in the extent to which animals attribute Pavlovian incentive motivational properties ("incentive salience") to reward cues. The purpose of this paper was to develop criteria for identifying and classifying individuals based on their propensity to attribute incentive salience to reward cues. To do this, we conducted a meta-analysis of a large sample of rats (N = 1,878) subjected to a classic Pavlovian conditioning procedure. We then used the propensity of animals to approach a cue predictive of reward (one index of the extent to which the cue was attributed with incentive salience), to characterize two behavioral phenotypes in this population: animals that approached the cue ("sign-trackers") vs. others that approached the location of reward delivery ("goal-trackers"). This variation in Pavlovian approach behavior predicted other behavioral indices of the propensity to attribute incentive salience to reward cues. Thus, the procedures reported here should be useful for making comparisons across studies and for assessing individual variation in incentive salience attribution in small samples of the population, or even for classifying single animals.     

Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees

When planning a series of actions, it is usually infeasible to consider all potential future sequences; instead, one must prune the decision tree. Provably optimal pruning is, however, still computationally ruinous and the specific approximations humans employ remain unknown. We designed a new sequential reinforcement-based task and showed that human subjects adopted a simple pruning strategy: during mental evaluation of a sequence of choices, they curtailed any further evaluation of a sequence as soon as they encountered a large loss. This pruning strategy was Pavlovian: it was reflexively evoked by large losses and persisted even when overwhelmingly counterproductive. It was also evident above and beyond loss aversion. We found that the tendency towards Pavlovian pruning was selectively predicted by the degree to which subjects exhibited sub-clinical mood disturbance, in accordance with theories that ascribe Pavlovian behavioural inhibition, via serotonin, a role in mood disorders. We conclude that Pavlovian behavioural inhibition shapes highly flexible, goal-directed choices in a manner that may be important for theories of decision-making in mood disorders.     

The discovery of the principles of reinforcement, extinction, generalization, and differentiation of conditional reflexes in Pavlov's laboratories

The discovery of reinforcement, extinction, generalization, and differentiation with the conditional reflex method in Pavlov's laboratories is described. Modern American introductory texts show that contemporary understanding of the experimental work on conditioning in Pavlov's laboratories is derived from a 1927 English translation of Pavlov's lectures on the conditional reflexes. The lectures present the discoveries topically, not chronologically. In contrast, this article presents a chronological account of the contributions of S.G. Vul'fson, I.F. Tolochinov, and B.P. Babkin, which led to the conceptualization of reinforcement and extinction, and the work of V.N. Boldyrev and N.A. Kashereninova, which led to the formulation of the concepts of generalization and differentiation. This historical approach avoids giving the impression that the development of the Pavlovian paradigm was a highly systematic pursuit.     

Exposure to cold: aversive Pavlovian conditioning in individual Drosophila melanogaster

Abstract. Temperature changes can be especially threatening for ectotherms, such as Drosophila melanogaster (Diptera: Drosophilidea Meigen, 1830 ), and in this study we tested whether flies can associate olfactory stimuli with a sudden drop in temperature. Such Pavlovian conditioning would allow them to make appropriate behavioural and/or physiological responses in the future. We found that exposing individual flies to one of two odours in the presence of a sudden drop in temperature resulted in Pavlovian conditioning with flies subsequently avoiding the odour paired with cold. The characteristics of Pavlovian conditioning in flies were comparable to those observed for mammalian species. Specifically, the strength of conditioning increased with increasing intensity of the cold and decreased as the time interval between the olfactory stimulus (CS) and cold (US) was lengthened. Finally, the order in which CS and US were presented affected the strength of conditioning. Learning was observed when the CS preceded US and when the US immediately preceded the CS, but not when the CS preceded the US by 30 s or more. These results provide further evidence for learning in individual flies, and confirm that Pavlovian conditioning is a general mechanism used by organisms to obtain information about their environment.