Contingency discriminability and the generalized matching law describe choice on concurrent ratio schedules of wheel-running reinforcement

Belke (2010) showed that on concurrent ratio schedules, the difference in ratio requirements required to produce near exclusive preference for the lower ratio alternative was substantively greater when the reinforcer was wheel running than when it was sucrose. The current study replicated this finding and showed that this choice behavior can be described by the matching law and the contingency discriminability model. Eight female Long Evans rats were exposed to concurrent VR schedules of wheel-running reinforcement (30s) and the schedule value of the initially preferred alternative was systematically increased. Two rats rapidly developed exclusive preference for the lower ratio alternative, but the majority did not - even when ratios differed by 20:1. Analysis showed that estimates of slopes from the matching law and the proportion of reinforcers misattributed from the contingency discriminability model were related to the ratios at which near exclusive preference developed. The fit of these models would be consistent with misattribution of reinforcers or poor discrimination between alternatives due to the long duration of wheel running.     

The stay/switch model describes choice among magnitudes of reinforcers

The stay/switch model is an alternative to the generalized matching law for describing choice in concurrent procedures. The purpose of the present experiment was to extend this model to choice among magnitudes of reinforcers. Rats were exposed to conditions in which the magnitude of reinforcers (number of food pellets) varied for staying at alternative 1, switching from alternative 1, staying at alternative 2 and switching from alternative 2. A changeover delay was not used. The results showed that the stay/switch model provided a good account of the data overall, and deviations from fits of the generalized matching law to response allocation data were in the direction predicted by the stay/switch model. In addition, comparisons among specific conditions suggested that varying the ratio of obtained reinforcers, as in the generalized matching law, was not necessary to change the response and time allocations. Other comparisons suggested that varying the ratio of obtained reinforcers was not sufficient to change response allocation. Taken together these results provide additional support for the stay/switch model of concurrent choice.     

Rapid acquisition in concurrent chains: effects of initial-link duration

Pigeons were trained in a concurrent chains procedure in which the terminal-link schedules in each session were either fixed-interval (FI) 10s FI 20s or FI 20s FI 10s, as determined by a pseudorandom binary series. The initial-link was a variable-interval (VI) 10-s schedule. Training continued until initial-link response allocation stabilized about midway through each session and was sensitive to the terminal-link immediacy ratio in that session. The initial-link schedule was then varied across sessions between VI 0.01 s and VI 30s according to an ascending and descending sequence. Initial-link response allocation was a bitonic function over the full range of durations. Preference for the FI 10-s terminal-link at first increased as programmed initial-link duration varied from 0.01 to 7.5s, and then decreased as initial-link duration increased to 30s. The bitonic function poses a potential challenge for existing models for steady-state choice, such as delay-reduction theory (DRT) [Fantino, E., 1969. Choice and rate of reinforcement. J. Exp. Anal. Behav. 12, 723-730], which predict a monotonic function. However, an extension of Grace and McLean's [Grace, R.C., McLean, A.P., 2006. Rapid acquisition in concurrent chains: evidence for a decision model. J. Exp. Anal. Behav. 85, 181-202] decision model predicted the bitonic function, and may ultimately provide an integrated account of choice in concurrent chains under both steady-state and dynamic conditions.     

Working and scrounging by zebra finches in an operant task

Groups of three zebra finches were housed in cages which could be adjoined to an operant chamber. Food was restricted and its availability was signalled by a red light projected behind an operant key (an autoshaping procedure). In addition to the shaping contingency, a single peck to the illuminated key, produced food immediately. Following the first peck to the illuminated key, by any subject in the group, the entire group was placed on a schedule of continuous reinforcement (CRF) wherein hopper operation was contingent upon a keypeck during the light. Comparison of individual's behavior revealed a differential pattern of operant working; one group member worked consistently, one worked periodically and one made few (if any) operant responses. All members consumed some reinforcers. The results suggest that zebra finches, under certain conditions, will form specialized labor roles. These data support similar demonstrations with primates (Chalmeau, R. and Galleau A., 1993. Behav. Processes, 28: 173–180), pigeons (Giraldeau, L. and Lefebvre L., 1987. Anim. Behav., 35: 387–394.) and rats (Thullier, F., Desor, D., Mos, J. and Krafft, B., 1992. Psychol. Behav., 53, 17–20).     

A two-stage model for concurrent sequences

Schneider and Davison [Schneider, S.M., Davison, M., 2005. Demarcated response sequences and the generalised matching law. Behav. Proc. 70, 51–61] showed that the generalised matching law applied to concurrent free-operant two-response sequences. When sufficient temporal spacing is required between the responses, however, neither the response-level nor the sequence-level forms of the generalised matching law provide good fits. An alternative “two-stage sensitivity” model with fewer free parameters features two types of sensitivity to the reinforcement contingencies on sequences. When temporal spacing between the responses is long, the “response distribution sensitivity” parameter describes sensitivity only of the individual responses to the sequence-level contingencies. At a threshold level, this sensitivity reaches a maximum. When spacing is shorter than threshold, the “response order sensitivity” parameter reflects a new sensitivity to the order of the responses within sequences. As this sensitivity approaches its maximum, sequence matching is achieved. For both stages, a changeover parameter describes bias against sequences that require changeovers between the two responses. The model fit data ranging from near-response matching with long minimum inter-response times (IRTs) to sequence matching with no minimum IRTs, using two species and a variety of sequence reinforcer distributions. Rats differed from pigeons in achieving sequence matching only at a nonzero minimum IRT. In a comparison based on pigeon data with no minimum IRT, the two-stage sensitivity model was more efficient than the generalised matching law according to the Akaike criterion. The logic of the model suggests a new way of understanding the mechanisms underlying behavioural units.     

Quantitative analyses of observing and attending

We review recent experiments examining whether simple models of the allocation and persistence of operant behavior are applicable to attending. In one series of experiments, observing responses of pigeons were used as an analog of attending. Maintenance of observing is often attributed to the conditioned reinforcing effects of a food-correlated stimulus (i.e., S+), so these experiments also may inform our understanding of conditioned reinforcement. Rates and allocations of observing were governed by rates of food or S+ delivery in a manner consistent with the matching law. Resistance to change of observing was well described by behavioral momentum theory only when rates of primary reinforcement in the context were considered. Rate and value of S+ deliveries did not affect resistance to change. Thus, persistence of attending to stimuli appears to be governed by primary reinforcement rates in the training context rather than conditioned reinforcing effects of the stimuli. An additional implication of these findings is that conditioned "reinforcers" may affect response rates through some mechanism other than response-strengthening. In a second series of experiments, we examined the applicability of the matching law to the allocation of attending to the elements of compound stimuli in a divided-attention task. The generalized matching law described performance well, and sensitivity to relative reinforcement varied with sample duration. The bias and sensitivity terms of the generalized matching law may provide measures of stimulus-driven and goal-driven control of divided attention. Further application of theories of operant behavior to performance on attention tasks may provide insights into what is referred to variously as endogenous, top-down, or goal-directed control of attention.     

Temporal generalization and diffusion in forgetting

Two processes may contribute to the decrement in discriminability with increasing temporal distance between the occasioning event and later choice. One is the length of the interval and the other is generalization decrement. In the model described by White and Wixted [J. Exp. Anal. Behav. 71 (1999) 91-113], choice was predicted by the relative payoff for correct delayed matching responses, conditional on the current value of the stimulus sampled from Thurstone-like probability distributions of the effect of the sample stimuli. In the model, discriminability decreased with increasing temporal distance because the variance of the distributions increased with time. However, White and Wixted did not specify the function relating variance to temporal distance. If a diffusion process is assumed, and if diffusion increases exponentially with time, the resulting forgetting function is a negative exponential. An additional process involves exponential generalization of remembering from one time to other times. Alternative diffusion functions result in hyperbolic or power forgetting functions. The combination of two exponential processes yields forgetting functions that are double exponential in form and which appear consistent with a wide range of data.     

Matching and ideal free distributions

Ideal free distributions characterise how a group of animals should be distributed between sources of food. In a simple case, the ratio of the number of animals matches the ratio of input rates, a result known as input matching. The matching law characterises how an animal should allocate responses to sources of food. If matching holds then the ratio of behaviour allocated to the sources matches the ratio of rewards obtained. Several authors have drawn attention to the analogy between input matching and the matching law. I present a critical review of this topic, and go on to investigate the claim that the distribution of animals should be analyzed in the same way that the generalized matching law is analyzed. This involves assuming that the relationship between the ratio of animals and the ratio of resources obeys a power law with two parameters, one corresponding to bias and the other to sensitivity. On this view, a plot of the log of the ratio of animals against the log of the ratio of input rates will be a straight line and its intercept and slope will provide estimates of bias and sensitivity. I show that this approach can give a good fit even when the underlying model does not result in a power law. A consequence is that the parameters estimated cannot be interpreted as bias and sensitivity. I conclude that there are no compelling reasons for analyzing the distribution of animals using log-log plots, and the method has the disadvantage of encouraging the mistaken view that deviations from input matching follow a power law.     

Model-checking techniques based on cumulative residuals

Residuals have long been used for graphical and numerical examinations of the adequacy of regression models. Conventional residual analysis based on the plots of raw residuals or their smoothed curves is highly subjective, whereas most numerical goodness-of-fit tests provide little information about the nature of model misspecification. In this paper, we develop objective and informative model-checking techniques by taking the cumulative sums of residuals over certain coordinates (e.g., covariates or fitted values) or by considering some related aggregates of residuals, such as moving sums and moving averages. For a variety of statistical models and data structures, including generalized linear models with independent or dependent observations, the distributions of these stochastic processes tinder the assumed model can be approximated by the distributions of certain zero-mean Gaussian processes whose realizations can be easily generated by computer simulation. Each observed process can then be compared, both graphically and numerically, with a number of realizations from the Gaussian process. Such comparisons enable one to assess objectively whether a trend seen in a residual plot reflects model misspecification or natural variation. The proposed techniques are particularly useful in checking the functional form of a covariate and the link function. Illustrations with several medical studies are provided.     

Physicochemical explanation of peptide binding to HLA-A*0201 major histocompatibility complex: a three-dimensional quantitative structure-activity relationship study

A three-dimensional quantitative structure-activity relationship method for the prediction of peptide binding affinities to the MHC class I molecule HLA-A*0201 was developed by applying the CoMSIA technique on a set of 266 peptides. To increase the self consistency of the initial CoMSIA model, the poorly predicted peptides were excluded from the training set in a stepwise manner and then included in the study as a test set. The final model, based on 236 peptides and considering the steric, electrostatic, hydrophobic, hydrogen bond donor, and hydrogen bond acceptor fields, had q2 = 0.683 and r2 = 0.891. The stability of this model was proven by cross-validations in two and five groups and by a bootstrap analysis of the non-cross-validated model. The residuals between the experimental pIC50 (-logIC50) values and those calculated by "leave-one-out" cross-validation were analyzed. According to the best model, 63.2% of the peptides were predicted with /residuals/ < or = 0.5 log unit; 29.3% with 1.0 < or = /residuals/ < 0.5; and 7.5% with /residuals/ > 1.0 log unit. The mean /residual/ value was 0.489. The coefficient contour maps identify the physicochemical property requirements at each position in the peptide molecule and suggest amino acid sequences for high-affinity binding to the HLA-A*0201 molecule.     

Prospective timing, attention and the switch. A response to 'Gating or switching? gating is a better model of prospective timing' by Zakay

In response to the 'Switching or gating' paper (Lejeune, H., 1998. Switching or gating? The attentional challenge in cognitive models of psychological time. Behav. Proc. 44, 127-145), Zakay argued that attention allocation to time should reflect attentional processes in general and suggested that the attentional gate model (AGM) has more explanatory power than the temporal information processing model (TIP) of Church (Church, R.M., 1984. Properties of the internal clock. In: Gibbon, J., Allan, L., (Eds.), Timing and Time Perception, vol. 423. Annals of the New York Academy of Sciences, New York, pp. 566-582). The first point might not be challenged, provided that the specificity of the temporal stimulus is taken into account. Concerning the second point, we argue that the TIP model can account for human prospective timing and discuss differences between attention versus expectancy or motivation. We prefer a 'satellite' attention allocation process, targeting the switch and reference memory (Meck, W.H., 1983. Selective adjustment of the speed of the internal clock and memory processes. J. Exp. Psychol.: Anim. Behav. Proc. 9, 171-201) to an attentional gate serially included in the TIP model of Church.     

Stimulus-dependent adjustment of reward prediction error in the midbrain

Previous reports have described that neural activities in midbrain dopamine areas are sensitive to unexpected reward delivery and omission. These activities are correlated with reward prediction error in reinforcement learning models, the difference between predicted reward values and the obtained reward outcome. These findings suggest that the reward prediction error signal in the brain updates reward prediction through stimulus-reward experiences. It remains unknown, however, how sensory processing of reward-predicting stimuli contributes to the computation of reward prediction error. To elucidate this issue, we examined the relation between stimulus discriminability of the reward-predicting stimuli and the reward prediction error signal in the brain using functional magnetic resonance imaging (fMRI). Before main experiments, subjects learned an association between the orientation of a perceptually salient (high-contrast) Gabor patch and a juice reward. The subjects were then presented with lower-contrast Gabor patch stimuli to predict a reward. We calculated the correlation between fMRI signals and reward prediction error in two reinforcement learning models: a model including the modulation of reward prediction by stimulus discriminability and a model excluding this modulation. Results showed that fMRI signals in the midbrain are more highly correlated with reward prediction error in the model that includes stimulus discriminability than in the model that excludes stimulus discriminability. No regions showed higher correlation with the model that excludes stimulus discriminability. Moreover, results show that the difference in correlation between the two models was significant from the first session of the experiment, suggesting that the reward computation in the midbrain was modulated based on stimulus discriminability before learning a new contingency between perceptually ambiguous stimuli and a reward. These results suggest that the human reward system can incorporate the level of the stimulus discriminability flexibly into reward computations by modulating previously acquired reward values for a typical stimulus.     

A computational model of selection by consequences: log survivor plots

[McDowell, J.J, 2004. A computational model of selection by consequences. J. Exp. Anal. Behav. 81, 297-317] instantiated the principle of selection by consequences in a virtual organism with an evolving repertoire of possible behaviors undergoing selection, reproduction, and mutation over many generations. The process is based on the computational approach, which is non-deterministic and rules-based. The model proposes a causal account for operant behavior. McDowell found that the virtual organism consistently showed a hyperbolic relationship between response and reinforcement rates according to the quantitative law of effect. To continue validation of the computational model, the present study examined its behavior on the molecular level by comparing the virtual organism's IRT distributions in the form of log survivor plots to findings from live organisms. Log survivor plots did not show the "broken-stick" feature indicative of distinct bouts and pauses in responding, although the bend in slope of the plots became more defined at low reinforcement rates. The shape of the virtual organism's log survivor plots was more consistent with the data on reinforced responding in pigeons. These results suggest that log survivor plot patterns of the virtual organism were generally consistent with the findings from live organisms providing further support for the computational model of selection by consequences as a viable account of operant behavior.     

Deletion diagnostics for alternating logistic regressions

Deletion diagnostics are introduced for the regression analysis of clustered binary outcomes estimated with alternating logistic regressions, an implementation of generalized estimating equations (GEE) that estimates regression coefficients in a marginal mean model and in a model for the intracluster association given by the log odds ratio. The diagnostics are developed within an estimating equations framework that recasts the estimating functions for association parameters based upon conditional residuals into equivalent functions based upon marginal residuals. Extensions of earlier work on GEE diagnostics follow directly, including computational formulae for one‐step deletion diagnostics that measure the influence of a cluster of observations on the estimated regression parameters and on the overall marginal mean or association model fit. The diagnostic formulae are evaluated with simulations studies and with an application concerning an assessment of factors associated with health maintenance visits in primary care medical practices. The application and the simulations demonstrate that the proposed cluster‐deletion diagnostics for alternating logistic regressions are good approximations of their exact fully iterated counterparts.     

Reactions to children's faces: Males are more affected by resemblance than females are, and so are their brains

The detection of genetic relatedness (i.e., kinship) affects the social, parental, and sexual behavior of many species. In humans, self-referent phenotype matching based on facial resemblance may indicate kinship, and it has been demonstrated that facial resemblance increases perceptions of trustworthiness and attractiveness [Proc. R. Soc. Lond., B Biol. Sci. 269 (2002) 1307–1312; Proc. R. Soc. Lond., B Biol. Sci. (in press)]. However, investigations of sex differences in reaction to facial resemblance have produced mixed results [Evol. Hum. Behav. 25 (2004) 142–154; Evol. Hum. Behav. 23 (2002) 159–166; Evol. Hum. Behav. 24 (2003) 81–87]. Here, we replicate the effects of Platek et al. [Evol. Hum. Behav. 23 (2002) 159–166] using high-resolution color morphing. We also extend these findings using functional magnetic resonance imaging (fMRI) to demonstrate a possible neural mechanism that may account for the observed sex difference. These data support the hypothesis that human males may use and favor facial resemblance as a paternity cue.     

Classical conditioning,signal detection,and evolution

Strength of classical conditioning is increased either by increasing discriminability of the conditioned stimulus (CS) from the background, or by increasing contingency between conditioned and unconditioned stimili (US).Classical conditioning can be regarded as a decision process in which the subject has to decide whether or not to respond with a conditioned response in the presence or absence of the CS. According to modern evolutionary theories, it might be assumed that this decision process maximizes the trade-off between cost and benefits.By assuming that the decision rule maximizes expected benefit, the empirical relationship between contingency and the strength of classical conditioning is theoretically derived. In addition, when the decision rule is incorporated to a signal detection paradigm, theoretical results describing the relationship between CS discriminability and CS – US contingency with the strength of classical conditioning are in agreement with experimental data.     

A novel method for assessing the 3-D orientation accuracy of inertial/magnetic sensors

A novel method for assessing the accuracy of inertial/magnetic sensors is presented. The method, referred to as the “residual matrix” method, is advantageous because it decouples the sensor's error with respect to Earth's gravity vector (attitude residual error: pitch and roll) from the sensor's error with respect to magnetic north (heading residual error), while remaining insensitive to singularity problems when the second Euler rotation is close to ±90°. As a demonstration, the accuracy of an inertial/magnetic sensor mounted to a participant's forearm was evaluated during a reaching task in a laboratory. Sensor orientation was measured internally (by the inertial/magnetic sensor) and externally using an optoelectronic measurement system with a marker cluster rigidly attached to the sensor's enclosure. Roll, pitch and heading residuals were calculated using the proposed novel method, as well as using a common orientation assessment method where the residuals are defined as the difference between the Euler angles measured by the inertial sensor and those measured by the optoelectronic system. Using the proposed residual matrix method, the roll and pitch residuals remained less than 1° and, as expected, no statistically significant difference between these two measures of attitude accuracy was found; the heading residuals were significantly larger than the attitude residuals but remained below 2°. Using the direct Euler angle comparison method, the residuals were in general larger due to singularity issues, and the expected significant difference between inertial/magnetic sensor attitude and heading accuracy was not present.