Choice in humans: Functional properties of reinforcers established by instruction

Adult human subjects chose between schedules containing stimuli (indicator lights) that the subjects were instructed to consider pleasurable. The schedules differed in amount of reinforcement (period of illumination) or delay (interval between a choice response and light onset). Although subjects preferred large to small amounts of reinforcement, they were essentially indifferent between immediate and delayed reinforcement. In contrast, previous data on video game reinforcement demonstrated preferences for both immediate and large amounts of reinforcement. The instructed reinforcer was thus partially effective in controlling choice but was not equivalent to a reinforcer that presumably had intrinsic value.     

A hybrid zone between Chorthippus parallelus parallelus and Chorthippus parallelus erythropus (Orthoptera: Acrididae): behavioural characters

Two subspecies of the grasshopper Chorthippus parallelus have ranges that abut in the Pyrenees. Males produce two types of song—calling song and courtship song. We have examined the use and structure of these songs in males from nine sites near Mont-Louis, Pyrénées-Onentales, France. These sites were previously used to identify the position of the hybrid zone on morphological characters. The subspecies differ in the use and structure of both types of song. Clines in these characters correspond in position with the morphological cline. At least one character may be relevant to mate choice but none of the characters show evidence of reinforcement.     

Assortative mating across a hybrid zone in Chorthippus parallelus (Orthoptera: Acrididae)

Two subspecies of the grasshopper Chorthippus parallelus meet and form a hybrid zone in the Pyrenees. Both strong hybrid dysfunction and a behavioural difference occur, which would seem to make the zone a suitable candidate for speciation by reinforcement. One of the classic ways of looking for this is to test for increased levels of assortative mating between populations from close to the region of contact. Here we show that, with virgin insects, such assortment decreases as one approaches the centre of the zone in C. parallelus. The pattern is different upon remating, with non-virgins showing a pattern more like that predicted by reinforcement. Overall there is little evidence for reinforcement. We argue that other tests of the model may be more appropriate for stable hybrid zones.     

LACK OF CALLING SONG DISPLACEMENT BETWEEN TWO CLOSELY RELATED GROUND CRICKETS

Thorough examinations of purported cases of reproductive character displacement are critical for reaching an understanding of the role of reinforcement in the evolution of reproductive barriers between closely related species. In this paper, we report the results of an extensive investigation of male calling song variation in the ground crickets, Allonemobius fasciatus and A. socius. Contrary to the results of an earlier study, we uncovered little evidence of displacement of songs in areas of overlap. We discuss explanations for the lack of displacement as well as for the discrepancies between the results of the current study and those of the earlier study.     

Estimating tree-based dynamic treatment regimes using observational data with restricted treatment sequences

A dynamic treatment regime (DTR) is a sequence of decision rules that provide guidance on how to treat individuals based on their static and time-varying status. Existing observational data are often used to generate hypotheses about effective DTRs. A common challenge with observational data, however, is the need for analysts to consider “restrictions” on the treatment sequences. Such restrictions may be necessary for settings where (1) one or more treatment sequences that were offered to individuals when the data were collected are no longer considered viable in practice, (2) specific treatment sequences are no longer available, or (3) the scientific focus of the analysis concerns a specific type of treatment sequences (eg, “stepped-up” treatments). To address this challenge, we propose a restricted tree–based reinforcement learning (RT-RL) method that searches for an interpretable DTR with the maximum expected outcome, given a (set of) user-specified restriction(s), which specifies treatment options (at each stage) that ought not to be considered as part of the estimated tree-based DTR. In simulations, we evaluate the performance of RT-RL versus the standard approach of ignoring the partial data for individuals not following the (set of) restriction(s). The method is illustrated using an observational data set to estimate a two-stage stepped-up DTR for guiding the level of care placement for adolescents with substance use disorder.     

Influence of handling and conditioning protocol on learning and memory of Microplitis croceipes

Microplitis croceipes (Cresson) (Hymenoptera: Braconidae) learns odors in association with both hosts and food. The food‐associated ‘seeking’ behavior of M. croceipes was investigated under various training protocols utilizing the conditioning odor, 3‐octanone. We investigated the effects of odor training, or its lack, training duration, training frequency, time elapsed after training, wasp hunger state, and training reinforcement, on the food‐seeking responses of M. croceipes females. We found that odor‐trained females show strong food seeking responses, whereas non‐odor‐trained females do not respond to the odor, and that a single 10 s association with the odor whilst feeding on sugar water subsequently conditioned the wasps to exhibiting significant responses to it. Increases in training time to more than 10 s did not improve their responses. Repetition of the food–odor associations increased a wasp's recall, as well as its response over time, compared to a single exposure. Repeated exposure to the learned odor in the absence of a food reward decreased the responses of less hungry individuals. However, the level of response increased significantly following a single reinforcement with the food–odor association. Understanding the factors that influence learning in parasitoids can enhance our ability to predict their foraging behavior, and opens up avenues for the development of effective biological detectors.     

Simulation of direct shear tests on rooted and non-rooted soil using finite element analysis

The finite element (FE) method has been used in recent years to simulate overturning processes in trees and to better comprehend plant anchorage mechanics. We aimed at understanding the fundamental mechanisms of root-soil reinforcement by simulating direct shear of rooted and non-rooted soil. Two- (2D) and three-dimensional (3D) FE simulations of direct shear box tests were carried out using readily available software for routine strength assessment of the root-soil composite. Both rooted and non-rooted blocks of soil were modelled using a simplified model of root distribution and root material properties representative of real roots. Linear elastic behaviour was assumed for roots and the soil was modelled as an ideally plastic medium. FE analysis showed that direct shear tests were dependent on the material properties specified for both the soil and roots. 2D and 3D simulations of direct shear of non-rooted soil produced similar results and any differences between 2D and 3D simulations could be explained with regard to the spatial complexity of roots used in the root distribution model. The application of FE methods was verified through direct shear tests on soil with analogue roots and the results compared to in situ tests on rooted soil in field conditions.     

Variations in the timing of reinforcement as a training technique for foals (Equus caballus)

Abstract

Horses are used worldwide for a range of activities. Their usefulness and welfare in these pursuits are strongly influenced by their trainability, which may in turn be influenced by learning ability. Handling and riding horses can expose both handler and horse to a considerable risk of injury. This risk can be reduced by employing correct handling procedures that can facilitate learning in horses. As with all training, efficacy is influenced by consistency and timing. To determine the optimum timing of reinforcement, 16 unweaned naïve foals, that had previously undergone minimal human–animal interaction (i.e., not had a headcollar previously applied), of warmblood (WB; n = 6), thoroughbred (TB; n = 5) or warmblood x thoroughbred (WB x TB; n = 6) breeding were randomly assigned to three treatment groups for testing on ten training days at approximately 14-day intervals. Pressure applied to a headcollar via a lead rope was used as the stimulus for each foal to walk forward, and this was repeated until the foal had walked a distance of 8 m. The effects of three different latencies of negative reinforcement were evaluated by releasing the pressure either immediately: as the first foreleg step commenced (Treatment 1); when the second step of the forelegs was completed (Treatment 2) or when the fourth step of the forelegs was completed (Treatment 3). Each foal's rate of learning was measured by the proportion of correct responses relative to the total number of responses performed. Behavioral responses exhibited (rears, strikes, head shakes, falls, sideways movement and hops) and the steps taken over the distance were also recorded.

Initially, the foals undergoing Treatment 1 appeared to learn more quickly than those foals undergoing Treatments 2 and 3, suggesting that Treatment 1 was associated with the greatest compliance and the quickest learning. However, the foals undergoing Treatment 3 ultimately achieved significantly (p < 0.001) more correct responses, suggesting that the longer delay of reinforcement (i.e., the longer duration of aversive stimulus) may enhance learning via the negative reinforcement inherent in lead training in foals. While some conflict behaviors were shown in all treatment groups, most were exhibited on training day 2. This was reflected in the analysis of composite behaviors performed, with training days 1 and 2 being different (p < 0.001) from training day 3, and training days 1–3 being different (p < 0.001) from training days 4–10. These changes indicate that learning occurred in all treatment groups.

The foals used in this study were sired by five different stallions. While the foals sired by stallion 2 (WB) performed significantly (p < 0.001) more correct responses, those foals sired by WB stallions were significantly (p < 0.001) less likely to perform correct responses when compared with those foals of TB or WB x TB breeding. Colts were significantly (p < 0.001) more likely to perform correct responses than were fillies.     

Training of laboratory-housed non-human primates in the UK

Abstract

Laboratory-housed non-human primates may experience a range of potential stressors, including physical and chemical restraint, venepuncture, injection, catching and cage-change. Training them to co-operate, using positive reinforcement techniques, is one means of significantly reducing the adverse impact of such procedures upon them and, therefore, is a refinement. Furthermore, the additional time that staff spend with the primates, and the need for individual recognition and close observation of animal behavior, mean that the trainer develops a relationship with each individual animal which can be beneficial for animal welfare (e.g., by reducing the occurrence of abnormal behavior and fear of humans). We surveyed use of training in thirteen UK establishments using and breeding primates, utilizing a mixed-mode questionnaire. The survey demonstrated that there is widespread awareness of training as a refinement and appreciation of its diverse benefits, but training is not used as widely or as fully as it might be. This is largely due to real constraints (principally staff and time and a lack of confidence in ability to train), and perceived constraints (such as a supposed lack of information on how to train and assessment of the benefits, and an overestimation of the time investment needed). There is also considerable variation between establishments in the purposes of training and techniques used, with a reliance on negative reinforcement in three. We conclude that there is considerable scope for refinement of common scientific, veterinary and husbandry procedures, and refer to some resources designed to help establishments take action (e.g., Prescott and Buchanan-Smith 2003).