The volume of reinforcement and the structure of rat behavior in a radial labyrinth

In rats choice reaction was elaborated in 12 passages radial labyrinth with different volumes of food only in six of twelve passages. Then the conditions were modified: the first group of rats in the beginning of the experiment was placed to another labyrinth sector, and in the second group the food volume was changed in passages. In new situations in animals of both groups the reaction efficiency temporarily decreased, simultaneously the correction of behaviour structure took place. The character of behavioural tactics (tendencies to visit passages with larger food volume at the beginning of the experiment, spatial-motor asymmetry etc.) is described in dynamics of adaptation to new conditions. Factors determining the rats behaviour structure are discussed.     

The choice of reinforcement quality depending on the delay time of the instrumental reaction in cats]

In experiments on seven cats instrumental delayed reflexes with different quality reinforcement were elaborated by method of "active choice", where the time of delay of the motor reaction served as a signal of quality of the reinforcement. Five cats used a strategy with long delay high-quality reinforcement (meat), and two cats preferred short delay and low-quality reinforcement (bread-meat mixture). In special experiments with the change of alimentary motivation (24-hour alimentary deprivation and saturation of animals) it was established that the level of alimentary motivation within the studied limits did not change the strategy of behaviour preferred by the given animal. Individually varying ability to choice of reaction determining more valuable reinforcement it is possible to consider as an experimental analogue of the mechanisms of will. The question is discussed about the role of typological properties of cats in choice of behaviour strategy.     

Current problems in the reinforcement and regulation of goal-directed behavior]

Contemporary problems and arguable questions of reinforcement and self-regulation of animals behaviour and human activity are considered conformably to organization and analysis of results of conditioned experiments with probable reinforcement on the basis of semantic and pragmatic diagrams of conceptual space of decisions making. Situations are analyzed when the subject not simply "receives" the reinforcement but must actively "procure" (earn) the useful result. Scientific substantiation of conceptions of reinforcement and activity self-regulation is of considerable importance not only in the higher nervous activity physiology but also in psychology, economics and sociology for the decision of actual problems of organization of the processes of control in contemporary society of all mankind values and norms of behaviour.     

Probability analysis of elements of rat behavior after sudden and signalled interruption of food reinforcement

Probability analysis was carried out of the appearance of single elements of rats behaviour in the process of extinction of a conditioned alimentary motor reflex. The dynamics of effector behavioural components at a sudden cessation of reinforcement (usual schedule of extinction) was compared with cessation of reinforcement signalled by a previously differentiated signal and with reinforcement cessation preceded by a stimulus initially unknown to the animal. If the reinforcement cessation is signalled by a previously differentiated (negative) stimulus, in response to its action the animals "loose the aim", what is revealed in a rapid complete reduction of all elements of the goal-directed alimentary behaviour. Obviously differentiation signal actualises the memory trace of "nonreinforcement" which was formed in the previous negative experience of the animal; this is revealed in accelerated inhibition of the alimentary motor reflex under extinction.     

Maladaptive choice behaviour by pigeons: an animal analogue and possible mechanism for gambling (sub-optimal human decision-making behaviour)

Consistent with human gambling behaviour but contrary to optimal foraging theory, pigeons showed maladaptive choice behaviour in experiment 1 by choosing an alternative that provided on average two food pellets over an alternative that provided a certain three food pellets. On 20 per cent of the trials, choice of the two-pellet alternative resulted in a stimulus that always predicted ten food pellets; on the remaining 80 per cent of the trials, the two-pellet alternative resulted in a different stimulus that always predicted zero food pellets. Choice of the three-pellet alternative always resulted in three food pellets. This choice behaviour mimics human monetary gambling in which the infrequent occurrence of a stimulus signalling the winning event (10 pellets) is overemphasized and the more frequent occurrence of a stimulus signalling the losing event (zero pellets) is underemphasized, compared with the certain outcome associated with not gambling (the signal for three pellets). In experiment 2, choice of the two-pellet alternative resulted in ten pellets with a probability of 20 per cent following presentation of either stimulus. Choice of the three-pellet alternative continued to result in three food pellets. In this case, the pigeons reliably chose the alternative that provided a certain three pellets over the alternative that provided an average of two pellets. Thus, in experiment 1, the pigeons were responding to obtain the discriminative stimuli signalling reinforcement and the absence of reinforcement, rather than to obtain the variability in reinforcement.     

Factors affecting crossing of red foxes and wildcats through non-wildlife passages across a high-speed railway

We used trail records on sand surface within non-wildlife passages to test whether foxes and wildcats used them regularly, and to identify passage features which may favour crossing across a railway and, therefore, may alleviate possible banner effects Both species crossed more in places and periods corresponding with assumed peaks in abundance and mobility, thus supporting the regular use hypothesis The vicinity of cover favoured crossing, but both species used infrequently passages near permanent sources of human perturbation (especially intense traffic) even in suitable habitats with abundant cover The presence of cover in the passage entrances further favoured fox and wildcat crossing Cover near entrances may be particularly important to improve carnivore crossing chances in open habitats and when human activity levels are high Passage design and dimensions had little effect on crossing rates Location of passages within or close to suitable habitats explained a greater amount of variance in crossing rates than favourable passage features There was a remarkable similarity in the behaviour of foxes and wildcats, possibly representing the rule for other carnivore species     

Characteristics of the role of spatial memory in the choice reactions of white rats

Behaviour of the rats, previously learnt to come back to one and the same place of reinforcement was studied in conditions of periodical changes of this place. It was found that after detection of the new place of reinforcement, the rats could optimize their behaviour according to disposition of this place. Optimization of behaviour consisted in shifts of running direction towards the new reinforcement place and (or) in selective displacements before blinds, according to the places of reinforcement in the experiment. It is suggested that an increase of the role of working memory and apparatus of probabilistic prognosis at choice lies in the basis of optimization.     

The relation of the typological classification of dogs to their behavioral characteristics in variations between the probability and the value of reinforcement]

In studying of dogs behaviour in coordinates system "Requirements and probability of their satisfaction" it has been established that selection of behaviour strategy is closely connected with individual typological properties of animals. In experiments with equally-probable alimentary reinforcement a direct correlation was found between the strength parameter of the nervous system and the strategy of dogs behaviour. Animals of strong types manifested the character of reaction which consisted in determination and goal-directedness, while the animals of the weak type manifested other character, having features of uncertainty, indefinition, hesitations. In conditions of conflict between probability and value of reinforcement the dogs manifested two opposite strategies of behaviour: orientation to highly probable events (choleric and phlegmatic) and to low-probable events (sanguinic and melancholic) what is connected with individual properties of functioning and the character of interaction of four brain structures (frontal cortex, hippocampus, hypothalamus, amygdala).     

The behavior of tamarin monkeys during the learning and realization of instrumental food habits in a competitive situation

Special features of competitive behavioural organization have been revealed during formation and realization of simple feeding and instrumental reflexes elaborated simultaneously in family pairs of marmoset monkeys. In contrast to other anthropoid species, male and female marmosets performed feeding reactions of different levels of complexity with equal efficiency. The differences between male and female marmosets concern the structure of competitive behaviour: males showed more active forms of competition under equal number of the given reinforcement, whereas passive forms of behaviour were more typical in females. The latter produced intensive vocalization in comparison with males. Stabilization of competitive activity was accompanied by the decrease of active forms of competition and by standardization of individual behaviour. The increase in the number of behavioural strategies was observed in family groups under sophisticated experimental conditions.     

Effect of acute and chronic deprivation of catecholaminergic system activity induced by 6-hydroxydopamine on the behavior of the rat

On 202 male rats of Wistar line, a study was carried out of the effect of chronic and acute deprivations of the brain CA-systems activity resulting from administration of 6-OHDA on investigating behaviour and learning. Chronic deprivation of CA-systems activity by neonatal administration of 6-OHDA (100 mg/kg subcutaneously) and their acute deprivation by intracerebral administration of 6-OHDA to adult rats (150 mkg in each lateral ventriculus) was accompanied by similar deep changes of behaviour. Both forms of deprivation reduced the investigating activity of the animals in the open field. In both cases, the above 6-OHDA dozes sharply impeded the learning of animals with emotionally negative reinforcement, with no significant influence on learning with emotionally positive reinforcement. Both forms of deprivation of CA-systems activity weakened the reaction of frustration elicited by a sharp reduction of food reinforcement.     

Generating adaptive behaviour within a memory-prediction framework

The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex.We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play "rocks, paper, scissors" against a predictable opponent.     

Quantifying the role of vegetation in slope stability: A case study in Tuscany (Italy)

Vegetation significantly affects hillslope hydrological and mechanical properties related to shallow landslide triggering. In view of the complexity of soil plant hydrological interactions, the quantification of root mechanical reinforcement remains a challenge. Herein we present a back analysis of mechanical stability criteria related to a well-characterized vegetated shallow landslide in Italy, focusing on the quantification of lateral and basal root reinforcement. Lateral root reinforcement is included in slope stability estimates by adding a stabilizing force proportional to the scarp surface and root distribution. This stabilizing force is added to the force balance equation for the infinite slope model for different landslide shapes and dimensions. To quantify root reinforcement, we use the Wu model and the fiber bundle model (WM and FBM, respectively). Implementation of the latter model allows the quantification of the stress–strain behaviour of a bundle of roots for different root distributions and mechanical properties. Results of these models are compared highlighting key differences between the two approaches. Calculations using the FBM can explain the overestimation of lateral root reinforcement using WM and the commonly observed overestimation in the factor of safety. The model also quantifies the displacement-dependent behaviour of root reinforcement on vegetated slopes. Lateral root reinforcement can strongly influence the stability of slopes up to a certain area (1000–2000 m²). The magnitude of this stabilizing effect depends on parameters such as inclination, soil mechanical properties, and root distribution.     

Probabilistic reinforcement and neurotic breakdown in unrestrained dogs]

The substitution of a constant reinforcement for a random one with a probability of 0.5 in experiments on two dogs with a simple motor stereotype was attended with nervous breakdowns with motor excitation, inadequate orienting reactions or passive-defensive behaviour. The change in the probability of reinforcement from 0.5 to 0.3 had a positive effect in experiments on one dog, while in the other it developed drowsiness. In two other dogs with a complicated stereotype, the change in the mode of reinforcement was attended with a peculiar preventive effect of probabilistic, but ordered reinforcement in experimental surroundings, including signals with a probabilistic random reinforcement as well. In this case the dogs displayed primarily signs of emotional stress only. Behaviour was somewhat disturbed in one animal only when testing signals with a probabilistic random reinforcement.     

Biological properties of the mumps virus. Behavior using the T50 marker

The authors studied the behaviour of 11 mumps virus strains or variants including the thermolabile standard Jeryl Lynn strain under thermal charge (50 degrees C/30 min). Varants were obtained from the Soviet vaccinal strains Leningrad-3 by cultivation under various conditions. Incubation temperature and cellular substrate played an important role therein. Variants with various behaviour in the marker T50 resulted. It was found that passages at 32 degrees C at limited dilutions as well as those on chick embryos or in cultures of chicken fibroblasts increased their thermolability. Possible correlations between their behaviour in the marker T50 and the degree of di attenuation are discussed. (Ta)     

The olfactory orientation traits of white rats in a maze]

Some characteristics of olfactory behaviour of albino rats in three-arm maze have been studied. It was found that the olfactory discrimination of different food reinforcements may be elaborated in passages as long as 90 cm. Choice reactions were determined by olfactory, not visual, stimuli on their simultaneous presentation.     

Behavioural analyses of quinine processing in choice, feeding and learning of larval Drosophila

Gustatory stimuli can support both immediate reflexive behaviour, such as choice and feeding, and can drive internal reinforcement in associative learning. For larval Drosophila, we here provide a first systematic behavioural analysis of these functions with respect to quinine as a study case of a substance which humans report as "tasting bitter". We describe the dose-effect functions for these different kinds of behaviour and find that a half-maximal effect of quinine to suppress feeding needs substantially higher quinine concentrations (2.0 mM) than is the case for internal reinforcement (0.6 mM). Interestingly, in previous studies (Niewalda et al. 2008, Schipanski et al 2008) we had found the reverse for sodium chloride and fructose/sucrose, such that dose-effect functions for those tastants were shifted towards lower concentrations for feeding as compared to reinforcement, arguing that the differences in dose-effect function between these behaviours do not reflect artefacts of the types of assay used. The current results regarding quinine thus provide a starting point to investigate how the gustatory system is organized on the cellular and/or molecular level to result in different behavioural tuning curves towards a bitter tastant.     

Risk‐taking behaviour variation over time in sea bass Dicentrarchus labrax: effects of day–night alternation,fish phenotypic characteristics and selection for growth

Differences in bold and shy personality on sea bass Dicentrarchus labrax were investigated between a population (wild) produced from wild‐brood fish and a population (selected) produced from selected‐brood fish. During the experiment (112 days), fish were reared under self‐feeding condition to characterize the feeding behaviour of each individual fish. Three risk‐taking tests (T1, T2 and T3 of 24 h with day–night alternation) were carried out at > 1 month intervals on 180 fish of each strain in order to monitor D. labrax behaviour over time and in relation to the light:dark period. A risk‐taking score was evaluated via a preference choice between a safe zone (without food) and a risky zone (potentially with food) by recording the number and the duration of individual passages through an opening in an opaque divider. Results showed that fish performed passages preferentially during the night period and that wild fish were generally bolder than selected fish during T1 and T2 but showed a decrease in risk taking during T3, contrary to selected fish which showed a constant increase in their risk‐taking behaviour. The phenotypic characteristics of the bold fish were different in the two strains: wild bold fish were the smallest within the wild strain and selected bold fish presented the higher growth rate within the selected strain. For both strains, these bold fish were also generally characterized by a high feed‐demand activity. Fish hunger state thus seemed to be the highest motivation for risk‐taking behaviour under the present conditions. Furthermore, behavioural variations over tests such as higher risk taking (number of passages) and faster exploratory responses (higher score emergence) could be interpreted as relevant indicators of the learning process and habituation. According to the results, however, no real difference in coping strategy between strains could be observed at this first stage of domestication and selection.     

抗体选择压作用下新城疫病毒HN和F基因的演化及其抗原性变异的比较分析

TZ060107株新城疫病毒(NDV)在含有对它抗体的鸡胚成纤维细胞(CEF)培养上分3个独立系列连传50代,每10代扩增其HN和F基因并测序。选择变异最大的系列A1-50病毒,再在含有抗A1-50抗体的CEF培养上分3个独立系列连续传50代,同时设3个不带抗体的独立传代系列作为对照。对第60、70、80、90、100代病毒的HN和F基因序列比较结果显示,有抗体组HN基因的非同义突变(NS)对同义突变(S)比值NS/S为5.25,明显高于无抗体组NS/S的2.375。前50代在抗体选择压作用下已发生的稳定NS突变在含有抗A1-50抗体的细胞培养中传代仍能稳定保持,且又出现了一个新的稳定的NS突变位点。在有抗体组经传50代后F基因发生的稳定非同义突变,在抗A1-50血清作用下再连传50代后也仍然保持,且又出现3个新的稳定的NS突变。不同传代病毒与原始病毒间的血清交叉血球凝聚抑制试验结果表明,随着在含有抗NDV血清的细胞培养上传代代数的增加,病毒与原始病毒间在抗原性的差异越来越大。     

Effect of reinforcement, reinforcer omission and extinction on a communicative response in domestic dogs (Canis familiaris)

There is a controversy about the mechanisms involved in the interspecific communicative behaviour in domestic dogs. The main question is whether this behaviour is a result of instrumental learning or higher cognitive skills are required. The present investigations were undertaken to study the effect of learning processes upon the gaze towards the human's face as a communicative response. To such purpose, in Study 1, gaze response was subjected to three types of reinforcement schedules: differential reinforcement, reinforcer omission, and extinction in a situation of “asking for food”. Results showed a significant increase in gaze duration in the differential reinforcement phase and a significant decrease in both the omission and extinction phases. These changes were quite rapid, since they occurred only after three training trials in each phase. Furthermore, extinction resulted in animal behaviour changes, such as an increase in the distance from the experimenter, the back position and lying behaviour. This is the first systematic evaluation of the behavioural changes caused by reward withdrawal (frustration) in dogs. In Study 2, the gaze response was studied in a situation where dogs walked along with their owners/trainers. These results show that learning plays an important role in this communicative response. The possible implications of these results for service dogs are discussed.     

Spiking neural networks with different reinforcement learning (RL) schemes in a multiagent setting

This paper investigates the effectiveness of spiking agents when trained with reinforcement learning (RL) in a challenging multiagent task. In particular, it explores learning through reward-modulated spike-timing dependent plasticity (STDP) and compares it to reinforcement of stochastic synaptic transmission in the general-sum game of the Iterated Prisoner's Dilemma (IPD). More specifically, a computational model is developed where we implement two spiking neural networks as two "selfish" agents learning simultaneously but independently, competing in the IPD game. The purpose of our system (or collective) is to maximise its accumulated reward in the presence of reward-driven competing agents within the collective. This can only be achieved when the agents engage in a behaviour of mutual cooperation during the IPD. Previously, we successfully applied reinforcement of stochastic synaptic transmission to the IPD game. The current study utilises reward-modulated STDP with eligibility trace and results show that the system managed to exhibit the desired behaviour by establishing mutual cooperation between the agents. It is noted that the cooperative outcome was attained after a relatively short learning period which enhanced the accumulation of reward by the system. As in our previous implementation, the successful application of the learning algorithm to the IPD becomes possible only after we extended it with additional global reinforcement signals in order to enhance competition at the neuronal level. Moreover it is also shown that learning is enhanced (as indicated by an increased IPD cooperative outcome) through: (i) strong memory for each agent (regulated by a high eligibility trace time constant) and (ii) firing irregularity produced by equipping the agents' LIF neurons with a partial somatic reset mechanism.