Cortical mechanisms for reinforcement learning in competitive games

Game theory analyses optimal strategies for multiple decision makers interacting in a social group. However, the behaviours of individual humans and animals often deviate systematically from the optimal strategies described by game theory. The behaviours of rhesus monkeys (Macaca mulatta) in simple zero-sum games showed similar patterns, but their departures from the optimal strategies were well accounted for by a simple reinforcement-learning algorithm. During a computer-simulated zero-sum game, neurons in the dorsolateral prefrontal cortex often encoded the previous choices of the animal and its opponent as well as the animal's reward history. By contrast, the neurons in the anterior cingulate cortex predominantly encoded the animal's reward history. Using simple competitive games, therefore, we have demonstrated functional specialization between different areas of the primate frontal cortex involved in outcome monitoring and action selection. Temporally extended signals related to the animal's previous choices might facilitate the association between choices and their delayed outcomes, whereas information about the choices of the opponent might be used to estimate the reward expected from a particular action. Finally, signals related to the reward history might be used to monitor the overall success of the animal's current decision-making strategy.     

Experimental analysis of the territorial establishment of ayu, <Emphasis Type="Italic">Plecoglossus altivelis</Emphasis>

Ayu mainly grazes benthic algae on river beds. To clarify the effects of individual behavioral differences on the territory establishment of ayu, we first recorded the number of active movements, aggression towards a wooden ayu model, and scraping feeding activities for 40 ayu, each in an aquarium with a ceramic plate with benthic algae. We then released 10 randomly chosen ayu and other cyprinids, 10 Tribolodon hakonensis and 10 Zacco platypus, into each of the four ponds with recirculating water. Four to six ayu established a territory for 1–20 of the 21days observed in each pond. Territories were unstable in the first half period but became stable in the latter period. Territorial period was positively correlated with the number of active movements in aquaria and negatively with the number of conspecific invasions. The percentage of attacks against other species was positively correlated with the number of scraping behavior in aquaria and negatively with standard length of ayu. The growth rate of non-territorial ayu was large in case the ayu scraped more than half of ceramic plates in aquaria. In contrast, the growth rate of territorial ayu was correlated negatively with the standard length of ayu, and the percentage of attacks against other species, indicating that individual behavioral differences and the defending cost were both important in territory establishment and growth.     

Defining the biological bases of individual differences in musicality

Advances in molecular technologies make it possible to pinpoint genomic factors associated with complex human traits. For cognition and behaviour, identification of underlying genes provides new entry points for deciphering the key neurobiological pathways. In the past decade, the search for genetic correlates of musicality has gained traction. Reports have documented familial clustering for different extremes of ability, including amusia and absolute pitch (AP), with twin studies demonstrating high heritability for some music-related skills, such as pitch perception. Certain chromosomal regions have been linked to AP and musical aptitude, while individual candidate genes have been investigated in relation to aptitude and creativity. Most recently, researchers in this field started performing genome-wide association scans. Thus far, studies have been hampered by relatively small sample sizes and limitations in defining components of musicality, including an emphasis on skills that can only be assessed in trained musicians. With opportunities to administer standardized aptitude tests online, systematic large-scale assessment of musical abilities is now feasible, an important step towards high-powered genome-wide screens. Here, we offer a synthesis of existing literatures and outline concrete suggestions for the development of comprehensive operational tools for the analysis of musical phenotypes.     

Differential contribution of storage pools to the extracellular amount of accumbal dopamine in high and low responders to novelty: effects of reserpine

The present study examined the effects of reserpine on the extracellular concentration of accumbal dopamine in high responders (HR) and low responders (LR) to novelty rats. Reserpine reduced the baseline concentration of extracellular accumbal dopamine more in HR than in LR, indicating that the dopamine release is more dependent on reserpine-sensitive storage vesicles in non-challenged HR than in non-challenged LR. In addition, reserpine reduced the novelty-induced increase of the extracellular concentration of accumbal dopamine in LR, but not in HR, indicating that the dopamine release in response to novelty depends on reserpine-sensitive storage vesicles only in LR, not in HR. Our data clearly demonstrate that HR and LR differ in the characteristics of those monoaminergic storage vesicles that mediate accumbal dopamine release.     

The human amygdala and orbital prefrontal cortex in behavioural regulation

Survival in complex environments depends on an ability to optimize future behaviour based on past experience. Learning from experience enables an organism to generate predictive expectancies regarding probable future states of the world, enabling deployment of flexible behavioural strategies. However, behavioural flexibility cannot rely on predictive expectancies alone and options for action need to be deployed in a manner that is responsive to a changing environment. Important moderators on learning-based predictions include those provided by context and inputs regarding an organism's current state, including its physiological state. In this paper, I consider human experimental approaches using functional magnetic resonance imaging that have addressed the role of the amygdala and prefrontal cortex (PFC), in particular the orbital PFC, in acquiring predictive information regarding the probable value of future events, updating this information, and shaping behaviour and decision processes on the basis of these value representations.     

Introduction. Neuroeconomics: the promise and the profit

Neuroeconomics investigates the neural mechanisms underlying decisions about rewarding or punishing outcomes ('economic' decisions). It combines the knowledge about the behavioural phenomena of economic decisions with the mechanistic explanatory power of neuroscience. Thus, it is about the neurobiological foundations of economic decision making. It is hoped that by 'opening the box' we can understand how decisions about gains and losses are directed by the brain of the individual decision maker. Perhaps we can even learn why some decisions are apparently paradoxical or pathological. The knowledge could be used to create situations that avoid suboptimal decisions and harm.     

A fruit in the hand or two in the bush? Divergent risk preferences in chimpanzees and bonobos

Human and non-human animals tend to avoid risky prospects. If such patterns of economic choice are adaptive, risk preferences should reflect the typical decision-making environments faced by organisms. However, this approach has not been widely used to examine the risk sensitivity in closely related species with different ecologies. Here, we experimentally examined risk-sensitive behaviour in chimpanzees (Pan troglodytes) and bonobos (Pan paniscus), closely related species whose distinct ecologies are thought to be the major selective force shaping their unique behavioural repertoires. Because chimpanzees exploit riskier food sources in the wild, we predicted that they would exhibit greater tolerance for risk in choices about food. Results confirmed this prediction: chimpanzees significantly preferred the risky option, whereas bonobos preferred the fixed option. These results provide a relatively rare example of risk-prone behaviour in the context of gains and show how ecological pressures can sculpt economic decision making.