An association between prediction errors and risk-seeking: Theory and behavioral evidence

Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.     

Anticipatory affect: neural correlates and consequences for choice

'Anticipatory affect' refers to emotional states that people experience while anticipating significant outcomes. Historically, technical limitations have made it difficult to determine whether anticipatory affect influences subsequent choice. Recent advances in the spatio-temporal resolution of functional magnetic resonance imaging, however, now allow researchers to visualize changes in neural activity seconds before choice occurs. We review evidence that activation in specific brain circuits changes during anticipation of monetary incentives, that this activation correlates with affective experience and that activity in these circuits may influence subsequent choice. Specifically, an activation likelihood estimate meta-analysis of cued response studies indicates that nucleus accumbens (NAcc) activation increases during gain anticipation relative to loss anticipation, while anterior insula activation increases during both loss and gain anticipation. Additionally, anticipatory NAcc activation correlates with self-reported positive arousal, whereas anterior insula activation correlates with both self-reported negative and positive arousal. Finally, NAcc activation precedes the purchase of desirable products and choice of high-risk gambles, whereas anterior insula activation precedes the rejection of overpriced products and choice of low-risk gambles. Together, these findings support a neurally plausible framework for understanding how anticipatory affect can influence choice.     

Isolating neural correlates of the pacemaker for food anticipation

Mice fed a single daily meal at intervals within the circadian range exhibit food anticipatory activity. Previous investigations strongly suggest that this behaviour is regulated by a circadian pacemaker entrained to the timing of fasting/refeeding. The neural correlate(s) of this pacemaker, the food entrainable oscillator (FEO), whether found in a neural network or a single locus, remain unknown. This study used a canonical property of circadian pacemakers, the ability to continue oscillating after removal of the entraining stimulus, to isolate activation within the neural correlates of food entrainable oscillator from all other mechanisms driving food anticipatory activity. It was hypothesized that continued anticipatory activation of central nuclei, after restricted feeding and a return to ad libitum feeding, would elucidate a neural representation of the signaling circuits responsible for the timekeeping component of the food entrainable oscillator. Animals were entrained to a temporally constrained meal then placed back on ad libitum feeding for several days until food anticipatory activity was abolished. Activation of nuclei throughout the brain was quantified using stereological analysis of c-FOS expressing cells and compared against both ad libitum fed and food entrained controls. Several hypothalamic and brainstem nuclei remained activated at the previous time of food anticipation, implicating them in the timekeeping mechanism necessary to track previous meal presentation. This study also provides a proof of concept for an experimental paradigm useful to further investigate the anatomical and molecular substrates of the FEO.     

Neural predictors of purchases

Microeconomic theory maintains that purchases are driven by a combination of consumer preference and price. Using event-related fMRI, we investigated how people weigh these factors to make purchasing decisions. Consistent with neuroimaging evidence suggesting that distinct circuits anticipate gain and loss, product preference activated the nucleus accumbens (NAcc), while excessive prices activated the insula and deactivated the mesial prefrontal cortex (MPFC) prior to the purchase decision. Activity from each of these regions independently predicted immediately subsequent purchases above and beyond self-report variables. These findings suggest that activation of distinct neural circuits related to anticipatory affect precedes and supports consumers' purchasing decisions.     

Developmental regulation of axon branching in the vertebrate nervous system

During nervous system development, axons generate branches to connect with multiple synaptic targets. As with axon growth and guidance, axon branching is tightly controlled in order to establish functional neural circuits, yet the mechanisms that regulate this important process are less well understood. Here, we review recent advances in the study of several common branching processes in the vertebrate nervous system. By focusing on each step in these processes we illustrate how different types of branching are regulated by extracellular cues and neural activity, and highlight some common principles that underlie the establishment of complex neural circuits in vertebrate development.     

Subjective optimality in finite sequential decision-making

Many decisions in life are sequential and constrained by a time window. Although mathematically derived optimal solutions exist, it has been reported that humans often deviate from making optimal choices. Here, we used a secretary problem, a classic example of finite sequential decision-making, and investigated the mechanisms underlying individuals’ suboptimal choices. Across three independent experiments, we found that a dynamic programming model comprising subjective value function explains individuals’ deviations from optimality and predicts the choice behaviors under fewer and more opportunities. We further identified that pupil dilation reflected the levels of decision difficulty and subsequent choices to accept or reject the stimulus at each opportunity. The value sensitivity, a model-based estimate that characterizes each individual’s subjective valuation, correlated with the extent to which individuals’ physiological responses tracked stimuli information. Our results provide model-based and physiological evidence for subjective valuation in finite sequential decision-making, rediscovering human suboptimality in subjectively optimal decision-making processes.     

Voting Intention and Choices: Are Voters Always Rational and Deliberative?

Human rationality–the ability to behave in order to maximize the achievement of their presumed goals (i.e., their optimal choices)–is the foundation for democracy. Research evidence has suggested that voters may not make decisions after exhaustively processing relevant information; instead, our decision-making capacity may be restricted by our own biases and the environment. In this paper, we investigate the extent to which humans in a democratic society can be rational when making decisions in a serious, complex situation–voting in a local political election. We believe examining human rationality in a political election is important, because a well-functioning democracy rests largely upon the rational choices of individual voters. Previous research has shown that explicit political attitudes predict voting intention and choices (i.e., actual votes) in democratic societies, indicating that people are able to reason comprehensively when making voting decisions. Other work, though, has demonstrated that the attitudes of which we may not be aware, such as our implicit (e.g., subconscious) preferences, can predict voting choices, which may question the well-functioning democracy. In this study, we systematically examined predictors on voting intention and choices in the 2014 mayoral election in Taipei, Taiwan. Results indicate that explicit political party preferences had the largest impact on voting intention and choices. Moreover, implicit political party preferences interacted with explicit political party preferences in accounting for voting intention, and in turn predicted voting choices. Ethnic identity and perceived voting intention of significant others were found to predict voting choices, but not voting intention. In sum, to the comfort of democracy, voters appeared to engage mainly explicit, controlled processes in making their decisions; but findings on ethnic identity and perceived voting intention of significant others may suggest otherwise.     

Fear, anxiety, and control in the zebrafish

Emotional responses are triggered by environmental signals and involve profound changes at multiple levels, from molecular to behavior. Much has been learnt about two emotions, fear and anxiety, by studying mammalian models. In particular, neural circuits and the corresponding molecular mechanisms essential for the learning and retention of fear, as well as the activation of anxiety, are well known. In contrast, little is known about how these emotions are terminated. The zebrafish is a newcomer to the world of emotion research. A number of assays for fear and anxiety now exist, but the underlying neural circuitry is largely undefined. Recent experiments, however, appear to provide a hint as to how anxiety is downregulated. In particular, they point to an essential role for a circuit involving the posterior septum, medial habenula, and interpeduncular nucleus. This evolutionarily conserved circuit may fulfill a similar function in mammals.     

Slow manifolds within network dynamics encode working memory efficiently and robustly

Working memory is a cognitive function involving the storage and manipulation of latent information over brief intervals of time, thus making it crucial for context-dependent computation. Here, we use a top-down modeling approach to examine network-level mechanisms of working memory, an enigmatic issue and central topic of study in neuroscience. We optimize thousands of recurrent rate-based neural networks on a working memory task and then perform dynamical systems analysis on the ensuing optimized networks, wherein we find that four distinct dynamical mechanisms can emerge. In particular, we show the prevalence of a mechanism in which memories are encoded along slow stable manifolds in the network state space, leading to a phasic neuronal activation profile during memory periods. In contrast to mechanisms in which memories are directly encoded at stable attractors, these networks naturally forget stimuli over time. Despite this seeming functional disadvantage, they are more efficient in terms of how they leverage their attractor landscape and paradoxically, are considerably more robust to noise. Our results provide new hypotheses regarding how working memory function may be encoded within the dynamics of neural circuits.     

Neural basis of quasi-rational decision making

Standard economic theories conceive homo economicus as a rational decision maker capable of maximizing utility. In reality, however, people tend to approximate optimal decision-making strategies through a collection of heuristic routines. Some of these routines are driven by emotional processes, and others are adjusted iteratively through experience. In addition, routines specialized for social decision making, such as inference about the mental states of other decision makers, might share their origins and neural mechanisms with the ability to simulate or imagine outcomes expected from alternative actions that an individual can take. A recent surge of collaborations across economics, psychology and neuroscience has provided new insights into how such multiple elements of decision making interact in the brain.     

GABAA Receptor-Mediated Tonic Depolarization in Developing Neural Circuits

The activation of GABA_A receptors (the type A receptors for γ-aminobutyric acid) produces two distinct forms of responses, phasic (i.e., transient) and tonic (i.e., persistent), that are mediated by synaptic and extrasynaptic GABA_A receptors, respectively. During development, the intracellular chloride levels are high so activation of these receptors causes a net outward flow of anions that leads to neuronal depolarization rather than hyperpolarization. Therefore, in developing neural circuits, tonic activation of GABA_A receptors may provide persistent depolarization. Recently, it became evident that GABA_A receptor-mediated tonic depolarization alters the structure of patterned spontaneous activity, a feature that is common in developing neural circuits and is important for neural circuit refinement. Thus, this persistent depolarization may lead to a long-lasting increase in intracellular calcium level that modulates network properties via calcium-dependent signaling cascades. This article highlights the features of GABA_A receptor-mediated tonic depolarization, summarizes the principles for discovery, reviews the current findings in diverse developing circuits, examines the underlying molecular mechanisms and modulation systems, and discusses their functional specializations for each developing neural circuit.     

Activity-induced long-term potentiation of excitatory synapses in developing zebrafish retina in vivo

Neural activity-induced long-term potentiation (LTP) of synaptic transmission is believed to be one of the cellular mechanisms underlying experience-dependent developmental refinement of neural circuits. Although it is well established that visual experience and neural activity are critical for the refinement of retinal circuits, whether and how LTP occurs in the retina remain unknown. Using in?vivo perforated whole-cell recording and two-photon calcium imaging, we find that both repeated electrical and visual stimulations can induce LTP at excitatory synapses formed by bipolar cells on retinal ganglion cells in larval but not juvenile zebrafish. LTP induction requires the activation of postsynaptic N-methyl-D-aspartate receptors, and its expression involves arachidonic acid-dependent presynaptic changes in calcium dynamics and neurotransmitter release. Physiologically, both electrical and visual stimulation-induced LTP can enhance visual responses of retinal ganglion cells. Thus, LTP exists in developing retinae with a presynaptic locus and may serve for visual experience-dependent refinement of retinal circuits.     

Human economic and financial behavior: The serotonergic hypothesis

Recent advances in the field of neuroeconomics and behavioral finance have shed new light on the biological correlates of human economic and financial behavior. In this context, a reduced serotonergic activity has been consistently linked to an increased rate of rejection of unfair offers in the ultimatum game (UG), a simple two-person bargaining between a proposer and a responder. Besides serotonin, increased testosterone levels have been associated to rejections of unfair UG offers, as well as to higher financial gains among professional traders operating in the London stock market. Since low serotonin and high testosterone levels in the central nervous system may interact to exert significant effects on the neural mechanisms involved in the expression of impulsivity and aggressive behavior, it is feasible to hypothesize that serotonergic neurotransmission might exert an important influence on investors' choices in real-world financial markets. Future studies in this area should explore whether tryptophan depletion may actually improve (or diminish) investors' trading performance.     

Autoradiographic measurements of protein synthesis in hippocampal slices from rats and guinea pigs.

Protein synthesis is an extremely important cell function and there is now good evidence that changes in synthesis play important roles both in neuronal cell damage from ischemic insults and in neural plasticity though the mechanisms of these effects are not at all clear. The brain slice, and particularly the hippocampal slice, is an excellent preparation for studying these effects although, as with all studies on slices, caution must be exercised in that regulation in the slice may be different from regulation in vivo. Studies on neural tissue need to take into account the heterogeneity of neural tissue as well as the very different compartments within neurons. Autoradiography at both the light and electron microscope levels is a very powerful method for doing this. Successful autoradiography depends on many factors. These include correct choice of precursor amino acid, mechanisms for estimating changes in the specific activity of the precursor amino acid pool, and reliable methods for quantitation of the autoradiographs. At a more technical level these factors include attention to detail in processing tissue sections so as to avoid light contamination during exposure and developing and, also, appropriate choices of the various parameters such as exposure time and section thickness. The power of autoradiography is illustrated here by its ability to discern effects of ischemia and of plasticity-related neural input on distinct cell types and also in distinct compartments of neurons. Ischemia inhibits protein synthesis in principal neurons but activates synthesis in other cell types of the brain slice. Plasticity-related neural input immediately enhances protein synthesis in dendrites but does not affect cell bodies.     

Smelling on the fly: sensory cues and strategies for olfactory navigation in Drosophila

Navigating toward (or away from) a remote odor source is a challenging problem that requires integrating olfactory information with visual and mechanosensory cues. Drosophila melanogaster is a useful organism for studying the neural mechanisms of these navigation behaviors. There are a wealth of genetic tools in this organism, as well as a history of inventive behavioral experiments. There is also a large and growing literature in Drosophila on the neural coding of olfactory, visual, and mechanosensory stimuli. Here we review recent progress in understanding how these stimulus modalities are encoded in the Drosophila nervous system. We also discuss what strategies a fly might use to navigate in a natural olfactory landscape while making use of all these sources of sensory information. We emphasize that Drosophila are likely to switch between multiple strategies for olfactory navigation, depending on the availability of various sensory cues. Finally, we highlight future research directions that will be important in understanding the neural circuits that underlie these behaviors.     

Neural control of behavioural choice in juvenile crayfish

Natural selection leads to behavioural choices that increase the animal''s fitness. The neuronal mechanisms underlying behavioural choice are still elusive and empirical evidence connecting neural circuit activation to adaptive behavioural output is sparse.We exposed foraging juvenile crayfish to approaching shadows of different velocities and found that slow-moving shadows predominantly activated a pair of giant interneurons, which mediate tail-flips that thrust the animals backwards and away from the approaching threat. Tail-flips also moved the animals farther away from an expected food source, and crayfish defaulted to freezing behaviour when faced with fast-approaching shadows. Under these conditions, tail-flipping, an ineffective and costly escape strategy was suppressed in favour of freezing, a more beneficial choice. The decision to freeze also dominated in the presence of a more desirable resource; however, the increased incentive was less effective in suppressing tail-flipping when paired with slow-moving visual stimuli that reliably evoked tail-flips in most animals. Together this suggests that crayfish make value-based decisions by weighing the costs and benefits of different behavioural options, and they select adaptive behavioural output based on the activation patterns of identifiable neural circuits.     

Early postural adjustments in preparation to whole-body voluntary sway

We studied postural adjustments associated with a quick voluntary postural sway under two conditions, self-paced and simple reaction-time. Standing subjects were required to produce quick discrete shifts of the center of pressure (COP) forward. About 400-500 ms prior to the instructed COP shift, there were deviations of the COP in the opposite direction (backwards) accompanied by changes in the activation levels of several postural muscles. Under the reaction-time conditions, the timing of those early postural adjustments did not change (repeated measures MANOVA: p > 0.05) while its magnitude increased significantly (confirmed by repeated measures MANOVA: p < 0.05). These observations are opposite to those reported for anticipatory postural adjustments under simple reaction time conditions (a significant change in the timing without major changes in the magnitude). We conclude that there are two types of feed-forward postural adjustments. Early postural adjustments prepare the body for the planned action and/or expected perturbation. Some of these preparatory actions may be mechanically necessary. Later, anticipatory postural adjustments generate net forces and moments of force acting against those associated with the expected perturbation. Both types of adjustments fit well the referent configuration hypothesis, which offers a unified view on movement-posture control.