Behavioral and electrophysiological indices of negative affect predict cocaine self-administration

The motivation to seek cocaine comes in part from a dysregulation of reward processing manifested in dysphoria, or affective withdrawal. Learning is a critical aspect of drug abuse; however, it remains unclear whether drug-associated cues can elicit the emotional withdrawal symptoms that promote cocaine use. Here we report that a cocaine-associated taste cue elicited a conditioned aversive state that was behaviorally and neurophysiologically quantifiable and predicted subsequent cocaine self-administration behavior. Specifically, brief intraoral infusions of a cocaine-predictive flavored saccharin solution elicited aversive orofacial responses that predicted early-session cocaine taking in rats. The expression of aversive taste reactivity also was associated with a shift in the predominant pattern of electrophysiological activity of nucleus accumbens (NAc) neurons from inhibitory to excitatory. The dynamic nature of this conditioned switch in affect and the neural code reveals a mechanism by which cues may exert control over drug self-administration.     

Effects of development and enculturation on number representation in the brain

A striking way in which humans differ from non-human primates is in their ability to represent numerical quantity using abstract symbols and to use these 'mental tools' to perform skills such as exact calculations. How do functional brain circuits for the symbolic representation of numerical magnitude emerge? Do neural representations of numerical magnitude change as a function of development and the learning of mental arithmetic? Current theories suggest that cultural number symbols acquire their meaning by being mapped onto non-symbolic representations of numerical magnitude. This Review provides an evaluation of this contention and proposes hypotheses to guide investigations into the neural mechanisms that constrain the acquisition of cultural representations of numerical magnitude.     

Learning deep representations of enzyme thermal adaptation

Temperature is a fundamental environmental factor that shapes the evolution of organisms. Learning thermal determinants of protein sequences in evolution thus has profound significance for basic biology, drug discovery, and protein engineering. Here, we use a data set of over 3 million BRENDA enzymes labeled with optimal growth temperatures (OGTs) of their source organisms to train a deep neural network model (DeepET). The protein‐temperature representations learned by DeepET provide a temperature‐related statistical summary of protein sequences and capture structural properties that affect thermal stability. For prediction of enzyme optimal catalytic temperatures and protein melting temperatures via a transfer learning approach, our DeepET model outperforms classical regression models trained on rationally designed features and other deep‐learning‐based representations. DeepET thus holds promise for understanding enzyme thermal adaptation and guiding the engineering of thermostable enzymes.     

Is avoiding an aversive outcome rewarding? Neural substrates of avoidance learning in the human brain

Avoidance learning poses a challenge for reinforcement-based theories of instrumental conditioning, because once an aversive outcome is successfully avoided an individual may no longer experience extrinsic reinforcement for their behavior. One possible account for this is to propose that avoiding an aversive outcome is in itself a reward, and thus avoidance behavior is positively reinforced on each trial when the aversive outcome is successfully avoided. In the present study we aimed to test this possibility by determining whether avoidance of an aversive outcome recruits the same neural circuitry as that elicited by a reward itself. We scanned 16 human participants with functional MRI while they performed an instrumental choice task, in which on each trial they chose from one of two actions in order to either win money or else avoid losing money. Neural activity in a region previously implicated in encoding stimulus reward value, the medial orbitofrontal cortex, was found to increase, not only following receipt of reward, but also following successful avoidance of an aversive outcome. This neural signal may itself act as an intrinsic reward, thereby serving to reinforce actions during instrumental avoidance.     

Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala

Animals and humans learn to approach and acquire pleasant stimuli and to avoid or defend against aversive ones. However, both pleasant and aversive stimuli can elicit arousal and attention, and their salience or intensity increases when they occur by surprise. Thus, adaptive behavior may require that neural circuits compute both stimulus valence--or value--and intensity. To explore how these computations may be implemented, we examined neural responses in the primate amygdala to unexpected reinforcement during learning. Many amygdala neurons responded differently to reinforcement depending upon whether or not it was expected. In some neurons, this modulation occurred only for rewards or aversive stimuli, but not both. In other neurons, expectation similarly modulated responses to both rewards and punishments. These different neuronal populations may subserve two sorts of processes mediated by the amygdala: those activated by surprising reinforcements of both valences-such as enhanced arousal and attention-and those that are valence-specific, such as fear or reward-seeking behavior.     

Aversive learning in honeybees revealed by the olfactory conditioning of the sting extension reflex

Invertebrates have contributed greatly to our understanding of associative learning because they allow learning protocols to be combined with experimental access to the nervous system. The honeybee Apis mellifera constitutes a standard model for the study of appetitive learning and memory since it was shown, almost a century ago, that bees learn to associate different sensory cues with a reward of sugar solution. However, up to now, no study has explored aversive learning in bees in such a way that simultaneous access to its neural bases is granted. Using odorants paired with electric shocks, we conditioned the sting extension reflex, which is exhibited by harnessed bees when subjected to a noxious stimulation. We show that this response can be conditioned so that bees learn to extend their sting in response to the odorant previously punished. Bees also learn to extend the proboscis to one odorant paired with sugar solution and the sting to a different odorant paired with electric shock, thus showing that they can master both appetitive and aversive associations simultaneously. Responding to the appropriate odorant with the appropriate response is possible because two different biogenic amines, octopamine and dopamine subserve appetitive and aversive reinforcement, respectively. While octopamine has been previously shown to substitute for appetitive reinforcement, we demonstrate that blocking of dopaminergic, but not octopaminergic, receptors suppresses aversive learning. Therefore, aversive learning in honeybees can now be accessed both at the behavioral and neural levels, thus opening new research avenues for understanding basic mechanisms of learning and memory.     

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.     

Representational drift as a window into neural and behavioural plasticity

Large-scale recordings of neural activity over days and weeks have revealed that neural representations of familiar tasks, precepts and actions continually evolve without obvious changes in behaviour. We hypothesise that this steady drift in neural activity and accompanying physiological changes is due in part to the continuous application of a learning rule at the cellular and population level. Explicit predictions of this drift can be found in neural network models that use iterative learning to optimise weights. Drift therefore provides a measurable signal that can reveal systems–level properties of biological plasticity mechanisms, such as their precision and effective learning rates.     

Neural fuzzy preference integration using neural preference Moore machines

This paper describes preference classes and preference Moore machines as a basis for integrating different hybrid neural representations. Preference classes are shown to provide a basic link between neural preferences and fuzzy representations at the preference class level. Preference Moore machines provide a link between recurrent neural networks and symbolic transducers at the preference Moore machine level. We demonstrate how the concepts of preference classes and preference Moore machines can be used to interpret neural network representations and to integrate knowledge from hybrid neural representations. One main contribution of this paper is the introduction and analysis of neural preference Moore machines and their link to a fuzzy interpretation. Furthermore, we illustrate the interpretation and combination of various neural preference Moore machines with additional real-world examples.     

Amygdala-prefrontal synchronization underlies resistance to extinction of aversive memories

Emotional memories can persist for a lifetime but can also undergo extinction. Although we know about the mechanisms involved in expression and extinction, we know very little about the mechanisms that determine whether a specific memory would persist or not. Here, we use partial reinforcement extinction effect (PREE) to explore the neural mechanisms that differentiate persistent from labile memories. We recorded the simultaneous activity of neurons in the amygdala and the dorsal anterior cingulate cortex (dACC) while monkeys engaged in tone-odor aversive conditioning. We report that under continuous reinforcement schedule (ConS), activity in the amygdala precedes behavioral response, whereas under partial schedule (ParS), dACC activity precedes it. Moreover, we find that ParS induced cross-regional pairwise correlations throughout the entire acquisition session, and their magnitude and precision predicted the later resistance to extinction. Our results suggest that memory persistence depends on distributed representations and, specifically, resistance to extinction of aversive memories is maintained by correlated amygdala-dACC activity.     

Fear memory and the amygdala: insights from a molecular perspective

The amygdala modulates memory consolidation and the storage of emotionally relevant information in other brain areas, and itself comprises a site of neural plasticity during aversive learning. These processes have been intensively studied in Pavlovian fear conditioning, a leading aversive learning paradigm that is dependent on the structural and functional integrity of the amygdala. The rapidness and persistence, and the relative ease, with which this conditioning paradigm can be applied to a great variety of species have made it an attractive model for neurochemical and electrophysiological investigations on memory formation. In this review we summarise recent studies which have begun to unravel cellular processes in the amygdala that are critical for the formation of long-term fear memory and have identified molecular factors and mechanisms of neural plasticity in this brain area.     

The influence of serotonin on fear learning

Learning of associations between aversive stimuli and predictive cues is the basis of Pavlovian fear conditioning and is driven by a mismatch between expectation and outcome. To investigate whether serotonin modulates the formation of such aversive cue-outcome associations, we used functional magnetic resonance imaging (fMRI) and dietary tryptophan depletion to reduce brain serotonin (5-HT) levels in healthy human subjects. In a Pavlovian fear conditioning paradigm, 5-HT depleted subjects compared to a non-depleted control group exhibited attenuated autonomic responses to cues indicating the upcoming of an aversive event. These results were closely paralleled by reduced aversive learning signals in the amygdala and the orbitofrontal cortex, two prominent structures of the neural fear circuit. In agreement with current theories of serotonin as a motivational opponent system to dopamine in fear learning, our data provide first empirical evidence for a role of serotonin in representing formally derived learning signals for aversive events.     

Genes, odours and the recognition of parasitized individuals by rodents

Social recognition, whereby animals identify and recognize other individual conspecifics, is a crucial prerequisite for a wide range of social behaviours. There are relationships among social odours (chemical signals), parasite recognition and avoidance that are associated with hormonal, neural and genomic mechanisms in rodents. Rodents use social odours to: (i) distinguish between infected and uninfected individuals; (ii) recognize specific infected individuals; and (iii) avoid and display aversive responses to infected individuals. There are genomic correlates of this parasite recognition and avoidance in which genes expressing the neuropeptide oxytocin have roles. In this article, we provide a framework ("micronet") by which the genetic, hormonal and neural interactions associated with social behaviours and recognition and avoidance of parasitized individuals can be explored.     

A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior

A neural network model of how dopamine and prefrontal cortex activity guides short- and long-term information processing within the cortico-striatal circuits during reward-related learning of approach behavior is proposed. The model predicts two types of reward-related neuronal responses generated during learning: (1) cell activity signaling errors in the prediction of the expected time of reward delivery and (2) neural activations coding for errors in the prediction of the amount and type of reward or stimulus expectancies. The former type of signal is consistent with the responses of dopaminergic neurons, while the latter signal is consistent with reward expectancy responses reported in the prefrontal cortex. It is shown that a neural network architecture that satisfies the design principles of the adaptive resonance theory of Carpenter and Grossberg (1987) can account for the dopamine responses to novelty, generalization, and discrimination of appetitive and aversive stimuli. These hypotheses are scrutinized via simulations of the model in relation to the delivery of free food outside a task, the timed contingent delivery of appetitive and aversive stimuli, and an asymmetric, instructed delay response task.     

Learning,fast and slow

Animals can learn efficiently from a single experience and change their future behavior in response. However, in other instances, animals learn very slowly, requiring thousands of experiences. Here, I survey tasks involving fast and slow learning and consider some hypotheses for what differentiates the underlying neural mechanisms. It has been proposed that fast learning relies on neural representations that favor efficient Hebbian modification of synapses. These efficient representations may be encoded in the genome, resulting in a repertoire of fast learning that differs across species. Alternatively, the required neural representations may be acquired from experience through a slow process of unsupervised learning from the environment.     

听觉皮层信号处理

听觉系统和视觉系统的不同之处在于：听觉系统在外周感受器和听皮层间具有更长的皮层下通路和更多的突触联系。该特殊结构反应了听觉系统从复杂听觉环境中提取与行为相关信号的机制与其他感觉系统不同。听皮层神经信号处理包括两种重要的转换机制,声音信号的非同构转换以及从声音感受到知觉层面的转换。听觉皮层神经编码机制同时也受到听觉反馈和语言或发声过程中发声信号的调控。听觉神经科学家和生物医学工程师所面临的挑战便是如何去理解大脑中这些转换的编码机制。我将会用我实验室最近的一些发现来阐述听觉信号是如何在原听皮层中进行处理的,并讨论其对于言语和音乐在大脑中的处理机制以及设计神经替代装置诸如电子耳蜗的意义。我们使用了结合神经电生理技术和量化工程学的方法来研究这些问题。     

Abstract and associatively based representations in human sequence learning

We give an analysis of performance in an artificial neural network for which the claim had been made that it could learn abstract representations. Our argument is that this network is associative in nature, and cannot develop abstract representations. The network thus converges to a solution that is solely based on the statistical regularities of the training set. Inspired by human experiments that have shown that humans can engage in both associative (statistical) and abstract learning, we present a new, hybrid computational model that combines associative and more abstract, cognitive processes. To cross-validate the model we attempted to predict human behaviour in further experiments. One of these experiments reveals some evidence for the use of abstract representations, whereas the others provide evidence for associatively based performance. The predictions of the hybrid model stand in line with our empirical data.     

Neural representation of visual objects: encoding and top-down activation

Knowledge or experiences are voluntarily recalled from memory by reactivation of their neural representations in the association cortex. Mnemonic representations of visual objects, located in the ventral processing stream of visual perception, provide the best indication of how neuronal codes are created, organized and reactivated. Associative codes are created by neurons that have the ability to link the representations of temporally associated stimuli. Recent experiments suggest that not only bottom-up signals from the retina but also top-down signals from the prefrontal cortex can trigger the retrieval of associative codes, which may serve as a neural basis for conscious recall.     

Neurobiology of addictive behavior

Addictive behavior developes after repeated substance use and it typically include a strong desire to take the drug, difficulties in controlling its use, persisting in its use despite harmful consequences, a higher priority given to the drug use than to other activities. Relapse, the resumption of drug taking after periods of abstinence, remains the major problem for the treatment of addiction. The process of drug addiction shares striking commonalities with neural plasticity associated with natural reward learning and memory and is caused primarily by drug-induced sensitization in the brain mesocorticolimbic systems that attribute incentive salience to reward-associated stimuli. The switch from controlled to compulsive drug seeking represents a transition at the neural level from prefrontal cortical to striatal control. Current neurophysiologic evidence suggests that the development of addiction is to some extent due to neurochemical stimulation of the midbrain dopaminergic system that is traditionally considered as a 'common neural currency' for rewards of most kinds. Addictions are a result of the interplay of multiple genetic and environmental factors. They are characterized by phenotypic and genetic heterogeneity as well as polygenicity. Environmental factors are crucial in addiction vulnerability and resistese too.