Learning, working memory, and intelligence revisited

Based on early findings showing low correlations between intelligence test scores and learning on laboratory tasks, psychologists typically have dismissed the role of learning in intelligence and emphasized the role of working memory instead. In 2006, however, B.A. Williams developed a verbal learning task inspired by three-term reinforcement contingencies and reported unexpectedly high correlations between this task and Raven's Advanced Progressive Matrices (RAPM) scores [Williams, B.A., Pearlberg, S.L., 2006. Learning of three-term contingencies correlates with Raven scores, but not with measures of cognitive processing. Intelligence 34, 177-191]. The present study replicated this finding: Performance on the three-term learning task explained almost 25% of the variance in RAPM scores. Adding complex verbal working memory span, measured using the operation span task, did not improve prediction. Notably, this was not due to a lack of correlation between complex working memory span and RAPM scores. Rather, it occurred because most of the variance captured by the complex working memory span was already accounted for by the three-term learning task. Taken together with the findings of Williams and Pearlberg, the present results make a strong case for the role of learning in performance on intelligence tests.     

A Spiking Neural Network Model of Model-Free Reinforcement Learning with High-Dimensional Sensory Input and Perceptual Ambiguity

A theoretical framework of reinforcement learning plays an important role in understanding action selection in animals. Spiking neural networks provide a theoretically grounded means to test computational hypotheses on neurally plausible algorithms of reinforcement learning through numerical simulation. However, most of these models cannot handle observations which are noisy, or occurred in the past, even though these are inevitable and constraining features of learning in real environments. This class of problem is formally known as partially observable reinforcement learning (PORL) problems. It provides a generalization of reinforcement learning to partially observable domains. In addition, observations in the real world tend to be rich and high-dimensional. In this work, we use a spiking neural network model to approximate the free energy of a restricted Boltzmann machine and apply it to the solution of PORL problems with high-dimensional observations. Our spiking network model solves maze tasks with perceptually ambiguous high-dimensional observations without knowledge of the true environment. An extended model with working memory also solves history-dependent tasks. The way spiking neural networks handle PORL problems may provide a glimpse into the underlying laws of neural information processing which can only be discovered through such a top-down approach.     

Auditory Discrimination Learning: Role of Working Memory

Perceptual training is generally assumed to improve perception by modifying the encoding or decoding of sensory information. However, this assumption is incompatible with recent demonstrations that transfer of learning can be enhanced by across-trial variation of training stimuli or task. Here we present three lines of evidence from healthy adults in support of the idea that the enhanced transfer of auditory discrimination learning is mediated by working memory (WM). First, the ability to discriminate small differences in tone frequency or duration was correlated with WM measured with a tone n-back task. Second, training frequency discrimination around a variable frequency transferred to and from WM learning, but training around a fixed frequency did not. The transfer of learning in both directions was correlated with a reduction of the influence of stimulus variation in the discrimination task, linking WM and its improvement to across-trial stimulus interaction in auditory discrimination. Third, while WM training transferred broadly to other WM and auditory discrimination tasks, variable-frequency training on duration discrimination did not improve WM, indicating that stimulus variation challenges and trains WM only if the task demands stimulus updating in the varied dimension. The results provide empirical evidence as well as a theoretic framework for interactions between cognitive and sensory plasticity during perceptual experience.     

Spatio-temporal credit assignment in neuronal population learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain.     

Episodic-like memory in animals: psychological criteria, neural mechanisms and the value of episodic-like tasks to investigate animal models of neurodegenerative disease

The question of whether any non-human species displays episodic memory is controversial. Associative accounts of animal learning recognize that behaviour can change in response to single events but this does not imply that animals need or are later able to recall representations of unique events at a different time and place. The lack of language is also relevant, being the usual medium for communicating about the world, but whether it is critical for the capacity to represent and recall events is a separate matter. One reason for suspecting that certain animals possess an episodic-like memory system is that a variety of learning and memory tasks have been developed that, even though they do not meet the strict criteria required for episodic memory, have an 'episodic-like' character. These include certain one-trial learning tasks, scene-specific discrimination learning, multiple reversal learning, delayed matching and non-matching tasks and, most recently, tasks demanding recollection of 'what, where and when' an event happened. Another reason is that the neuronal architecture of brain areas thought to be involved in episodic memory (including the hippocampal formation) are substantially similar in mammals and, arguably, all vertebrates. Third, our developing understanding of activity-dependent synaptic plasticity (which is a candidate neuronal mechanism for encoding memory traces) suggests that its expression reflects certain physiological characteristics that are ideal components of a neuronal episodic memory system. These include the apparently digital character of synaptic change at individual terminals and the variable persistence of potentiation accounted for by the synaptic tag hypothesis. A further value of studying episodic-like memory in animals is the opportunity it affords to model certain kinds of neurodegenerative disease that, in humans, affect episodic memory. An example is recent work on a transgenic mouse that over-expresses a mutation of human amyloid precursor protein (APP) that occurs in familial Alzheimer's disease, under the control of platelet derived (PD) growth factor promoter (the PDAPP mouse). A striking age- and amyloid plaque-related deficit is seen using a task in which the mice have to keep changing their memory representation of the world rather than learn a single fact.     

Dynamics of population code for working memory in the prefrontal cortex

Some neurons (delay cells) in the prefrontal cortex elevate their activities throughout the time period during which the animal is required to remember past events and prepare future behavior, suggesting that working memory is mediated by continuous neural activity. It is unknown, however, how working memory is represented within a population of prefrontal cortical neurons. We recorded from neuronal ensembles in the prefrontal cortex as rats learned a new delayed alternation task. Ensemble activities changed in parallel with behavioral learning so that they increasingly allowed correct decoding of previous and future goal choices. In well-trained rats, considerable decoding was possible based on only a few neurons and after removing continuously active delay cells. These results show that neural activity in the prefrontal cortex changes dynamically during new task learning so that working memory is robustly represented and that working memory can be mediated by sequential activation of different neural populations.     

Models of heterogeneous dopamine signaling in an insect learning and memory center

The Drosophila mushroom body exhibits dopamine dependent synaptic plasticity that underlies the acquisition of associative memories. Recordings of dopamine neurons in this system have identified signals related to external reinforcement such as reward and punishment. However, other factors including locomotion, novelty, reward expectation, and internal state have also recently been shown to modulate dopamine neurons. This heterogeneity is at odds with typical modeling approaches in which these neurons are assumed to encode a global, scalar error signal. How is dopamine dependent plasticity coordinated in the presence of such heterogeneity? We develop a modeling approach that infers a pattern of dopamine activity sufficient to solve defined behavioral tasks, given architectural constraints informed by knowledge of mushroom body circuitry. Model dopamine neurons exhibit diverse tuning to task parameters while nonetheless producing coherent learned behaviors. Notably, reward prediction error emerges as a mode of population activity distributed across these neurons. Our results provide a mechanistic framework that accounts for the heterogeneity of dopamine activity during learning and behavior.     

Modular inverse reinforcement learning for visuomotor behavior

In a large variety of situations one would like to have an expressive and accurate model of observed animal or human behavior. While general purpose mathematical models may capture successfully properties of observed behavior, it is desirable to root models in biological facts. Because of ample empirical evidence for reward-based learning in visuomotor tasks, we use a computational model based on the assumption that the observed agent is balancing the costs and benefits of its behavior to meet its goals. This leads to using the framework of reinforcement learning, which additionally provides well-established algorithms for learning of visuomotor task solutions. To quantify the agent’s goals as rewards implicit in the observed behavior, we propose to use inverse reinforcement learning, which quantifies the agent’s goals as rewards implicit in the observed behavior. Based on the assumption of a modular cognitive architecture, we introduce a modular inverse reinforcement learning algorithm that estimates the relative reward contributions of the component tasks in navigation, consisting of following a path while avoiding obstacles and approaching targets. It is shown how to recover the component reward weights for individual tasks and that variability in observed trajectories can be explained succinctly through behavioral goals. It is demonstrated through simulations that good estimates can be obtained already with modest amounts of observation data, which in turn allows the prediction of behavior in novel configurations.     

Controlling attention to nociceptive stimuli with working memory

Background

Because pain often signals the occurrence of potential tissue damage, a nociceptive stimulus has the capacity to involuntarily capture attention and take priority over other sensory inputs. Whether distraction by nociception actually occurs may depend upon the cognitive characteristics of the ongoing activities. The present study tested the role of working memory in controlling the attentional capture by nociception.

Methodology and Principal Findings

Participants performed visual discrimination and matching tasks in which visual targets were shortly preceded by a tactile distracter. The two tasks were chosen because of the different effects the involvement of working memory produces on performance, in order to dissociate the specific role of working memory in the control of attention from the effect of general resource demands. Occasionally (i.e. 17% of the trials), tactile distracters were replaced by a novel nociceptive stimulus in order to distract participants from the visual tasks. Indeed, in the control conditions (no working memory), reaction times to visual targets were increased when the target was preceded by a novel nociceptive distracter as compared to the target preceded by a frequent tactile distracter, suggesting attentional capture by the novel nociceptive stimulus. However, when the task required an active rehearsal of the visual target in working memory, the novel nociceptive stimulus no longer induced a lengthening of reaction times to visual targets, indicating a reduction of the distraction produced by the novel nociceptive stimulus. This effect was independent of the overall task demands.

Conclusion and Significance

Loading working memory with pain-unrelated information may reduce the ability of nociceptive input to involuntarily capture attention, and shields cognitive processing from nociceptive distraction. An efficient control of attention over pain is best guaranteed by the ability to maintain active goal priorities during achievement of cognitive activities and to keep pain-related information out of task settings.     

A Potential Spatial Working Memory Training Task to Improve Both Episodic Memory and Fluid Intelligence

One current challenge in cognitive training is to create a training regime that benefits multiple cognitive domains, including episodic memory, without relying on a large battery of tasks, which can be time-consuming and difficult to learn. By giving careful consideration to the neural correlates underlying episodic and working memory, we devised a computerized working memory training task in which neurologically healthy participants were required to monitor and detect repetitions in two streams of spatial information (spatial location and scene identity) presented simultaneously (i.e. a dual n-back paradigm). Participants’ episodic memory abilities were assessed before and after training using two object and scene recognition memory tasks incorporating memory confidence judgments. Furthermore, to determine the generalizability of the effects of training, we also assessed fluid intelligence using a matrix reasoning task. By examining the difference between pre- and post-training performance (i.e. gain scores), we found that the trainers, compared to non-trainers, exhibited a significant improvement in fluid intelligence after 20 days. Interestingly, pre-training fluid intelligence performance, but not training task improvement, was a significant predictor of post-training fluid intelligence improvement, with lower pre-training fluid intelligence associated with greater post-training gain. Crucially, trainers who improved the most on the training task also showed an improvement in recognition memory as captured by d-prime scores and estimates of recollection and familiarity memory. Training task improvement was a significant predictor of gains in recognition and familiarity memory performance, with greater training improvement leading to more marked gains. In contrast, lower pre-training recollection memory scores, and not training task improvement, led to greater recollection memory performance after training. Our findings demonstrate that practice on a single working memory task can potentially improve aspects of both episodic memory and fluid intelligence, and that an extensive training regime with multiple tasks may not be necessary.     

Effects of atropine on operant test battery performance in rhesus monkeys.

The acute behavioral effects of atropine sulfate were assessed using a battery of complex food-reinforced operant tasks that included: temporal response differentiation (TRD, n = 7); delayed matching-to-sample (DMTS, n = 6), progressive ratio (PR, n = 8), incremental repeated acquisition (IRA, n = 8), and conditioned position responding (CPR, n = 8). Performance in these tasks is thought to depend primarily upon specific brain functions such as time perception, short-term memory and attention, motivation, learning, and color and position discrimination, respectively. Atropine sulfate (0.01-0.56 mg/kg iv), given 15-min pretesting, produced significant dose-dependent decreases in the number of reinforcers obtained in all tasks. Response rates decreased significantly at greater than or equal to 0.03 mg/kg for the learning and discrimination tasks, at greater than or equal to 0.10 mg/kg for the motivation and short-term memory and attention tasks, and at greater than or equal to 0.30 mg/kg for the time perception task. Response accuracies were significantly decreased at doses greater than or equal to 0.10 mg/kg for the learning, discrimination, and short-term memory and attention tasks, and at greater than or equal to 0.30 mg/kg for the time perception task. Thus, the order of task sensitivity to any disruption by atropine is learning = color and position discrimination greater than time perception = short-term memory and attention = motivation (IRA = CPR greater than TRD = DMTS = PR). Thus in monkeys, the rates of responding in operant tasks designed to model learning and color and position discrimination were the most sensitive measures to atropine's behavioral effects. Accuracy in these same task was also disrupted but at higher doses. These data support the hypothesis that cholinergic systems play a greater role in the speed (but not accuracy) of performance of our learning and discrimination tasks compared to all other tasks. Accuracy of responding in these and the short-term memory task, all of which involve the use of lights as visual stimuli, was more sensitive to disruption by atropine than those tasks which did not utilize such strong visual stimuli.     

Chronic stress effects on memory: sex differences in performance and monoaminergic activity

Increasing evidence suggests that the time course of advantageous versus deleterious effects of stress on physiologic function is also apparent in some brain functions, including learning and memory. This article reviews the effects of chronic stress on behavioral performance and, more importantly, shows that sex of the subject, as well as duration and intensity of stress, is an important determinant of the functional/behavioral, neurochemical, and anatomical consequences of the stress. Following chronic stress (7-28 days of restraint, 6 h/day), male and female rats were tested on a visual memory task (object recognition) and two spatial memory tasks (object placement and radial arm maze). At 21 days, stress impaired males on all tasks while females were either enhanced (spatial memory tasks) or not impaired (nonspatial memory tasks). Additionally, the influence of the hypothalamic-pituitary-adrenocortical axis in mediating the sex-specific responses to stress is considered. Behavioral and neurochemical assessments following chronic stress in ovariectomized females, with and without estradiol, suggest that estrogen exerts both organizational and activational influences on the observed sex differences in response to stress. Furthermore, stress differentially affected central transmitter levels in the frontal cortex, hippocampus, and amygdala depending on sex. The possible role of these sex-specific changes in neurotransmitter levels in mediating behavioral differences in response to stress is discussed. While these results are thus far limited to a few studies and require both further investigation and verification, chronic stress appears to be associated with distinct, sex-differentiated behavioral/cognitive and neurochemical responses. We conclude that sex differences must be taken into account when investigating or describing stress and associated sequalae.     

Working Memory in the Processing of the Iowa Gambling Task: An Individual Differences Approach

The Iowa Gambling Task (IGT) is a sequential learning task in which participants develop a tendency towards advantageous options arising from the outcomes associated with their previous decisions. The role of working memory in this complex task has been largely debated in the literature. On one hand, low working memory resources lead to a decrease in the number of advantageous decisions and make a significant part of participants unable to report explicitly which options are the most profitable. On the other hand, several studies have shown no contribution of working memory to the IGT decision patterns. In order to investigate this apparent incompatibility of results, we used an individual differences approach, which has proven an effective method to investigate the role of working memory in cognition. We compared the IGT decision patterns of participants as a function of their working memory capacity (WMC). As expected, contrary to low WMC participants, high WMC participants developed a tendency towards advantageous decisions. These findings lead us to discuss the role of WMC in decision making tasks.     

Stochastic Dynamics Underlying Cognitive Stability and Flexibility

Cognitive stability and flexibility are core functions in the successful pursuit of behavioral goals. While there is evidence for a common frontoparietal network underlying both functions and for a key role of dopamine in the modulation of flexible versus stable behavior, the exact neurocomputational mechanisms underlying those executive functions and their adaptation to environmental demands are still unclear. In this work we study the neurocomputational mechanisms underlying cue based task switching (flexibility) and distractor inhibition (stability) in a paradigm specifically designed to probe both functions. We develop a physiologically plausible, explicit model of neural networks that maintain the currently active task rule in working memory and implement the decision process. We simplify the four-choice decision network to a nonlinear drift-diffusion process that we canonically derive from a generic winner-take-all network model. By fitting our model to the behavioral data of individual subjects, we can reproduce their full behavior in terms of decisions and reaction time distributions in baseline as well as distractor inhibition and switch conditions. Furthermore, we predict the individual hemodynamic response timecourse of the rule-representing network and localize it to a frontoparietal network including the inferior frontal junction area and the intraparietal sulcus, using functional magnetic resonance imaging. This refines the understanding of task-switch-related frontoparietal brain activity as reflecting attractor-like working memory representations of task rules. Finally, we estimate the subject-specific stability of the rule-representing attractor states in terms of the minimal action associated with a transition between different rule states in the phase-space of the fitted models. This stability measure correlates with switching-specific thalamocorticostriatal activation, i.e., with a system associated with flexible working memory updating and dopaminergic modulation of cognitive flexibility. These results show that stochastic dynamical systems can implement the basic computations underlying cognitive stability and flexibility and explain neurobiological bases of individual differences.     

Lactate produced by glycogenolysis in astrocytes regulates memory processing

When administered either systemically or centrally, glucose is a potent enhancer of memory processes. Measures of glucose levels in extracellular fluid in the rat hippocampus during memory tests reveal that these levels are dynamic, decreasing in response to memory tasks and loads; exogenous glucose blocks these decreases and enhances memory. The present experiments test the hypothesis that glucose enhancement of memory is mediated by glycogen storage and then metabolism to lactate in astrocytes, which provide lactate to neurons as an energy substrate. Sensitive bioprobes were used to measure brain glucose and lactate levels in 1-sec samples. Extracellular glucose decreased and lactate increased while rats performed a spatial working memory task. Intrahippocampal infusions of lactate enhanced memory in this task. In addition, pharmacological inhibition of astrocytic glycogenolysis impaired memory and this impairment was reversed by administration of lactate or glucose, both of which can provide lactate to neurons in the absence of glycogenolysis. Pharmacological block of the monocarboxylate transporter responsible for lactate uptake into neurons also impaired memory and this impairment was not reversed by either glucose or lactate. These findings support the view that astrocytes regulate memory formation by controlling the provision of lactate to support neuronal functions.     

Effect of the BDNF V166M polymorphism on working memory in healthy adolescents

Brain-derived neurotrophic factor (BDNF) may play a role in modulating memory function and there is growing evidence that the BDNF V166M polymorphism may influence episodic memory in humans. However, previous association studies examining this polymorphism and working memory are inconsistent. The current study examined this association in a large sample of adolescent twin-pairs and siblings (785 individuals from 439 families). A range of measures (event-related potential, general performance and reaction time) was obtained from a delayed-response working-memory task and total association was examined using the quantitative transmission disequilibrium tests (QTDT) program. Analyses had approximately 93-97% power (alpha= 0.05) to detect an association accounting for as little as 2% of the variance in the phenotypes examined. Results indicated that the BDNF V166M polymorphism is not associated with variation in working memory in healthy adolescents.     

Construct validity of some episodic memory tests for psychogeriatric patients

Episodic memory is the conscious recollection of personal experiences. In clinical practice several episodic memory tests are used, but their validity as measures of an episodic memory construct is not clear. The sensitivity and specificity of variables hypothesized to represent a distinct psychological construct is established by a pattern of convergent and discriminant validity. Confirmatory factor analysis was used to analyse the correlations between four episodic memory tests: a standardized orientation questionnaire, free recall, and two delayed recognition memory tests. The episodic memory construct was contrasted with an executive functioning construct, measured by three variables: two tasks of semantic word fluency and the Expanded Mental Control Test. The measures were taken from 813 consecutive visitors of a psychogeriatric day care centre. As a set the four indicators of episodic memory provided reliable measurement of the construct. The same was true for the three measures of the executive functioning construct. However, the strong correlation (0.89) between the two constructs implies a lack of discriminant validity and suggests that processes of executive control contribute to the successful performance on episodic memory tasks.     

Anterior medial prefrontal cortex exhibits activation during task preparation but deactivation during task execution

Background

The anterior prefrontal cortex (PFC) exhibits activation during some cognitive tasks, including episodic memory, reasoning, attention, multitasking, task sets, decision making, mentalizing, and processing of self-referenced information. However, the medial part of anterior PFC is part of the default mode network (DMN), which shows deactivation during various goal-directed cognitive tasks compared to a resting baseline. One possible factor for this pattern is that activity in the anterior medial PFC (MPFC) is affected by dynamic allocation of attentional resources depending on task demands. We investigated this possibility using an event related fMRI with a face working memory task.

Methodology/Principal Findings

Sixteen students participated in a single fMRI session. They were asked to form a task set to remember the faces (Face memory condition) or to ignore them (No face memory condition), then they were given 6 seconds of preparation period before the onset of the face stimuli. During this 6-second period, four single digits were presented one at a time at the center of the display, and participants were asked to add them and to remember the final answer. When participants formed a task set to remember faces, the anterior MPFC exhibited activation during a task preparation period but deactivation during a task execution period within a single trial.

Conclusions/Significance

The results suggest that the anterior MPFC plays a role in task set formation but is not involved in execution of the face working memory task. Therefore, when attentional resources are allocated to other brain regions during task execution, the anterior MPFC shows deactivation. The results suggest that activation and deactivation in the anterior MPFC are affected by dynamic allocation of processing resources across different phases of processing.     

A reduction in hippocampal GABAA receptor α5 subunits disrupts the memory for location of objects in mice

The memory for location of objects, which binds information about objects to discrete positions or spatial contexts of occurrence, is a form of episodic memory particularly sensitive to hippocampal damage. Its early decline is symptomatic for elderly dementia. Substances that selectively reduce α5‐GABA_A receptor function are currently developed as potential cognition enhancers for Alzheimer's syndrome and other dementia, consistent with genetic studies implicating these receptors that are highly expressed in hippocampus in learning performance. Here we explored the consequences of reduced GABA_Aα5‐subunit contents, as occurring in α5(H105R) knock‐in mice, on the memory for location of objects. This required the behavioral characterization of α5(H105R) and wild‐type animals in various tasks examining learning and memory retrieval strategies for objects, locations, contexts and their combinations. In mutants, decreased amounts of α5‐subunits and retained long‐term potentiation in hippocampus were confirmed. They exhibited hyperactivity with conserved circadian rhythm in familiar actimeters, and normal exploration and emotional reactivity in novel places, allocentric spatial guidance, and motor pattern learning acquisition, inhibition and flexibility in T‐ and eight‐arm mazes. Processing of object, position and context memories and object‐guided response learning were spared. Genotype difference in object‐in‐place memory retrieval and in encoding and response learning strategies for object–location combinations manifested as a bias favoring object‐based recognition and guidance strategies over spatial processing of objects in the mutants. These findings identify in α5(H105R) mice a behavioral–cognitive phenotype affecting basal locomotion and the memory for location of objects indicative of hippocampal dysfunction resulting from moderately decreased α5‐subunit contents.     

Evidence for hippocampal role in place avoidance other than merely memory storage

Spatial navigation is used as a popular animal model of higher cognitive functions in people. The data suggest that the hippocampus is important for both storing spatial memories and for performing spatial computations necessary for navigation. Animals use multiple behavioral strategies to solve spatial tasks often using multiple memory systems. We investigated how inactivation of the rat hippocampus affects performance in a place avoidance task to determine if the role of the hippocampus in this task could be attributed to memory storage/retrieval or to the computations needed for navigation. Injecting tetrodotoxin (TTX) into both hippocampi impaired conditioned place avoidance, but after injecting only one hippocampus, the rats learned the place avoidance as well as without any injections. Retention of the place avoidance learned with one hippocampus was not impaired when the injection was switched to the hippocampus that had not been injected during learning. The result suggests that during learning, the hippocampus did not store the place avoidance memory.