Emerging Object Representations in the Visual System Predict Reaction Times for Categorization

Recognizing an object takes just a fraction of a second, less than the blink of an eye. Applying multivariate pattern analysis, or “brain decoding”, methods to magnetoencephalography (MEG) data has allowed researchers to characterize, in high temporal resolution, the emerging representation of object categories that underlie our capacity for rapid recognition. Shortly after stimulus onset, object exemplars cluster by category in a high-dimensional activation space in the brain. In this emerging activation space, the decodability of exemplar category varies over time, reflecting the brain’s transformation of visual inputs into coherent category representations. How do these emerging representations relate to categorization behavior? Recently it has been proposed that the distance of an exemplar representation from a categorical boundary in an activation space is critical for perceptual decision-making, and that reaction times should therefore correlate with distance from the boundary. The predictions of this distance hypothesis have been born out in human inferior temporal cortex (IT), an area of the brain crucial for the representation of object categories. When viewed in the context of a time varying neural signal, the optimal time to “read out” category information is when category representations in the brain are most decodable. Here, we show that the distance from a decision boundary through activation space, as measured using MEG decoding methods, correlates with reaction times for visual categorization during the period of peak decodability. Our results suggest that the brain begins to read out information about exemplar category at the optimal time for use in choice behaviour, and support the hypothesis that the structure of the representation for objects in the visual system is partially constitutive of the decision process in recognition.     

Slowness: An Objective for Spike-Timing–Dependent Plasticity?

Our nervous system can efficiently recognize objects in spite of changes in contextual variables such as perspective or lighting conditions. Several lines of research have proposed that this ability for invariant recognition is learned by exploiting the fact that object identities typically vary more slowly in time than contextual variables or noise. Here, we study the question of how this “temporal stability” or “slowness” approach can be implemented within the limits of biologically realistic spike-based learning rules. We first show that slow feature analysis, an algorithm that is based on slowness, can be implemented in linear continuous model neurons by means of a modified Hebbian learning rule. This approach provides a link to the trace rule, which is another implementation of slowness learning. Then, we show analytically that for linear Poisson neurons, slowness learning can be implemented by spike-timing–dependent plasticity (STDP) with a specific learning window. By studying the learning dynamics of STDP, we show that for functional interpretations of STDP, it is not the learning window alone that is relevant but rather the convolution of the learning window with the postsynaptic potential. We then derive STDP learning windows that implement slow feature analysis and the “trace rule.” The resulting learning windows are compatible with physiological data both in shape and timescale. Moreover, our analysis shows that the learning window can be split into two functionally different components that are sensitive to reversible and irreversible aspects of the input statistics, respectively. The theory indicates that irreversible input statistics are not in favor of stable weight distributions but may generate oscillatory weight dynamics. Our analysis offers a novel interpretation for the functional role of STDP in physiological neurons.     

Egocentric Representation Acquired from Offline Map Learning

It is widely accepted that people establish allocentric spatial representation after learning a map. However, it is unknown whether people can directly acquire egocentric representation after map learning. In two experiments, the participants learned a distal environment through a map and then performed the egocentric pointing tasks in that environment under three conditions: with the heading aligned with the learning perspective (baseline), after 240° rotation from the baseline (updating), and after disorientation (disorientation). Disorientation disrupted the internal consistency of pointing among objects when the participants learned the sequentially displayed map, on which only one object name was displayed at a time while the location of “self” remained on the screen all the time. However, disorientation did not affect the internal consistency of pointing among objects when the participants learned the simultaneously displayed map. These results suggest that the egocentric representation can be acquired from a sequentially presented map.     

Social Network Analysis Shows Direct Evidence for Social Transmission of Tool Use in Wild Chimpanzees

Social network analysis methods have made it possible to test whether novel behaviors in animals spread through individual or social learning. To date, however, social network analysis of wild populations has been limited to static models that cannot precisely reflect the dynamics of learning, for instance, the impact of multiple observations across time. Here, we present a novel dynamic version of network analysis that is capable of capturing temporal aspects of acquisition—that is, how successive observations by an individual influence its acquisition of the novel behavior. We apply this model to studying the spread of two novel tool-use variants, “moss-sponging” and “leaf-sponge re-use,” in the Sonso chimpanzee community of Budongo Forest, Uganda. Chimpanzees are widely considered the most “cultural” of all animal species, with 39 behaviors suspected as socially acquired, most of them in the domain of tool-use. The cultural hypothesis is supported by experimental data from captive chimpanzees and a range of observational data. However, for wild groups, there is still no direct experimental evidence for social learning, nor has there been any direct observation of social diffusion of behavioral innovations. Here, we tested both a static and a dynamic network model and found strong evidence that diffusion patterns of moss-sponging, but not leaf-sponge re-use, were significantly better explained by social than individual learning. The most conservative estimate of social transmission accounted for 85% of observed events, with an estimated 15-fold increase in learning rate for each time a novice observed an informed individual moss-sponging. We conclude that group-specific behavioral variants in wild chimpanzees can be socially learned, adding to the evidence that this prerequisite for culture originated in a common ancestor of great apes and humans, long before the advent of modern humans.     

Same or different? Abstract relational concept use in juvenile bamboo sharks and Malawi cichlids

Sorting objects and events into categories and concepts is an important cognitive prerequisite that spares an individual the learning of every object or situation encountered in its daily life. Accordingly, specific items are classified in general groups that allow fast responses to novel situations. The present study assessed whether bamboo sharks Chiloscyllium griseum and Malawi cichlids Pseudotropheus zebra can distinguish sets of stimuli (each stimulus consisting of two abstract, geometric objects) that meet two conceptual preconditions, i.e., (1) “sameness” versus “difference” and (2) a certain spatial arrangement of both objects. In two alternative forced choice experiments, individuals were first trained to choose two different, vertically arranged objects from two different but horizontally arranged ones. Pair discriminations were followed by extensive transfer test experiments. Transfer tests using stimuli consisting of (a) black and gray circles and (b) squares with novel geometric patterns provided conflicting information with respect to the learnt rule “choose two different, vertically arranged objects”, thereby investigating (1) the individuals’ ability to transfer previously gained knowledge to novel stimuli and (2) the abstract relational concept(s) or rule(s) applied to categorize these novel objects. Present results suggest that the level of processing and usage of both abstract concepts differed considerably between bamboo sharks and Malawi cichlids. Bamboo sharks seemed to combine both concepts—although not with equal but hierarchical prominence—pointing to advanced cognitive capabilities. Conversely, Malawi cichlids had difficulties in discriminating between symbols and failed to apply the acquired training knowledge on new sets of geometric and, in particular, gray-level transfer stimuli.     

Auditory Conflict Resolution Correlates with Medial–Lateral Frontal Theta/Alpha Phase Synchrony

When multiple persons speak simultaneously, it may be difficult for the listener to direct attention to correct sound objects among conflicting ones. This could occur, for example, in an emergency situation in which one hears conflicting instructions and the loudest, instead of the wisest, voice prevails. Here, we used cortically-constrained oscillatory MEG/EEG estimates to examine how different brain regions, including caudal anterior cingulate (cACC) and dorsolateral prefrontal cortices (DLPFC), work together to resolve these kinds of auditory conflicts. During an auditory flanker interference task, subjects were presented with sound patterns consisting of three different voices, from three different directions (45° left, straight ahead, 45° right), sounding out either the letters “A” or “O”. They were asked to discriminate which sound was presented centrally and ignore the flanking distracters that were phonetically either congruent (50%) or incongruent (50%) with the target. Our cortical MEG/EEG oscillatory estimates demonstrated a direct relationship between performance and brain activity, showing that efficient conflict resolution, as measured with reduced conflict-induced RT lags, is predicted by theta/alpha phase coupling between cACC and right lateral frontal cortex regions intersecting the right frontal eye fields (FEF) and DLPFC, as well as by increased pre-stimulus gamma (60–110 Hz) power in the left inferior fontal cortex. Notably, cACC connectivity patterns that correlated with behavioral conflict-resolution measures were found during both the pre-stimulus and the pre-response periods. Our data provide evidence that, instead of being only transiently activated upon conflict detection, cACC is involved in sustained engagement of attentional resources required for effective sound object selection performance.     

24-Month-Olds’ Selective Learning Is Not an All-or-None Phenomenon

Evidence that children maintain some memories of labels that are unlikely to be shared by the broader linguistic community suggests that children’s selective learning is not an all-or-none phenomenon. Across three experiments, we examine the contexts in which 24-month-olds show selective learning and whether they adjust their selective learning if provided with cues of in-context relevance. In each experiment, toddlers were first familiarized with a source who acted on familiar objects in either typical or atypical ways (e.g., used a car to mimic driving or hop like a rabbit) or labeled familiar objects incorrectly (e.g., called a spoon a “brush”). The source then labeled unfamiliar objects using either a novel word (e.g., fep; Experiment 1) or sound (e.g., ring; Experiments 2 and 3). Results indicated that toddlers learnt words from the typical source but not from the atypical or inaccurate source. In contrast, toddlers extended sound labels only when a source who had previously acted atypically provided the sound labels. Thus, toddlers, like preschoolers, avoid forming semantic representations of new object labels that are unlikely to be relevant in the broader community, but will form event-based memories of such labels if they have reason to suspect such labels will have in-context relevance.     

Generalization of Auditory Sensory and Cognitive Learning in Typically Developing Children

Despite the well-established involvement of both sensory (“bottom-up”) and cognitive (“top-down”) processes in literacy, the extent to which auditory or cognitive (memory or attention) learning transfers to phonological and reading skills remains unclear. Most research has demonstrated learning of the trained task or even learning transfer to a closely related task. However, few studies have reported “far-transfer” to a different domain, such as the improvement of phonological and reading skills following auditory or cognitive training. This study assessed the effectiveness of auditory, memory or attention training on far-transfer measures involving phonological and reading skills in typically developing children. Mid-transfer was also assessed through untrained auditory, attention and memory tasks. Sixty 5- to 8-year-old children with normal hearing were quasi-randomly assigned to one of five training groups: attention group (AG), memory group (MG), auditory sensory group (SG), placebo group (PG; drawing, painting), and a control, untrained group (CG). Compliance, mid-transfer and far-transfer measures were evaluated before and after training. All trained groups received 12 x 45-min training sessions over 12 weeks. The CG did not receive any intervention. All trained groups, especially older children, exhibited significant learning of the trained task. On pre- to post-training measures (test-retest), most groups exhibited improvements on most tasks. There was significant mid-transfer for a visual digit span task, with highest span in the MG, relative to other groups. These results show that both sensory and cognitive (memory or attention) training can lead to learning in the trained task and to mid-transfer learning on a task (visual digit span) within the same domain as the trained tasks. However, learning did not transfer to measures of language (reading and phonological awareness), as the PG and CG improved as much as the other trained groups. Further research is required to investigate the effects of various stimuli and lengths of training on the generalization of sensory and cognitive learning to literacy skills.     

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.     

The 50s Cliff: A Decline in Perceptuo-Motor Learning,Not a Deficit in Visual Motion Perception

Previously, we measured perceptuo-motor learning rates across the lifespan and found a sudden drop in learning rates between ages 50 and 60, called the “50s cliff.” The task was a unimanual visual rhythmic coordination task in which participants used a joystick to oscillate one dot in a display in coordination with another dot oscillated by a computer. Participants learned to produce a coordination with a 90° relative phase relation between the dots. Learning rates for participants over 60 were half those of younger participants. Given existing evidence for visual motion perception deficits in people over 60 and the role of visual motion perception in the coordination task, it remained unclear whether the 50s cliff reflected onset of this deficit or a genuine decline in perceptuo-motor learning. The current work addressed this question. Two groups of 12 participants in each of four age ranges (20s, 50s, 60s, 70s) learned to perform a bimanual coordination of 90° relative phase. One group trained with only haptic information and the other group with both haptic and visual information about relative phase. Both groups were tested in both information conditions at baseline and post-test. If the 50s cliff was caused by an age dependent deficit in visual motion perception, then older participants in the visual group should have exhibited less learning than those in the haptic group, which should not exhibit the 50s cliff, and older participants in both groups should have performed less well when tested with visual information. Neither of these expectations was confirmed by the results, so we concluded that the 50s cliff reflects a genuine decline in perceptuo-motor learning with aging, not the onset of a deficit in visual motion perception.     

Ready…Go: Amplitude of the fMRI Signal Encodes Expectation of Cue Arrival Time

What happens when the brain awaits a signal of uncertain arrival time, as when a sprinter waits for the starting pistol? And what happens just after the starting pistol fires? Using functional magnetic resonance imaging (fMRI), we have discovered a novel correlate of temporal expectations in several brain regions, most prominently in the supplementary motor area (SMA). Contrary to expectations, we found little fMRI activity during the waiting period; however, a large signal appears after the “go” signal, the amplitude of which reflects learned expectations about the distribution of possible waiting times. Specifically, the amplitude of the fMRI signal appears to encode a cumulative conditional probability, also known as the cumulative hazard function. The fMRI signal loses its dependence on waiting time in a “countdown” condition in which the arrival time of the go cue is known in advance, suggesting that the signal encodes temporal probabilities rather than simply elapsed time. The dependence of the signal on temporal expectation is present in “no-go” conditions, demonstrating that the effect is not a consequence of motor output. Finally, the encoding is not dependent on modality, operating in the same manner with auditory or visual signals. This finding extends our understanding of the relationship between temporal expectancy and measurable neural signals.     

Early Prefrontal Brain Responses to the Hedonic Quality of Emotional Words – A Simultaneous EEG and MEG Study

The hedonic meaning of words affects word recognition, as shown by behavioral, functional imaging, and event-related potential (ERP) studies. However, the spatiotemporal dynamics and cognitive functions behind are elusive, partly due to methodological limitations of previous studies. Here, we account for these difficulties by computing combined electro-magnetoencephalographic (EEG/MEG) source localization techniques. Participants covertly read emotionally high-arousing positive and negative nouns, while EEG and MEG were recorded simultaneously. Combined EEG/MEG current-density reconstructions for the P1 (80–120 ms), P2 (150–190 ms) and EPN component (200–300 ms) were computed using realistic individual head models, with a cortical constraint. Relative to negative words, the P1 to positive words predominantly involved language-related structures (left middle temporal and inferior frontal regions), and posterior structures related to directed attention (occipital and parietal regions). Effects shifted to the right hemisphere in the P2 component. By contrast, negative words received more activation in the P1 time-range only, recruiting prefrontal regions, including the anterior cingulate cortex (ACC). Effects in the EPN were not statistically significant. These findings show that different neuronal networks are active when positive versus negative words are processed. We account for these effects in terms of an “emotional tagging” of word forms during language acquisition. These tags then give rise to different processing strategies, including enhanced lexical processing of positive words and a very fast language-independent alert response to negative words. The valence-specific recruitment of different networks might underlie fast adaptive responses to both approach- and withdrawal-related stimuli, be they acquired or biological.     

Does Recall after Sleep-Dependent Memory Consolidation Reinstate Sensitivity to Retroactive Interference?

Previous studies have shown that newly encoded memories are more resistant to retroactive interference when participants are allowed to sleep after learning the original material, suggesting a sleep-related strengthening of memories. In the present study, we investigated delayed, long-term effects of sleep vs. sleep deprivation (SD) on the first post-training night on memory consolidation and resistance to interference. On day 1, participants learned a list of unrelated word pairs (AB), either in the morning or in the evening, then spent the post-training night in a sleep or sleep deprivation condition, in a within-subject paradigm. On day 4, at the same time of day, they learned a novel list of word pairs (AC) in which 50% of the word pairs stemmed with the same word than in the AB list, resulting in retroactive interference. Participants had then to recall items from the AB list upon presentation of the “A” stem. Recall was marginally improved in the evening, as compared to the morning learning group. Most importantly, retroactive interference effects were found in the sleep evening group only, contrary to the hypothesis that sleep exerts a protective role against intrusion by novel but similar learning. We tentatively suggest that these results can be explained in the framework of the memory reconsolidation theory, stating that exposure to similar information sets back consolidated items in a labile form again sensitive to retroactive interference. In this context, sleep might not protect against interference but would promote an update of existing episodic memories while preventing saturation of the memory network due to the accumulation of dual traces.     

Ready…Go: Amplitude of the fMRI Signal Encodes Expectation of Cue Arrival Time

What happens when the brain awaits a signal of uncertain arrival time, as when a sprinter waits for the starting pistol? And what happens just after the starting pistol fires? Using functional magnetic resonance imaging (fMRI), we have discovered a novel correlate of temporal expectations in several brain regions, most prominently in the supplementary motor area (SMA). Contrary to expectations, we found little fMRI activity during the waiting period; however, a large signal appears after the “go” signal, the amplitude of which reflects learned expectations about the distribution of possible waiting times. Specifically, the amplitude of the fMRI signal appears to encode a cumulative conditional probability, also known as the cumulative hazard function. The fMRI signal loses its dependence on waiting time in a “countdown” condition in which the arrival time of the go cue is known in advance, suggesting that the signal encodes temporal probabilities rather than simply elapsed time. The dependence of the signal on temporal expectation is present in “no-go” conditions, demonstrating that the effect is not a consequence of motor output. Finally, the encoding is not dependent on modality, operating in the same manner with auditory or visual signals. This finding extends our understanding of the relationship between temporal expectancy and measurable neural signals.     

Interval time-place learning in young children

While previous research has investigated the ability of animals to learn the spatial and temporal contingencies of biologically significant events (known as time-place learning), this ability has not been studied in humans. Children ranging from 5 to 10 years old were tested on a modified interval time-place learning task using a touchscreen computer. Results demonstrate the children were able to quickly learn both the timing and the sequence of this task. Despite a lack of anticipation on baseline trials, the children continued to follow the spatio-temporal contingencies in probe sessions where these contingencies were removed. Performance on the probe sessions provide strong evidence that the children had learned the spatio-temporal contingencies. Future research is needed to determine what age-related changes in iTPL occur. Furthermore, it is argued that this procedure can be used to extend interval timing in research in children, including, but not limited to, investigation of scalar timing with longer durations than have previously been investigated.     

Mesoscopic Patterns of Neural Activity Support Songbird Cortical Sequences

Time-locked sequences of neural activity can be found throughout the vertebrate forebrain in various species and behavioral contexts. From “time cells” in the hippocampus of rodents to cortical activity controlling movement, temporal sequence generation is integral to many forms of learned behavior. However, the mechanisms underlying sequence generation are not well known. Here, we describe a spatial and temporal organization of the songbird premotor cortical microcircuit that supports sparse sequences of neural activity. Multi-channel electrophysiology and calcium imaging reveal that neural activity in premotor cortex is correlated with a length scale of 100 µm. Within this length scale, basal-ganglia–projecting excitatory neurons, on average, fire at a specific phase of a local 30 Hz network rhythm. These results show that premotor cortical activity is inhomogeneous in time and space, and that a mesoscopic dynamical pattern underlies the generation of the neural sequences controlling song.     

Time Slices: What Is the Duration of a Percept?

We experience the world as a seamless stream of percepts. However, intriguing illusions and recent experiments suggest that the world is not continuously translated into conscious perception. Instead, perception seems to operate in a discrete manner, just like movies appear continuous although they consist of discrete images. To explain how the temporal resolution of human vision can be fast compared to sluggish conscious perception, we propose a novel conceptual framework in which features of objects, such as their color, are quasi-continuously and unconsciously analyzed with high temporal resolution. Like other features, temporal features, such as duration, are coded as quantitative labels. When unconscious processing is “completed,” all features are simultaneously rendered conscious at discrete moments in time, sometimes even hundreds of milliseconds after stimuli were presented.     

Intrinsic Motivations Drive Learning of Eye Movements: An Experiment with Human Adults

Intrinsic motivations drive the acquisition of knowledge and skills on the basis of novel or surprising stimuli or the pleasure to learn new skills. In so doing, they are different from extrinsic motivations that are mainly linked to drives that promote survival and reproduction. Intrinsic motivations have been implicitly exploited in several psychological experiments but, due to the lack of proper paradigms, they are rarely a direct subject of investigation. This article investigates how different intrinsic motivation mechanisms can support the learning of visual skills, such as “foveate a particular object in space”, using a gaze contingency paradigm. In the experiment participants could freely foveate objects shown in a computer screen. Foveating each of two “button” pictures caused different effects: one caused the appearance of a simple image (blue rectangle) in unexpected positions, while the other evoked the appearance of an always-novel picture (objects or animals). The experiment studied how two possible intrinsic motivation mechanisms might guide learning to foveate one or the other button picture. One mechanism is based on the sudden, surprising appearance of a familiar image at unpredicted locations, and a second one is based on the content novelty of the images. The results show the comparative effectiveness of the mechanism based on image novelty, whereas they do not support the operation of the mechanism based on the surprising location of the image appearance. Interestingly, these results were also obtained with participants that, according to a post experiment questionnaire, had not understood the functions of the different buttons suggesting that novelty-based intrinsic motivation mechanisms might operate even at an unconscious level.