Unsupervised Feature Learning Improves Prediction of Human Brain Activity in Response to Natural Images

Encoding and decoding in functional magnetic resonance imaging has recently emerged as an area of research to noninvasively characterize the relationship between stimulus features and human brain activity. To overcome the challenge of formalizing what stimulus features should modulate single voxel responses, we introduce a general approach for making directly testable predictions of single voxel responses to statistically adapted representations of ecologically valid stimuli. These representations are learned from unlabeled data without supervision. Our approach is validated using a parsimonious computational model of (i) how early visual cortical representations are adapted to statistical regularities in natural images and (ii) how populations of these representations are pooled by single voxels. This computational model is used to predict single voxel responses to natural images and identify natural images from stimulus-evoked multiple voxel responses. We show that statistically adapted low-level sparse and invariant representations of natural images better span the space of early visual cortical representations and can be more effectively exploited in stimulus identification than hand-designed Gabor wavelets. Our results demonstrate the potential of our approach to better probe unknown cortical representations.     

Monkeys match the number of voices they hear to the number of faces they see

Convergent evidence demonstrates that adult humans possess numerical representations that are independent of language [1, 2, 3, 4, 5 and 6]. Human infants and nonhuman animals can also make purely numerical discriminations, implicating both developmental and evolutionary bases for adult humans' language-independent representations of number [7 and 8]. Recent evidence suggests that the nonverbal representations of number held by human adults are not constrained by the sensory modality in which they were perceived [9]. Previous studies, however, have yielded conflicting results concerning whether the number representations held by nonhuman animals and human infants are tied to the modality in which they were established [10, 11, 12, 13, 14 and 15]. Here, we report that untrained monkeys preferentially looked at a dynamic video display depicting the number of conspecifics that matched the number of vocalizations they heard. These findings suggest that number representations held by monkeys, like those held by adult humans, are unfettered by stimulus modality.     

An Exploration of the Relations between External Representations and Working Memory

It is commonly hypothesized that external representations serve as memory aids and improve task performance by means of expanding the limited capacity of working memory. However, very few studies have directly examined this memory aid hypothesis. By systematically manipulating how information is available externally versus internally in a sequential number comparison task, three experiments were designed to investigate the relation between external representations and working memory. The experimental results show that when the task requires information from both external representations and working memory, it is the interaction of information from the two sources that determines task performance. In particular, when information from the two sources does not match well, external representations hinder instead of enhance task performance. The study highlights the important role the coordination among different representations plays in distributed cognition. The general relations between external representations and working memory are discussed.     

Species of thought: A comment on evolutionary epistemology

The primary outcome of natural selection is adaptation to an environment. The primary concern of epistemology is the acquistion of knowledge. Evolutionary epistemology must therefore draw a fundamental connection between adaptation and knowledge. Existing frameworks in evolutionary epistemology do this in two ways; (a) by treating adaptation as a form of knowledge, and (b) by treating the ability to acquire knowledge as a biologically evolved adaptation. I criticize both frameworks for failing to appreciate that mental representations can motivate behaviors that are adaptive in the real world without themselves directly corresponding to the real world. I suggest a third framework in which mental representations are to reality as species are to ecosystems. This is a many-to-one relationship that predicts a diversity of adaptive representations in the minds of interacting people. As “species of thought”, mental representations share a number of properties with biological species, including isolating mechanisms that prevent them from blending with other representations. Species of thought also are amenable to the empirical methods that evolutionists use to study adaptation in biological species. Empirical studies of mental representations in everyday life might even be necessary for science to succeed as a normative “truth-seeking” discipline.     

Accepting immigrants as fellow citizens: citizenship representations in relation to migration policy preferences

Citizenship representations within national populations have mainly been deduced from state policies on migration. Yet, at the individual level, no studies have investigated whether citizenship representations are reliably associated with preferences for specific migration policies (i.e. the underlying assumption for deducing citizenship representations from state policies). Because several studies have shown that state policies may not reflect understandings of citizenship within national populations, it may be more relevant to study citizenship representations at the individual level, in relation to personal preferences regarding migration policies. This study examined how ethnic, cultural and civic citizenship representations relate to migration policy preferences at the individual level among majority group high-school students (N = 1,734) in seven EU countries. Findings add to the understanding of citizenship representations and may have implications for the implementation of migration policies.     

From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach

People learn modality-independent, conceptual representations from modality-specific sensory signals. Here, we hypothesize that any system that accomplishes this feat will include three components: a representational language for characterizing modality-independent representations, a set of sensory-specific forward models for mapping from modality-independent representations to sensory signals, and an inference algorithm for inverting forward models—that is, an algorithm for using sensory signals to infer modality-independent representations. To evaluate this hypothesis, we instantiate it in the form of a computational model that learns object shape representations from visual and/or haptic signals. The model uses a probabilistic grammar to characterize modality-independent representations of object shape, uses a computer graphics toolkit and a human hand simulator to map from object representations to visual and haptic features, respectively, and uses a Bayesian inference algorithm to infer modality-independent object representations from visual and/or haptic signals. Simulation results show that the model infers identical object representations when an object is viewed, grasped, or both. That is, the model’s percepts are modality invariant. We also report the results of an experiment in which different subjects rated the similarity of pairs of objects in different sensory conditions, and show that the model provides a very accurate account of subjects’ ratings. Conceptually, this research significantly contributes to our understanding of modality invariance, an important type of perceptual constancy, by demonstrating how modality-independent representations can be acquired and used. Methodologically, it provides an important contribution to cognitive modeling, particularly an emerging probabilistic language-of-thought approach, by showing how symbolic and statistical approaches can be combined in order to understand aspects of human perception.     

Coding of cognitive magnitude: compressed scaling of numerical information in the primate prefrontal cortex

Whether cognitive representations are better conceived as language-based, symbolic representations or perceptually related, analog representations is a subject of debate. If cognitive processes parallel perceptual processes, then fundamental psychophysical laws should hold for each. To test this, we analyzed both behavioral and neuronal representations of numerosity in the prefrontal cortex of rhesus monkeys. The data were best described by a nonlinearly compressed scaling of numerical information, as postulated by the Weber-Fechner law or Stevens' law for psychophysical/sensory magnitudes. This nonlinear compression was observed on the neural level during the acquisition phase of the task and maintained through the memory phase with no further compression. These results suggest that certain cognitive and perceptual/sensory representations share the same fundamental mechanisms and neural coding schemes.     

Modelling brain representations of abstract concepts

Abstract conceptual representations are critical for human cognition. Despite their importance, key properties of these representations remain poorly understood. Here, we used computational models of distributional semantics to predict multivariate fMRI activity patterns during the activation and contextualization of abstract concepts. We devised a task in which participants had to embed abstract nouns into a story that they developed around a given background context. We found that representations in inferior parietal cortex were predicted by concept similarities emerging in models of distributional semantics. By constructing different model families, we reveal the models’ learning trajectories and delineate how abstract and concrete training materials contribute to the formation of brain-like representations. These results inform theories about the format and emergence of abstract conceptual representations in the human brain.     

Neural representation of visual objects: encoding and top-down activation

Knowledge or experiences are voluntarily recalled from memory by reactivation of their neural representations in the association cortex. Mnemonic representations of visual objects, located in the ventral processing stream of visual perception, provide the best indication of how neuronal codes are created, organized and reactivated. Associative codes are created by neurons that have the ability to link the representations of temporally associated stimuli. Recent experiments suggest that not only bottom-up signals from the retina but also top-down signals from the prefrontal cortex can trigger the retrieval of associative codes, which may serve as a neural basis for conscious recall.     

A neural field model of the somatosensory cortex: formation, maintenance and reorganization of ordered topographic maps

We investigate the formation and maintenance of ordered topographic maps in the primary somatosensory cortex as well as the reorganization of representations after sensory deprivation or cortical lesion. We consider both the critical period (postnatal) where representations are shaped and the post-critical period where representations are maintained and possibly reorganized. We hypothesize that feed-forward thalamocortical connections are an adequate site of plasticity while cortico-cortical connections are believed to drive a competitive mechanism that is critical for learning. We model a small skin patch located on the distal phalangeal surface of a digit as a set of 256 Merkel ending complexes (MEC) that feed a computational model of the primary somatosensory cortex (area 3b). This model is a two-dimensional neural field where spatially localized solutions (a.k.a. bumps) drive cortical plasticity through a Hebbian-like learning rule. Simulations explain the initial formation of ordered representations following repetitive and random stimulations of the skin patch. Skin lesions as well as cortical lesions are also studied and results confirm the possibility to reorganize representations using the same learning rule and depending on the type of the lesion. For severe lesions, the model suggests that cortico-cortical connections may play an important role in complete recovery.     

Introduction: gender,migration and the media

Abstract

Mediated representations of gender, ethnicity and migration play an increasingly important role in the way these categories are understood in the public sphere and the private realm. As media often intervene in processes of individual and institutional communication, they provide frameworks for the production and consumption of representations of these categories. Thus media – in their production, representations and consumption – need to be analysed, not only as reflections as pre-existing socio-political realities, but also as constitutive elements in the production of meanings of the self and the Other. This special issue includes a number of articles that examine the articulations of gendered ethnic identities and of gendered citizenship as these are shaped in media production, media representations and media consumption.     

Analytical processing of binary mixture information by olfactory bulb glomeruli

Odors are rarely composed of a single compound, but rather contain a large and complex variety of chemical components. Often, these mixtures are perceived as having unique qualities that can be quite different than the combination of their components. In many cases, a majority of the components of a mixture cannot be individually identified. This synthetic processing of odor information suggests that individual component representations of the mixture must interact somewhere along the olfactory pathway. The anatomical nature of sensory neuron input into segregated glomeruli with the bulb suggests that initial input of odor information into the bulb is analytic. However, a large network of interneurons within the olfactory bulb could allow for mixture interactions via mechanisms such as lateral inhibition. Currently in mammals, it is unclear if postsynaptic mitral/tufted cell glomerular mixture responses reflect the analytical mixture input, or provide the initial basis for synthetic processing with the olfactory system. To address this, olfactory bulb glomerular binary mixture representations were compared to representations of each component using transgenic mice expressing the calcium indicator G-CaMP2 in olfactory bulb mitral/tufted cells. Overall, dorsal surface mixture representations showed little mixture interaction and often appeared as a simple combination of the component representations. Based on this, it is concluded that dorsal surface glomerular mixture representations remain largely analytical with nearly all component information preserved.     

Cortical mechanisms of space-based and object-based attentional control

Visual attention, the mechanism by which observers select relevant or important information from scenes, can be deployed to locations in space or to spatially invariant object representations. Studies have examined both the modulatory effects of attention on the strength of extrastriate cortical representations, and the control of attention by parietal and frontal cortical circuits. Subregions of parietal and frontal cortex are transiently active when attention is voluntarily shifted between spatial locations or object representations. This transient activity may reflect an abrupt shift in the attentional set of the observer, complementing sustained signals that are thought to maintain a given attentive state.     

How Lateral Connections and Spiking Dynamics May Separate Multiple Objects Moving Together

Over successive stages, the ventral visual system of the primate brain develops neurons that respond selectively to particular objects or faces with translation, size and view invariance. The powerful neural representations found in Inferotemporal cortex form a remarkably rapid and robust basis for object recognition which belies the difficulties faced by the system when learning in natural visual environments. A central issue in understanding the process of biological object recognition is how these neurons learn to form separate representations of objects from complex visual scenes composed of multiple objects. We show how a one-layer competitive network comprised of ‘spiking’ neurons is able to learn separate transformation-invariant representations (exemplified by one-dimensional translations) of visual objects that are always seen together moving in lock-step, but separated in space. This is achieved by combining ‘Mexican hat’ functional lateral connectivity with cell firing-rate adaptation to temporally segment input representations of competing stimuli through anti-phase oscillations (perceptual cycles). These spiking dynamics are quickly and reliably generated, enabling selective modification of the feed-forward connections to neurons in the next layer through Spike-Time-Dependent Plasticity (STDP), resulting in separate translation-invariant representations of each stimulus. Variations in key properties of the model are investigated with respect to the network’s ability to develop appropriate input representations and subsequently output representations through STDP. Contrary to earlier rate-coded models of this learning process, this work shows how spiking neural networks may learn about more than one stimulus together without suffering from the ‘superposition catastrophe’. We take these results to suggest that spiking dynamics are key to understanding biological visual object recognition.     

Coding of distributed, topographic and non-specific representations within the brain

This article explores the theoretical basis of coding within topographic representations, where neurons encoding specific features such as locations, are arranged into maps. A novel type of representation, termed non-specific, where each neuron does not encode specific features is also postulated. In common with the previously described distributed representations [Rolls, E.T., Treves, A., 1998. Neural Networks and Brain Function. Oxford University Press, Oxford], topographic representations display an exponential relationship between stimuli encoded and both number of neurons and maximum firing rate of those neurons. The non-specific representations described here display a binomial expansion between the number of stimuli encoded and the sum of the number of neurons and the maximum firing rate; therefore groups of non-specific neurons usually encode less stimuli than equivalent topographic layers of neurons. Lower and higher order sensory regions of the brain use either topographic or distributed representations to encode information. It is proposed that non-specific representations may occur in regions of the brain where different types of information may be represented by the same neurons, as occurs in the prefrontal cortex.     

Espaces de représentation et psychopathologie

A subjective world can be conceptualized as a “representational space” — that is, as a universe displayed from memory, in which the subject “is sailing”. Recent cognitive theory provides some conceptual tools for describing such a space in a relevant way for clinical purposes. The construction of a representational space is based on mental representations: (a) analogical representations, which display a content in working memory; (b) symbolic representations, which code and link up analogical representations to form a represented world. The dynamics of representations and affects is ruled by a principle of unification of representational space. The topology of a representational space depends on objective and subjective constraints which cause some “folds” and limit the display of such a space. The interaction between an event and a subjective memory can be analyzed within this framework, so that the concepts of trauma and defence processes can be defined. Clinical syndromes are defensive configurations that tend to close the representational space.     

Neural fuzzy preference integration using neural preference Moore machines

This paper describes preference classes and preference Moore machines as a basis for integrating different hybrid neural representations. Preference classes are shown to provide a basic link between neural preferences and fuzzy representations at the preference class level. Preference Moore machines provide a link between recurrent neural networks and symbolic transducers at the preference Moore machine level. We demonstrate how the concepts of preference classes and preference Moore machines can be used to interpret neural network representations and to integrate knowledge from hybrid neural representations. One main contribution of this paper is the introduction and analysis of neural preference Moore machines and their link to a fuzzy interpretation. Furthermore, we illustrate the interpretation and combination of various neural preference Moore machines with additional real-world examples.