A computational model of the development of separate representations of facial identity and expression in the primate visual system

Experimental studies have provided evidence that the visual processing areas of the primate brain represent facial identity and facial expression within different subpopulations of neurons. For example, in non-human primates there is evidence that cells within the inferior temporal gyrus (TE) respond primarily to facial identity, while cells within the superior temporal sulcus (STS) respond to facial expression. More recently, it has been found that the orbitofrontal cortex (OFC) of non-human primates contains some cells that respond exclusively to changes in facial identity, while other cells respond exclusively to facial expression. How might the primate visual system develop physically separate representations of facial identity and expression given that the visual system is always exposed to simultaneous combinations of facial identity and expression during learning? In this paper, a biologically plausible neural network model, VisNet, of the ventral visual pathway is trained on a set of carefully-designed cartoon faces with different identities and expressions. The VisNet model architecture is composed of a hierarchical series of four Self-Organising Maps (SOMs), with associative learning in the feedforward synaptic connections between successive layers. During learning, the network develops separate clusters of cells that respond exclusively to either facial identity or facial expression. We interpret the performance of the network in terms of the learning properties of SOMs, which are able to exploit the statistical indendependence between facial identity and expression.     

Learning invariant object recognition in the visual system with continuous transformations

The cerebral cortex utilizes spatiotemporal continuity in the world to help build invariant representations. In vision, these might be representations of objects. The temporal continuity typical of objects has been used in an associative learning rule with a short-term memory trace to help build invariant object representations. In this paper, we show that spatial continuity can also provide a basis for helping a system to self-organize invariant representations. We introduce a new learning paradigm “continuous transformation learning” which operates by mapping spatially similar input patterns to the same postsynaptic neurons in a competitive learning system. As the inputs move through the space of possible continuous transforms (e.g. translation, rotation, etc.), the active synapses are modified onto the set of postsynaptic neurons. Because other transforms of the same stimulus overlap with previously learned exemplars, a common set of postsynaptic neurons is activated by the new transforms, and learning of the new active inputs onto the same postsynaptic neurons is facilitated. We demonstrate that a hierarchical model of cortical processing in the ventral visual system can be trained with continuous transform learning, and highlight differences in the learning of invariant representations to those achieved by trace learning.     

Synaptic plasticity and connectivity requirements to produce stimulus-pair specific responses in recurrent networks of spiking neurons

Animals must respond selectively to specific combinations of salient environmental stimuli in order to survive in complex environments. A task with these features, biconditional discrimination, requires responses to select pairs of stimuli that are opposite to responses to those stimuli in another combination. We investigate the characteristics of synaptic plasticity and network connectivity needed to produce stimulus-pair neural responses within randomly connected model networks of spiking neurons trained in biconditional discrimination. Using reward-based plasticity for synapses from the random associative network onto a winner-takes-all decision-making network representing perceptual decision-making, we find that reliably correct decision making requires upstream neurons with strong stimulus-pair selectivity. By chance, selective neurons were present in initial networks; appropriate plasticity mechanisms improved task performance by enhancing the initial diversity of responses. We find long-term potentiation of inhibition to be the most beneficial plasticity rule by suppressing weak responses to produce reliably correct decisions across an extensive range of networks.     

Neural network features distinguish chemosensory stimuli in Caenorhabditis elegans

Nervous systems extract and process information from the environment to alter animal behavior and physiology. Despite progress in understanding how different stimuli are represented by changes in neuronal activity, less is known about how they affect broader neural network properties. We developed a framework for using graph-theoretic features of neural network activity to predict ecologically relevant stimulus properties, in particular stimulus identity. We used the transparent nematode, Caenorhabditis elegans, with its small nervous system to define neural network features associated with various chemosensory stimuli. We first immobilized animals using a microfluidic device and exposed their noses to chemical stimuli while monitoring changes in neural activity of more than 50 neurons in the head region. We found that graph-theoretic features, which capture patterns of interactions between neurons, are modulated by stimulus identity. Further, we show that a simple machine learning classifier trained using graph-theoretic features alone, or in combination with neural activity features, can accurately predict salt stimulus. Moreover, by focusing on putative causal interactions between neurons, the graph-theoretic features were almost twice as predictive as the neural activity features. These results reveal that stimulus identity modulates the broad, network-level organization of the nervous system, and that graph theory can be used to characterize these changes.     

Binding by temporal structure in multiple feature domains of an oscillatory neuronal network

An important step in visual processing is the segregation of objects in a visual scene from one another and from the embedding background. According to current theories of visual neuroscience, the different features of a particular object are represented by cells which are spatially distributed across multiple visual areas in the brain. The segregation of an object therefore requires the unique identification and integration of the pertaining cells which have to be “bound” into one assembly coding for the object in question. Several authors have suggested that such a binding of cells could be achieved by the selective synchronization of temporally structured responses of the neurons activated by features of the same stimulus. This concept has recently gained support by the observation of stimulus-dependent oscillatory activity in the visual system of the cat, pigeon and monkey. Furthermore, experimental evidence has been found for the formation and segregation of synchronously active cell assemblies representing different stimuli in the visual field. In this study, we investigate temporally structured activity in networks with single and multiple feature domains. As a first step, we examine the formation and segregation of cell assemblies by synchronizing and desynchronizing connections within a single feature module. We then demonstrate that distributed assemblies can be appropriately bound in a network comprising three modules selective for stimulus disparity, orientation and colour, respectively. In this context, we address the principal problem of segregating assemblies representing spatially overlapping stimuli in a distributed architecture. Using synchronizing as well as desynchronizing mechanisms, our simulations demonstrate that the binding problem can be solved by temporally correlated responses of cells which are distributed across multiple feature modules. Received: 25 March 1993/Accepted in revised form: 8 September 1993     

Occasion setting in Pavlovian ambiguous target discriminations

Rats were trained in a Pavlovian serial ambiguous target discrimination, in which a target cue was reinforced if it was preceded by one stimulus (P-->T+) but was not reinforced if it was preceded by another stimulus (N-->T-). Test performance indicated that stimulus control by these features was weaker than that acquired by features trained within separate serial feature positive (P-->T+, T-) and serial feature negative (N-->W-, W+) discriminations. The form of conditioned responding and the patterns of transfer observed suggested that the serial ambiguous target discrimination was solved by occasion setting. The data are discussed in terms of the use of retrospective coding strategies when solving Pavlovian serial conditional discriminations, and the acquisition of special properties by both feature and target stimuli.     

Shape Invariant Coding of Motion Direction in Somatosensory Cortex

Invariant representations of stimulus features are thought to play an important role in producing stable percepts of objects. In the present study, we assess the invariance of neural representations of tactile motion direction with respect to other stimulus properties. To this end, we record the responses evoked in individual neurons in somatosensory cortex of primates, including areas 3b, 1, and 2, by three types of motion stimuli, namely scanned bars and dot patterns, and random dot displays, presented to the fingertips of macaque monkeys. We identify a population of neurons in area 1 that is highly sensitive to the direction of stimulus motion and whose motion signals are invariant across stimulus types and conditions. The motion signals conveyed by individual neurons in area 1 can account for the ability of human observers to discriminate the direction of motion of these stimuli, as measured in paired psychophysical experiments. We conclude that area 1 contains a robust representation of motion and discuss similarities in the neural mechanisms of visual and tactile motion processing.     

Learning to Attend: Modeling the Shaping of Selectivity in Infero-temporal Cortex in a Categorization Task

Recent experiments on behaving monkeys have shown that learning a visual categorization task makes the neurons in infero-temporal cortex (ITC) more selective to the task-relevant features of the stimuli (Sigala and Logothetis in Nature 415 318–320, 2002). We hypothesize that such a selectivity modulation emerges from the interaction between ITC and other cortical area, presumably the prefrontal cortex (PFC), where the previously learned stimulus categories are encoded. We propose a biologically inspired model of excitatory and inhibitory spiking neurons with plastic synapses, modified according to a reward based Hebbian learning rule, to explain the experimental results and test the validity of our hypothesis. We assume that the ITC neurons, receiving feature selective inputs, form stronger connections with the category specific neurons to which they are consistently associated in rewarded trials. After learning, the top-down influence of PFC neurons enhances the selectivity of the ITC neurons encoding the behaviorally relevant features of the stimuli, as observed in the experiments. We conclude that the perceptual representation in visual areas like ITC can be strongly affected by the interaction with other areas which are devoted to higher cognitive functions. Electronic Supplementary Material: Supplementary material is available in the online: version of this article at http://dx.doi.org/10.007/s00422-006-0054-z     

Neural circuit mechanisms for pattern detection and feature combination in olfactory cortex

Odors are initially encoded in the brain as a set of distinct physicochemical characteristics but are ultimately perceived as a unified sensory object--a "smell." It remains unclear how chemical features encoded by diverse odorant receptors and segregated glomeruli in the main olfactory bulb (MOB) are assembled into integrated cortical representations. Combining patterned optical microstimulation of MOB with in vivo electrophysiological recordings in anterior piriform cortex (PCx), we assessed how cortical neurons decode complex activity patterns distributed across MOB glomeruli. PCx firing was insensitive to single-glomerulus photostimulation. Instead, individual cells reported higher-order combinations of coactive glomeruli resembling odor-evoked sensory maps. Intracellular recordings revealed a corresponding circuit architecture providing each cortical neuron with weak synaptic input from a distinct subpopulation of MOB glomeruli. PCx neurons thus detect specific glomerular ensembles, providing an explicit neural representation of chemical feature combinations that are the hallmark of complex odor stimuli.     

Representation of moving stimuli by somatosensory neurons.

The findings obtained in neurophysiological and psychophysical investigations using tactile stimuli that move at constant velocity across the skin are reviewed. For certain neurons in the postcentral gyrus of the cerebral cortex (S-I) of macaque monkeys, direction of stimulus motion is a "trigger feature" i.e., moving tactile stimuli evoke vigorous discharge activity in these neurons only if the stimuli are moved in a particular direction across the receptive field. This directional selectivity is maximal when stimulus velocity is between 5 and 50 cm/sec, and falls off rapidly at lower or higher velocities. The capacity for human subjects to correctly identify the direction of stimulus motion on the skin exhibits a similar dependence on stimulus velocity. The similar effects of velocity on neural and psychophysical measures of directional sensitivity support the idea that direction of stimulus motion on the skin can only be recognized if the moving stimulus optimally activates the group of S-I neurons for which that directions of simulus motion is the trigger feature.     

Coding processes involved in the cortical representation of complex tactile stimuli

To understand how information is coded in the primary somatosensory cortex (S1) we need to decipher the relationship between neural activity and tactile stimuli. Such a relationship can be formally measured by mutual information. The present study was designed to determine how S1 neuronal populations code for the multidimensional kinetic features (i.e. random, time-varying patterns of force) of complex tactile stimuli, applied at different locations of the rat forepaw. More precisely, the stimulus localization and feature extraction were analyzed as two independent processes, using both rate coding and temporal coding strategies. To model the process of stimulus kinetic feature extraction, multidimensional stimuli were projected onto lower dimensional subspace and then clustered according to their similarity. Different combinations of stimuli clustering were applied to differentiate each stimulus identification process. Information analyses show that both processes are synergistic, this synergy is enhanced within the temporal coding framework. The stimulus localization process is faster than the stimulus feature extraction process. The latter provides more information quantity with rate coding strategy, whereas the localization process maximizes the mutual information within the temporal coding framework. Therefore, combining mutual information analysis with robust clustering of complex stimuli provides a framework to study neural coding mechanisms related to complex stimuli discrimination.     

A neural network model of ventriloquism effect and aftereffect

Presenting simultaneous but spatially discrepant visual and auditory stimuli induces a perceptual translocation of the sound towards the visual input, the ventriloquism effect. General explanation is that vision tends to dominate over audition because of its higher spatial reliability. The underlying neural mechanisms remain unclear. We address this question via a biologically inspired neural network. The model contains two layers of unimodal visual and auditory neurons, with visual neurons having higher spatial resolution than auditory ones. Neurons within each layer communicate via lateral intra-layer synapses; neurons across layers are connected via inter-layer connections. The network accounts for the ventriloquism effect, ascribing it to a positive feedback between the visual and auditory neurons, triggered by residual auditory activity at the position of the visual stimulus. Main results are: i) the less localized stimulus is strongly biased toward the most localized stimulus and not vice versa; ii) amount of the ventriloquism effect changes with visual-auditory spatial disparity; iii) ventriloquism is a robust behavior of the network with respect to parameter value changes. Moreover, the model implements Hebbian rules for potentiation and depression of lateral synapses, to explain ventriloquism aftereffect (that is, the enduring sound shift after exposure to spatially disparate audio-visual stimuli). By adaptively changing the weights of lateral synapses during cross-modal stimulation, the model produces post-adaptive shifts of auditory localization that agree with in-vivo observations. The model demonstrates that two unimodal layers reciprocally interconnected may explain ventriloquism effect and aftereffect, even without the presence of any convergent multimodal area. The proposed study may provide advancement in understanding neural architecture and mechanisms at the basis of visual-auditory integration in the spatial realm.     

Developmental changes in hierarchical visual stimulus recognition in children aged 5-10 years

Reaction time (RT) and performance accuracy in hierarchical visual stimulus recognition at local and global levels were studied in 95 healthy 5-6, 6-7, 7-8 and 9-10-year-old children and 10 adults. Task performance of all examined subjects, both children and adults, was faster and more accurate during global feature recognition (global advantage effect), with increased RT to incongruent stimuli in local condition (global interference effect). Significant inter-individual differences were found in the youngest group (5-6-year-olds): 7 children from the total number of 37 subjects failed to show the global advantage and global interference effects. Significant progressive shifts in performance accuracy during hierarchical stimulus recognition at both local and global levels were observed in the period between 6-7 and 7-8 years and then between 9-10 years and adulthood. The time course of age-dependent changes in recognition time was different for the global and local features of the hierarchical stimuli: the RT significantly decreased in every successive age group for local feature recognition beginning from 6-7-year-old children, whereas there was no significant difference between 7-8 and 9-10-year-old children in the RT of the recognition of the global feature. In the two younger groups (5-6 and 6-7 years), the stimulus type affected performance in a specific manner: RT increased to both incongruent and neutral stimuli irrespective of the level of the target feature. It was assumed that nonlinear developmental trends in hierarchical stimulus recognition parameters depend on both maturation of visual information processing and development of executive functions, especially those related to selection of relevant signals.     

Predictive features of persistent activity emergence in regular spiking and intrinsic bursting model neurons

Proper functioning of working memory involves the expression of stimulus-selective persistent activity in pyramidal neurons of the prefrontal cortex (PFC), which refers to neural activity that persists for seconds beyond the end of the stimulus. The mechanisms which PFC pyramidal neurons use to discriminate between preferred vs. neutral inputs at the cellular level are largely unknown. Moreover, the presence of pyramidal cell subtypes with different firing patterns, such as regular spiking and intrinsic bursting, raises the question as to what their distinct role might be in persistent firing in the PFC. Here, we use a compartmental modeling approach to search for discriminatory features in the properties of incoming stimuli to a PFC pyramidal neuron and/or its response that signal which of these stimuli will result in persistent activity emergence. Furthermore, we use our modeling approach to study cell-type specific differences in persistent activity properties, via implementing a regular spiking (RS) and an intrinsic bursting (IB) model neuron. We identify synaptic location within the basal dendrites as a feature of stimulus selectivity. Specifically, persistent activity-inducing stimuli consist of activated synapses that are located more distally from the soma compared to non-inducing stimuli, in both model cells. In addition, the action potential (AP) latency and the first few inter-spike-intervals of the neuronal response can be used to reliably detect inducing vs. non-inducing inputs, suggesting a potential mechanism by which downstream neurons can rapidly decode the upcoming emergence of persistent activity. While the two model neurons did not differ in the coding features of persistent activity emergence, the properties of persistent activity, such as the firing pattern and the duration of temporally-restricted persistent activity were distinct. Collectively, our results pinpoint to specific features of the neuronal response to a given stimulus that code for its ability to induce persistent activity and predict differential roles of RS and IB neurons in persistent activity expression.     

Second order dimensionality reduction using minimum and maximum mutual information models

Conventional methods used to characterize multidimensional neural feature selectivity, such as spike-triggered covariance (STC) or maximally informative dimensions (MID), are limited to Gaussian stimuli or are only able to identify a small number of features due to the curse of dimensionality. To overcome these issues, we propose two new dimensionality reduction methods that use minimum and maximum information models. These methods are information theoretic extensions of STC that can be used with non-Gaussian stimulus distributions to find relevant linear subspaces of arbitrary dimensionality. We compare these new methods to the conventional methods in two ways: with biologically-inspired simulated neurons responding to natural images and with recordings from macaque retinal and thalamic cells responding to naturalistic time-varying stimuli. With non-Gaussian stimuli, the minimum and maximum information methods significantly outperform STC in all cases, whereas MID performs best in the regime of low dimensional feature spaces.     

A template theory to relate visual processing to digital circuitry

Simple stimulus patterns, in this case visual, are represented by spatiotemporal Boolean functions that can be summarized in a 4 x 4 look-up table of 16 templates behind each sensory neuron. These groups of templates correspond to groups of neurons in columns behind each receptor. They abstract specific combinations of input in simple combinations and include two successive states in time. A template is like a neuron field at threshold, and responds as the field is convolved with the stimulus pattern. The same structure can be repeated in successive layers to make progressive categorization and to reject inappropriate combinations. At any level, the templates act in groups, so providing a very large number of combinations that can represent more complex stimulus patterns at deeper levels.     

Neural coding properties based on spike timing and pattern correlation of retinal ganglion cells

Correlation between spike trains or neurons sometimes indicates certain neural coding rules in the visual system. In this paper, the relationship between spike timing correlation and pattern correlation is discussed, and their ability to represent stimulus features is compared to examine their coding strategies not only in individual neurons but also in population. Two kinds of stimuli, natural movies and checkerboard, are used to arouse firing activities in chicken retinal ganglion cells. The spike timing correlation and pattern correlation are calculated by cross-correlation function and Lempel–Ziv distance respectively. According to the correlation values, it is demonstrated that spike trains with similar spike patterns are not necessarily concerted in firing time. Moreover, spike pattern correlation values between individual neurons’ responses reflect the difference of natural movies and checkerboard; neurons cooperate with each other with higher pattern correlation values which represent spatiotemporal correlations during response to natural movies. Spike timing does not reflect stimulus features as obvious as spike patterns, caused by their particular coding properties or physiological foundation. As a result, separating the pattern correlation out of traditional timing correlation concept uncover additional insight in neural coding.     

Deep Supervised,but Not Unsupervised,Models May Explain IT Cortical Representation

Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.     

The effect of intracortical competition on the formation of topographic maps in models of Hebbian learning

Correlation-based learning (CBL) models and self-organizing maps (SOM) are two classes of Hebbian models that have both been proposed to explain the activity-driven formation of cortical maps. Both models differ significantly in the way lateral cortical interactions are treated, leading to different predictions for the formation of receptive fields. The linear CBL models predict that receptive field profiles are determined by the average values and the spatial correlations of the second order of the afferent activity patterns, wheras SOM models map stimulus features. Here, we investigate a class of models which are characterized by a variable degree of lateral competition and which have the CBL and SOM models as limit cases. We show that there exists a critical value for intracortical competition below which the model exhibits CBL properties and above which feature mapping sets in. The class of models is then analyzed with respect to the formation of topographic maps between two layers of neurons. For Gaussian input stimuli we find that localized receptive fields and topographic maps emerge above the critical value for intracortical competition, and we calculate this value as a function of the size of the input stimuli and the range of the lateral interaction function. Additionally, we show that the learning rule can be derived via the optimization of a global cost function in a framework of probabilistic output neurons which represent a set of input stimuli by a sparse code. Received: 23 June 1999 / Accepted in revised form: 05 November 1999     

A neural network model of general olfactory coding in the insect antennal lobe.

A central problem in olfaction is understanding how the quality of olfactory stimuli is encoded in the insect antennal lobe (or in the analogously structured vertebrate olfactory bulb) for perceptual processing in the mushroom bodies of the insect protocerebrum (or in the vertebrate olfactory cortex). In the study reported here, a relatively simple neural network model, inspired by our current knowledge of the insect antennal lobes, is used to investigate how each of several features and elements of the network, such as synapse strengths, feedback circuits and the steepness of neural activation functions, influences the formation of an olfactory code in neurons that project from the antennal lobes to the mushroom bodies (or from mitral cells to olfactory cortex). An optimal code in these projection neurons (PNs) should minimize potential errors by the mushroom bodies in misidentifying the quality of an odor across a range of concentrations while maximizing the ability of the mushroom bodies to resolve odors of different quality. Simulation studies demonstrate that the network is able to produce codes independent or virtually independent of concentration over a given range. The extent of this range is moderately dependent on a parameter that characterizes how long it takes for the voltage in an activated neuron to decay back to its resting potential, strongly dependent on the strength of excitatory feedback by the PNs onto antennal lobe intrinsic neurons (INs), and overwhelmingly dependent on the slope of the activation function that transforms the voltage of depolarized neurons into the rate at which spikes are produced. Although the code in the PNs is degraded by large variations in the concentration of odor stimuli, good performance levels are maintained when the complexity of stimuli, as measured by the number of component odorants, is doubled. When excitatory feedback from the PNs to the INs is strong, the activity in the PNs undergoes transitions from initial states to stimulus-specific equilibrium states that are maintained once the stimulus is removed. When this PN-IN feedback is weak the PNs are more likely to relax back to a stimulus-independent equilibrium state, in which case the code is not maintained beyond the application of the stimulus. Thus, for the architecture simulated here, strong feedback from the PNs onto the INs, together with step-like neuronal activation functions, could well be important in producing easily discriminable odor quality codes that are invariant over several orders of magnitude in stimulus concentration.