Pattern storage in a sparsely coded neural network with cyclic activation

We investigate an artificial neural network model with a modified Hebb rule. It is an auto-associative neural network similar to the Hopfield model and to the Willshaw model. It has properties of both of these models. Another property is that the patterns are sparsely coded and are stored in cycles of synchronous neural activities. The cycles of activity for some ranges of parameter increase the capacity of the model. We discuss basic properties of the model and some of the implementation issues, namely optimizing of the algorithms. We describe the modification of the Hebb learning rule, the learning algorithm, the generation of patterns, decomposition of patterns into cycles and pattern recall.     

Category learning induces position invariance of pattern recognition across the visual field

Human object recognition is considered to be largely invariant to translation across the visual field. However, the origin of this invariance to positional changes has remained elusive, since numerous studies found that the ability to discriminate between visual patterns develops in a largely location-specific manner, with only a limited transfer to novel visual field positions. In order to reconcile these contradicting observations, we traced the acquisition of categories of unfamiliar grey-level patterns within an interleaved learning and testing paradigm that involved either the same or different retinal locations. Our results show that position invariance is an emergent property of category learning. Pattern categories acquired over several hours at a fixed location in either the peripheral or central visual field gradually become accessible at new locations without any position-specific feedback. Furthermore, categories of novel patterns presented in the left hemifield are distinctly faster learnt and better generalized to other locations than those learnt in the right hemifield. Our results suggest that during learning initially position-specific representations of categories based on spatial pattern structure become encoded in a relational, position-invariant format. Such representational shifts may provide a generic mechanism to achieve perceptual invariance in object recognition.     

Learning and recall in a dynamic theory of coordination patterns

A dynamic theory of learning and recall of coordination patterns is developed in the context of relative timing skills. Characterizing the coordination patterns in such skills by the collective variable, relative phase, we choose a model system in which the intrinsic pattern dynamics as well as the influence of environmental and memorized information are well understood from previous experimental and theoretical work. To describe learning we endow memorized information with dynamics which is determined by a phenomenological strategy. Similarly, additional degrees of freedom must be introduced to understand recall. As such recall variables we choose the relative strengths with which each memorized pattern acts on the pattern dynamics and model their dynamics phenomenologically. The resulting dynamical system that resembles models used in pattern recognition theory is shown to adequately describe the learning and recall processes. Moreover, due to the operational character of the theory, several predictions emerge that are open to experimental test. In particular, we show under which conditions phase transitions occur in the dynamics of the coordination patterns during learning and during recall. Considering different time scales and their relations we demonstrate how these phase transitions can be identified and observed. Other predictions include the influence of the intrinsic pattern dynamics on the recall process and the existence of history and hysteresis effects in recall. We discuss different forms of forgetting and differentiation of memorized information. The results show how a new theoretical view of learning and recall as change of behavioral dynamics can lead to a different understanding of these processes by providing testable predictions.     

Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns

A new learning rule (Precise-Spike-Driven (PSD) Synaptic Plasticity) is proposed for processing and memorizing spatiotemporal patterns. PSD is a supervised learning rule that is analytically derived from the traditional Widrow-Hoff rule and can be used to train neurons to associate an input spatiotemporal spike pattern with a desired spike train. Synaptic adaptation is driven by the error between the desired and the actual output spikes, with positive errors causing long-term potentiation and negative errors causing long-term depression. The amount of modification is proportional to an eligibility trace that is triggered by afferent spikes. The PSD rule is both computationally efficient and biologically plausible. The properties of this learning rule are investigated extensively through experimental simulations, including its learning performance, its generality to different neuron models, its robustness against noisy conditions, its memory capacity, and the effects of its learning parameters. Experimental results show that the PSD rule is capable of spatiotemporal pattern classification, and can even outperform a well studied benchmark algorithm with the proposed relative confidence criterion. The PSD rule is further validated on a practical example of an optical character recognition problem. The results again show that it can achieve a good recognition performance with a proper encoding. Finally, a detailed discussion is provided about the PSD rule and several related algorithms including tempotron, SPAN, Chronotron and ReSuMe.     

Neural Population-Level Memory Traces in the Mouse Hippocampus

One of the fundamental goals in neurosciences is to elucidate the formation and retrieval of brain''s associative memory traces in real-time. Here, we describe real-time neural ensemble transient dynamics in the mouse hippocampal CA1 region and demonstrate their relationships with behavioral performances during both learning and recall. We employed the classic trace fear conditioning paradigm involving a neutral tone followed by a mild foot-shock 20 seconds later. Our large-scale recording and decoding methods revealed that conditioned tone responses and tone-shock association patterns were not present in CA1 during the first pairing, but emerged quickly after multiple pairings. These encoding patterns showed increased immediate-replay, correlating tightly with increased immediate-freezing during learning. Moreover, during contextual recall, these patterns reappeared in tandem six-to-fourteen times per minute, again correlating tightly with behavioral recall. Upon traced tone recall, while various fear memories were retrieved, the shock traces exhibited a unique recall-peak around the 20-second trace interval, further signifying the memory of time for the expected shock. Therefore, our study has revealed various real-time associative memory traces during learning and recall in CA1, and demonstrates that real-time memory traces can be decoded on a moment-to-moment basis over any single trial.     

Learning of oscillatory correlated patterns in a cortical network by a STDP-based learning rule

In this paper, we propose an iterative learning rule that allows the imprinting of correlated oscillatory patterns in a model of the hippocampus able to work as an associative memory for oscillatory spatio-temporal patterns. We analyze the dynamics in the Fourier domain, showing how the network selectively amplify or distort the Fourier components of the input, in a manner which depends on the imprinted patterns. We also prove that the proposed iterative local rule converges to the pseudo-inverse rule generalized to oscillatory patterns.     

Frequency-multiplexing ability of complex-valued Hebbian learning in logic gates

Lightwave has attractive characteristics such as spatial parallelism, temporal rapidity in signal processing, and frequency band vastness. In particular, the vast carrier frequency bandwidth promises novel information processing. In this paper, we propose a novel optical logic gate that learns multiple functions at frequencies different from one another, and analyze the frequency-domain multiplexing ability in the learning based on complex-valued Hebbian rule. We evaluate the averaged error function values in the learning process and the error probabilities in the realized logic functions. We investigate optimal learning parameters as well as performance dependence on the number of learning iterations and the number of parallel paths per neuron. Results show a trade-off among the learning parameters such as learning time constant and learning gain. We also find that when we prepare 10 optical path differences and conduct 200 learning iterations, the error probability completely decreases to zero in a three-function multiplexing case. However, at the same time, the error probability is tolerant of the path number. That is, even if the path number is reduced by half, error probability is found almost zero. The results can be useful to determine neural parameters for future optical neural network systems and devices that utilize the vast frequency bandwidth for frequency-domain multiplexing.     

A model of associative learning in the mushroom body

The mushroom body is a prominent invertebrate neuropil strongly associated with learning and memory. We built a high-level computational model of this structure using simplified but realistic models of neurons and synapses, and developed a learning rule based on activity dependent pre-synaptic facilitation. We show that our model, which is consistent with mushroom body Drosophila data and incorporates Aplysia learning, is able to both acquire and later recall CS-US associations. We demonstrate that a highly divergent input connectivity to the mushroom body and strong periodic inhibition both serve to improve overall learning performance. We also examine the problem of how synaptic conductance, driven by successive training events, obtains a value appropriate for the stimulus being learnt. We employ two feedback mechanisms: one stabilises strength at an initial level appropriate for an association; another prevents strength increase for established associations.     

EEG Beta Power but Not Background Music Predicts the Recall Scores in a Foreign-Vocabulary Learning Task

As tantalizing as the idea that background music beneficially affects foreign vocabulary learning may seem, there is—partly due to a lack of theory-driven research—no consistent evidence to support this notion. We investigated inter-individual differences in the effects of background music on foreign vocabulary learning. Based on Eysenck’s theory of personality we predicted that individuals with a high level of cortical arousal should perform worse when learning with background music compared to silence, whereas individuals with a low level of cortical arousal should be unaffected by background music or benefit from it. Participants were tested in a paired-associate learning paradigm consisting of three immediate word recall tasks, as well as a delayed recall task one week later. Baseline cortical arousal assessed with spontaneous EEG measurement in silence prior to the learning rounds was used for the analyses. Results revealed no interaction between cortical arousal and the learning condition (background music vs. silence). Instead, we found an unexpected main effect of cortical arousal in the beta band on recall, indicating that individuals with high beta power learned more vocabulary than those with low beta power. To substantiate this finding we conducted an exact replication of the experiment. Whereas the main effect of cortical arousal was only present in a subsample of participants, a beneficial main effect of background music appeared. A combined analysis of both experiments suggests that beta power predicts the performance in the word recall task, but that there is no effect of background music on foreign vocabulary learning. In light of these findings, we discuss whether searching for effects of background music on foreign vocabulary learning, independent of factors such as inter-individual differences and task complexity, might be a red herring. Importantly, our findings emphasize the need for sufficiently powered research designs and exact replications of theory-driven experiments when investigating effects of background music and inter-individual variation on task performance.     

Learning of Chunking Sequences in Cognition and Behavior

We often learn and recall long sequences in smaller segments, such as a phone number 858 534 22 30 memorized as four segments. Behavioral experiments suggest that humans and some animals employ this strategy of breaking down cognitive or behavioral sequences into chunks in a wide variety of tasks, but the dynamical principles of how this is achieved remains unknown. Here, we study the temporal dynamics of chunking for learning cognitive sequences in a chunking representation using a dynamical model of competing modes arranged to evoke hierarchical Winnerless Competition (WLC) dynamics. Sequential memory is represented as trajectories along a chain of metastable fixed points at each level of the hierarchy, and bistable Hebbian dynamics enables the learning of such trajectories in an unsupervised fashion. Using computer simulations, we demonstrate the learning of a chunking representation of sequences and their robust recall. During learning, the dynamics associates a set of modes to each information-carrying item in the sequence and encodes their relative order. During recall, hierarchical WLC guarantees the robustness of the sequence order when the sequence is not too long. The resulting patterns of activities share several features observed in behavioral experiments, such as the pauses between boundaries of chunks, their size and their duration. Failures in learning chunking sequences provide new insights into the dynamical causes of neurological disorders such as Parkinson’s disease and Schizophrenia.     

Learning invariant object recognition in the visual system with continuous transformations

The cerebral cortex utilizes spatiotemporal continuity in the world to help build invariant representations. In vision, these might be representations of objects. The temporal continuity typical of objects has been used in an associative learning rule with a short-term memory trace to help build invariant object representations. In this paper, we show that spatial continuity can also provide a basis for helping a system to self-organize invariant representations. We introduce a new learning paradigm “continuous transformation learning” which operates by mapping spatially similar input patterns to the same postsynaptic neurons in a competitive learning system. As the inputs move through the space of possible continuous transforms (e.g. translation, rotation, etc.), the active synapses are modified onto the set of postsynaptic neurons. Because other transforms of the same stimulus overlap with previously learned exemplars, a common set of postsynaptic neurons is activated by the new transforms, and learning of the new active inputs onto the same postsynaptic neurons is facilitated. We demonstrate that a hierarchical model of cortical processing in the ventral visual system can be trained with continuous transform learning, and highlight differences in the learning of invariant representations to those achieved by trace learning.     

Optimal learning rules for familiarity detection

It has been suggested that the mammalian memory system has both familiarity and recollection components. Recently, a high-capacity network to store familiarity has been proposed. Here we derive analytically the optimal learning rule for such a familiarity memory using a signal- to-noise ratio analysis. We find that in the limit of large networks the covariance rule, known to be the optimal local, linear learning rule for pattern association, is also the optimal learning rule for familiarity discrimination. In the limit of large networks, the capacity is independent of the sparseness of the patterns and the corresponding information capacity is 0.057 bits per synapse, which is somewhat less than typically found for associative networks.     

Fine-tuning and the stability of recurrent neural networks

A central criticism of standard theoretical approaches to constructing stable, recurrent model networks is that the synaptic connection weights need to be finely-tuned. This criticism is severe because proposed rules for learning these weights have been shown to have various limitations to their biological plausibility. Hence it is unlikely that such rules are used to continuously fine-tune the network in vivo. We describe a learning rule that is able to tune synaptic weights in a biologically plausible manner. We demonstrate and test this rule in the context of the oculomotor integrator, showing that only known neural signals are needed to tune the weights. We demonstrate that the rule appropriately accounts for a wide variety of experimental results, and is robust under several kinds of perturbation. Furthermore, we show that the rule is able to achieve stability as good as or better than that provided by the linearly optimal weights often used in recurrent models of the integrator. Finally, we discuss how this rule can be generalized to tune a wide variety of recurrent attractor networks, such as those found in head direction and path integration systems, suggesting that it may be used to tune a wide variety of stable neural systems.     

Pseudo-relaxation learning algorithm for complex-valued associative memory

HAM (Hopfield Associative Memory) and BAM (Bidirectinal Associative Memory) are representative associative memories by neural networks. The storage capacity by the Hebb rule, which is often used, is extremely low. In order to improve it, some learning methods, for example, pseudo-inverse matrix learning and gradient descent learning, have been introduced. Oh introduced pseudo-relaxation learning algorithm to HAM and BAM. In order to accelerate it, Hattori proposed quick learning. Noest proposed CAM (Complex-valued Associative Memory), which is complex-valued HAM. The storage capacity of CAM by the Hebb rule is also extremely low. Pseudo-inverse matrix learning and gradient descent learning have already been generalized to CAM. In this paper, we apply pseudo-relaxation learning algorithm to CAM in order to improve the capacity.     

A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback

Reward-modulated spike-timing-dependent plasticity (STDP) has recently emerged as a candidate for a learning rule that could explain how behaviorally relevant adaptive changes in complex networks of spiking neurons could be achieved in a self-organizing manner through local synaptic plasticity. However, the capabilities and limitations of this learning rule could so far only be tested through computer simulations. This article provides tools for an analytic treatment of reward-modulated STDP, which allows us to predict under which conditions reward-modulated STDP will achieve a desired learning effect. These analytical results imply that neurons can learn through reward-modulated STDP to classify not only spatial but also temporal firing patterns of presynaptic neurons. They also can learn to respond to specific presynaptic firing patterns with particular spike patterns. Finally, the resulting learning theory predicts that even difficult credit-assignment problems, where it is very hard to tell which synaptic weights should be modified in order to increase the global reward for the system, can be solved in a self-organizing manner through reward-modulated STDP. This yields an explanation for a fundamental experimental result on biofeedback in monkeys by Fetz and Baker. In this experiment monkeys were rewarded for increasing the firing rate of a particular neuron in the cortex and were able to solve this extremely difficult credit assignment problem. Our model for this experiment relies on a combination of reward-modulated STDP with variable spontaneous firing activity. Hence it also provides a possible functional explanation for trial-to-trial variability, which is characteristic for cortical networks of neurons but has no analogue in currently existing artificial computing systems. In addition our model demonstrates that reward-modulated STDP can be applied to all synapses in a large recurrent neural network without endangering the stability of the network dynamics.     

Time-Warp–Invariant Neuronal Processing

Fluctuations in the temporal durations of sensory signals constitute a major source of variability within natural stimulus ensembles. The neuronal mechanisms through which sensory systems can stabilize perception against such fluctuations are largely unknown. An intriguing instantiation of such robustness occurs in human speech perception, which relies critically on temporal acoustic cues that are embedded in signals with highly variable duration. Across different instances of natural speech, auditory cues can undergo temporal warping that ranges from 2-fold compression to 2-fold dilation without significant perceptual impairment. Here, we report that time-warp–invariant neuronal processing can be subserved by the shunting action of synaptic conductances that automatically rescales the effective integration time of postsynaptic neurons. We propose a novel spike-based learning rule for synaptic conductances that adjusts the degree of synaptic shunting to the temporal processing requirements of a given task. Applying this general biophysical mechanism to the example of speech processing, we propose a neuronal network model for time-warp–invariant word discrimination and demonstrate its excellent performance on a standard benchmark speech-recognition task. Our results demonstrate the important functional role of synaptic conductances in spike-based neuronal information processing and learning. The biophysics of temporal integration at neuronal membranes can endow sensory pathways with powerful time-warp–invariant computational capabilities.     

A quantitative dynamical systems approach to differential learning: self-organization principle and order parameter equations

Differential learning is a learning concept that assists subjects to find individual optimal performance patterns for given complex motor skills. To this end, training is provided in terms of noisy training sessions that feature a large variety of between-exercises differences. In several previous experimental studies it has been shown that performance improvement due to differential learning is higher than due to traditional learning and performance improvement due to differential learning occurs even during post-training periods. In this study we develop a quantitative dynamical systems approach to differential learning. Accordingly, differential learning is regarded as a self-organized process that results in the emergence of subject- and context-dependent attractors. These attractors emerge due to noise-induced bifurcations involving order parameters in terms of learning rates. In contrast, traditional learning is regarded as an externally driven process that results in the emergence of environmentally specified attractors. Performance improvement during post-training periods is explained as an hysteresis effect. An order parameter equation for differential learning involving a fourth-order polynomial potential is discussed explicitly. New predictions concerning the relationship between traditional and differential learning are derived.     

Some results on translation invariance in the human visual system

Four experiments were conducted to study the nature of visual translation invariance in humans. In all the experiments, subjects were trained to discriminate between a previously unknown target and two non-target distractors presented at a fixed retinal location to one side of the fixation point. In a subsequent test phase, this performance was compared with the performance when the patterns were presented either centrally at the fixation point or at a location on the other side of the fixation point, opposite to the location where the patterns were learned, but where acuity was identical to what it was at the learned location. Two different experimental paradigms were used. One used an eye movement control device (Experiment 1) to ensure the eye could not move relative to the patterns to be learned. In the other three experiments, presentation duration of the patterns was restricted to a short enough period to preclude eye movements. During the training period in Experiments 1 and 2, presentation location of the patterns was centered at 2.4 deg in the periphery, whereas in Experiments 3 and 4 presentation eccentricity was reduced to 0.86 and 0.49 deg. In all four experiments performance dropped when the pattern had to be recognized at new test positions. This result suggests that the visual system does not apply a global transposition transformation to the retinal image to compensate for translations. We propose that, instead, it decomposes the image into simple features which themselves are more-or-less translation invariant. If in a given task, patterns can be discriminated using these simple features, then translation invariance will occur. If not, then translation invariance will fail or be incomplete.     

A topological paradigm for hippocampal spatial map formation using persistent homology

An animal's ability to navigate through space rests on its ability to create a mental map of its environment. The hippocampus is the brain region centrally responsible for such maps, and it has been assumed to encode geometric information (distances, angles). Given, however, that hippocampal output consists of patterns of spiking across many neurons, and downstream regions must be able to translate those patterns into accurate information about an animal's spatial environment, we hypothesized that 1) the temporal pattern of neuronal firing, particularly co-firing, is key to decoding spatial information, and 2) since co-firing implies spatial overlap of place fields, a map encoded by co-firing will be based on connectivity and adjacency, i.e., it will be a topological map. Here we test this topological hypothesis with a simple model of hippocampal activity, varying three parameters (firing rate, place field size, and number of neurons) in computer simulations of rat trajectories in three topologically and geometrically distinct test environments. Using a computational algorithm based on recently developed tools from Persistent Homology theory in the field of algebraic topology, we find that the patterns of neuronal co-firing can, in fact, convey topological information about the environment in a biologically realistic length of time. Furthermore, our simulations reveal a "learning region" that highlights the interplay between the parameters in combining to produce hippocampal states that are more or less adept at map formation. For example, within the learning region a lower number of neurons firing can be compensated by adjustments in firing rate or place field size, but beyond a certain point map formation begins to fail. We propose that this learning region provides a coherent theoretical lens through which to view conditions that impair spatial learning by altering place cell firing rates or spatial specificity.     

A rule is not a rule if it changes from case to case (a reply to Marshall's comment)

This is a reply to “Queller's rule ok: Comment on van Veelen ‘when inclusive fitness is right and when it can be wrong’ ” by James Marshall in the Journal of Theoretical Biology, in this issue.In order to circumvent the disagreement about the Price equation and focus on the issue of the predictive power of inclusive fitness for group selection models, I derive Queller's and Marshall's rule without the Price equation. Both rules however need a translation step in order to be able to link them to the group selection model in van Veelen (2009). Queller's rule applies to games with 2 players and 2 strategies, and is general. Marshall's rule on the other hand applies only to a small subset of 3-player games. His rule is correct, but for other, similarly small subsets we would get other rules. This implies that if we want a rule that applies to all symmetric games with 3 players and 2 strategies, it will have to use a vector of dimension 2 that represents population structure. More in general: for group selection models with groups of size n, a correct and general prediction will need to use a vector of dimension n−1 that represents population structure.