Sequence learning recodes cortical representations instead of strengthening initial ones

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.     

Statistical and Prosodic Cues for Song Segmentation Learning by Bengalese Finches (Lonchura striata var. domestica)

Juvenile songbirds learn their songs from adults. Birds do not simply learn songs verbatim but they sometimes learn parts of songs from multiple tutors and recombine these into one song sequence. How they segment a particular part and select that as a chunk and how these chunks are recombined are interesting questions to ask, because such segmentation and chunking is also considered to be a basic mechanism in human language acquisition. The song of the Bengalese finch has a complex syntax with variable note‐to‐note transition probabilities and could thus be suitable for the study of segmentation and chunking in birdsong. Thirty‐two male Bengalese finch chicks were reared in a large aviary where 11 adult tutors and 10 adult females were breeding freely. In this environment most male chicks learned songs from several tutors. The song note‐chunks that juveniles copied had higher transition probabilities and shorter silent intervals than did the boundaries of the chunks, suggesting that Bengalese finches segment songs using both statistical and prosodic cues. Thus, the Bengalese finch could prove to be an excellent model in which to study neural and behavioral mechanism for sound segmentation.     

Learning and recall in a dynamic theory of coordination patterns

A dynamic theory of learning and recall of coordination patterns is developed in the context of relative timing skills. Characterizing the coordination patterns in such skills by the collective variable, relative phase, we choose a model system in which the intrinsic pattern dynamics as well as the influence of environmental and memorized information are well understood from previous experimental and theoretical work. To describe learning we endow memorized information with dynamics which is determined by a phenomenological strategy. Similarly, additional degrees of freedom must be introduced to understand recall. As such recall variables we choose the relative strengths with which each memorized pattern acts on the pattern dynamics and model their dynamics phenomenologically. The resulting dynamical system that resembles models used in pattern recognition theory is shown to adequately describe the learning and recall processes. Moreover, due to the operational character of the theory, several predictions emerge that are open to experimental test. In particular, we show under which conditions phase transitions occur in the dynamics of the coordination patterns during learning and during recall. Considering different time scales and their relations we demonstrate how these phase transitions can be identified and observed. Other predictions include the influence of the intrinsic pattern dynamics on the recall process and the existence of history and hysteresis effects in recall. We discuss different forms of forgetting and differentiation of memorized information. The results show how a new theoretical view of learning and recall as change of behavioral dynamics can lead to a different understanding of these processes by providing testable predictions.     

Enhancement of accuracy and efficiency for RNA secondary structure prediction by sequence segmentation and MapReduce

Background

Ribonucleic acid (RNA) molecules play important roles in many biological processes including gene expression and regulation. Their secondary structures are crucial for the RNA functionality, and the prediction of the secondary structures is widely studied. Our previous research shows that cutting long sequences into shorter chunks, predicting secondary structures of the chunks independently using thermodynamic methods, and reconstructing the entire secondary structure from the predicted chunk structures can yield better accuracy than predicting the secondary structure using the RNA sequence as a whole. The chunking, prediction, and reconstruction processes can use different methods and parameters, some of which produce more accurate predictions than others. In this paper, we study the prediction accuracy and efficiency of three different chunking methods using seven popular secondary structure prediction programs that apply to two datasets of RNA with known secondary structures, which include both pseudoknotted and non-pseudoknotted sequences, as well as a family of viral genome RNAs whose structures have not been predicted before. Our modularized MapReduce framework based on Hadoop allows us to study the problem in a parallel and robust environment.

Results

On average, the maximum accuracy retention values are larger than one for our chunking methods and the seven prediction programs over 50 non-pseudoknotted sequences, meaning that the secondary structure predicted using chunking is more similar to the real structure than the secondary structure predicted by using the whole sequence. We observe similar results for the 23 pseudoknotted sequences, except for the NUPACK program using the centered chunking method. The performance analysis for 14 long RNA sequences from the Nodaviridae virus family outlines how the coarse-grained mapping of chunking and predictions in the MapReduce framework exhibits shorter turnaround times for short RNA sequences. However, as the lengths of the RNA sequences increase, the fine-grained mapping can surpass the coarse-grained mapping in performance.

Conclusions

By using our MapReduce framework together with statistical analysis on the accuracy retention results, we observe how the inversion-based chunking methods can outperform predictions using the whole sequence. Our chunk-based approach also enables us to predict secondary structures for very long RNA sequences, which is not feasible with traditional methods alone.

     

A neurodynamic account of spontaneous behaviour

The current article suggests that deterministic chaos self-organized in cortical dynamics could be responsible for the generation of spontaneous action sequences. Recently, various psychological observations have suggested that humans and primates can learn to extract statistical structures hidden in perceptual sequences experienced during active environmental interactions. Although it has been suggested that such statistical structures involve chunking or compositional primitives, their neuronal implementations in brains have not yet been clarified. Therefore, to reconstruct the phenomena, synthetic neuro-robotics experiments were conducted by using a neural network model, which is characterized by a generative model with intentional states and its multiple timescales dynamics. The experimental results showed that the robot successfully learned to imitate tutored behavioral sequence patterns by extracting the underlying transition probability among primitive actions. An analysis revealed that a set of primitive action patterns was embedded in the fast dynamics part, and the chaotic dynamics of spontaneously sequencing these action primitive patterns was structured in the slow dynamics part, provided that the timescale was adequately set for each part. It was also shown that self-organization of this type of functional hierarchy ensured robust action generation by the robot in its interactions with a noisy environment. This article discusses the correspondence of the synthetic experiments with the known hierarchy of the prefrontal cortex, the supplementary motor area, and the primary motor cortex for action generation. We speculate that deterministic dynamical structures organized in the prefrontal cortex could be essential because they can account for the generation of both intentional behaviors of fixed action sequences and spontaneous behaviors of pseudo-stochastic action sequences by the same mechanism.     

Unsupervised learning and recall of temporal sequences: an application to robotics

This paper describes an unsupervised neural network model for learning and recall of temporal patterns. The model comprises two groups of synaptic weights, named competitive feedforward and Hebbian feedback, which are responsible for encoding the static and temporal features of the sequence respectively. Three additional mechanisms allow the network to deal with complex sequences: context units, a neuron commitment equation, and redundancy in the representation of sequence states. The proposed network encodes a set of robot trajectories which may contain states in common, and retrieves them accurately in the correct order. Further tests evaluate the fault-tolerance and noise sensitivity of the proposed model.     

Unsupervised learning and temporal context to recall complex robot trajectories

An unsupervised neural network is proposed to learn and recall complex robot trajectories. Two cases are considered: (i) A single trajectory in which a particular arm configuration (state) may occur more than once, and (ii) trajectories sharing states with each other. Ambiguities occur in both cases during recall of such trajectories. The proposed model consists of two groups of synaptic weights trained by competitive and Hebbian learning laws. They are responsible for encoding spatial and temporal features of the input sequences, respectively. Three mechanisms allow the network to deal with repeated or shared states: local and global context units, neurons disabled from learning, and redundancy. The network reproduces the current and the next state of the learned sequences and is able to resolve ambiguities. The model was simulated over various sets of robot trajectories in order to evaluate learning and recall, trajectory sampling effects and robustness.     

Simulating protein folding initiation sites using an alpha‐carbon‐only knowledge‐based force field

Protein folding is a hierarchical process where structure forms locally first, then globally. Some short sequence segments initiate folding through strong structural preferences that are independent of their three‐dimensional context in proteins. We have constructed a knowledge‐based force field in which the energy functions are conditional on local sequence patterns, as expressed in the hidden Markov model for local structure (HMMSTR). Carbon‐alpha force field (CALF) builds sequence specific statistical potentials based on database frequencies for α‐carbon virtual bond opening and dihedral angles, pair‐wise contacts and hydrogen bond donor‐acceptor pairs, and simulates folding via Brownian dynamics. We introduce hydrogen bond donor and acceptor potentials as α‐carbon probability fields that are conditional on the predicted local sequence. Constant temperature simulations were carried out using 27 peptides selected as putative folding initiation sites, each 12 residues in length, representing several different local structure motifs. Each 0.6 μs trajectory was clustered based on structure. Simulation convergence or representativeness was assessed by subdividing trajectories and comparing clusters. For 21 of the 27 sequences, the largest cluster made up more than half of the total trajectory. Of these 21 sequences, 14 had cluster centers that were at most 2.6 Å root mean square deviation (RMSD) from their native structure in the corresponding full‐length protein. To assess the adequacy of the energy function on nonlocal interactions, 11 full length native structures were relaxed using Brownian dynamics simulations. Equilibrated structures deviated from their native states but retained their overall topology and compactness. A simple potential that folds proteins locally and stabilizes proteins globally may enable a more realistic understanding of hierarchical folding pathways. Proteins 2009. © 2008 Wiley‐Liss, Inc.     

Dynamic Sensorimotor Planning during Long-Term Sequence Learning: The Role of Variability,Response Chunking and Planning Errors

Many everyday skills are learned by binding otherwise independent actions into a unified sequence of responses across days or weeks of practice. Here we looked at how the dynamics of action planning and response binding change across such long timescales. Subjects (N = 23) were trained on a bimanual version of the serial reaction time task (32-item sequence) for two weeks (10 days total). Response times and accuracy both showed improvement with time, but appeared to be learned at different rates. Changes in response speed across training were associated with dynamic changes in response time variability, with faster learners expanding their variability during the early training days and then contracting response variability late in training. Using a novel measure of response chunking, we found that individual responses became temporally correlated across trials and asymptoted to set sizes of approximately 7 bound responses at the end of the first week of training. Finally, we used a state-space model of the response planning process to look at how predictive (i.e., response anticipation) and error-corrective (i.e., post-error slowing) processes correlated with learning rates for speed, accuracy and chunking. This analysis yielded non-monotonic association patterns between the state-space model parameters and learning rates, suggesting that different parts of the response planning process are relevant at different stages of long-term learning. These findings highlight the dynamic modulation of response speed, variability, accuracy and chunking as multiple movements become bound together into a larger set of responses during sequence learning.     

Differential recruitment of the sensorimotor putamen and frontoparietal cortex during motor chunking in humans

Motor chunking facilitates movement production by combining motor elements into integrated units of behavior. Previous research suggests that chunking involves two processes: concatenation, aimed at the formation of motor-motor associations between elements or sets of elements, and segmentation, aimed at the parsing of multiple contiguous elements into shorter action sets. We used fMRI to measure the trial-wise recruitment of brain regions associated with these chunking processes as healthy subjects performed a cued-sequence production task. A dynamic network analysis identified chunking structure for a set of motor sequences acquired during fMRI and collected over 3?days of training. Activity in the bilateral sensorimotor putamen positively correlated with chunk concatenation, whereas?a left-hemisphere frontoparietal network was correlated with chunk segmentation. Across subjects, there was an aggregate increase in chunk strength (concatenation) with training, suggesting that subcortical circuits play a direct role in the creation of fluid transitions across chunks.     

Modelling how cleaner fish approach an ephemeral reward task demonstrates a role for ecologically tuned chunking in the evolution of advanced cognition

What makes cognition “advanced” is an open and not precisely defined question. One perspective involves increasing the complexity of associative learning, from conditioning to learning sequences of events (“chaining”) to representing various cue combinations as “chunks.” Here we develop a weighted graph model to study the mechanism enabling chunking ability and the conditions for its evolution and success, based on the ecology of the cleaner fish Labroides dimidiatus. In some environments, cleaners must learn to serve visitor clients before resident clients, because a visitor leaves if not attended while a resident waits for service. This challenge has been captured in various versions of the ephemeral reward task, which has been proven difficult for a range of cognitively capable species. We show that chaining is the minimal requirement for solving this task in its common simplified laboratory format that involves repeated simultaneous exposure to an ephemeral and permanent food source. Adding ephemeral–ephemeral and permanent–permanent combinations, as cleaners face in the wild, requires individuals to have chunking abilities to solve the task. Importantly, chunking parameters need to be calibrated to ecological conditions in order to produce adaptive decisions. Thus, it is the fine-tuning of this ability, which may be the major target of selection during the evolution of advanced associative learning.

What makes cognition ‘advanced’ is an open and not precisely defined question. In this study, a cognitive model of cleaner fish learning the ephemeral-reward task demonstrates how a critical step in cognitive evolution may be understood as the evolution of chunking and its tuning to fit ecological conditions.     

Dynamics control of semantic processes in a hierarchical associative memory

A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks.     

Dynamics control of semantic processes in a hierarchical associative memory

 A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks. Received: 19 April 1995/Accepted in revised form: 8 August 1995     

Improved Coral Population Estimation Reveals Trends at Multiple Scales on Australia’s Great Barrier Reef

The global erosion of biodiversity presents unique challenges for identifying major changes in population dynamics, establishing their causes, and managing and conserving affected ecosystems at broad spatial scales. Adaptive learning approaches connecting different spatial scales through the transfer of hierarchical information are powerful tools to address such challenges. Here, we use a Semi-Parametric Bayesian Hierarchical (SPa-BaH) model to estimate coral cover trajectories using 16 years of a broad-scale survey on Australia’s Great Barrier Reef (GBR). The spatiotemporal variability of coral populations has been considered by separating three-tiered spatial scales and allowing for alternating phases of increasing and decreasing in the estimation of their trajectories. Model estimates revealed coral cover trajectories that were highly variable according to location but that fairly consistently declined at a regional spatial scale. Notwithstanding this general trend, individual reefs within subregions in the central part of the GBR often displayed different trajectory types between sites separated by only a few hundred meters. These coral dynamics were also associated with reduced recovery rates in the Cairns and Swain subregions. Our study highlights the importance of accounting for local variation in coral cover when estimating the spatiotemporal trends in coral cover trajectories, in this case, at the GBR scale. By retaining information at different hierarchical spatial scales, our SPa-BaH model supports better estimation of large-scale coral cover trajectories. The quantitative approaches developed here can also be applied to other species with complex dynamics thereby enhancing estimations of their trajectories at local- and larger-scales and options for their management.     

Encoding strategies dissociate prefrontal activity from working memory demand

It is often proposed that prefrontal cortex is important in organization and control of working memory contents. In some cases, effective reorganization can decrease task difficulty, implying a dissociation between frontal activity and basic memory demand. In a spatial working memory task, we studied the improvement of performance that occurs when materials can be reorganized into higher level groups or chunks. Structured sequences, encouraging reorganization and chunking, were compared with unstructured sequences. Though structured sequences were easier to remember, event-related functional magnetic resonance imaging (fMRI) showed increased activation of lateral frontal cortex, in particular during memory encoding. The results show that, even when memory demand decreases, organization of working memory contents into higher level chunks is associated with increased prefrontal activity.     

Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

Sequential behaviour is often compositional and organised across multiple time scales: a set of individual elements developing on short time scales (motifs) are combined to form longer functional sequences (syntax). Such organisation leads to a natural hierarchy that can be used advantageously for learning, since the motifs and the syntax can be acquired independently. Despite mounting experimental evidence for hierarchical structures in neuroscience, models for temporal learning based on neuronal networks have mostly focused on serial methods. Here, we introduce a network model of spiking neurons with a hierarchical organisation aimed at sequence learning on multiple time scales. Using biophysically motivated neuron dynamics and local plasticity rules, the model can learn motifs and syntax independently. Furthermore, the model can relearn sequences efficiently and store multiple sequences. Compared to serial learning, the hierarchical model displays faster learning, more flexible relearning, increased capacity, and higher robustness to perturbations. The hierarchical model redistributes the variability: it achieves high motif fidelity at the cost of higher variability in the between-motif timings.     

A categorization approach to automated ontological function annotation

Automated function prediction (AFP) methods increasingly use knowledge discovery algorithms to map sequence, structure, literature, and/or pathway information about proteins whose functions are unknown into functional ontologies, typically (a portion of) the Gene Ontology (GO). While there are a growing number of methods within this paradigm, the general problem of assessing the accuracy of such prediction algorithms has not been seriously addressed. We present first an application for function prediction from protein sequences using the POSet Ontology Categorizer (POSOC) to produce new annotations by analyzing collections of GO nodes derived from annotations of protein BLAST neighborhoods. We then also present hierarchical precision and hierarchical recall as new evaluation metrics for assessing the accuracy of any predictions in hierarchical ontologies, and discuss results on a test set of protein sequences. We show that our method provides substantially improved hierarchical precision (measure of predictions made that are correct) when applied to the nearest BLAST neighbors of target proteins, as compared with simply imputing that neighborhood's annotations to the target. Moreover, when our method is applied to a broader BLAST neighborhood, hierarchical precision is enhanced even further. In all cases, such increased hierarchical precision performance is purchased at a modest expense of hierarchical recall (measure of all annotations that get predicted at all).     

Recall of sequences based on the position of the first cue stimulus in rats

We investigated the ability of rats to recall sequences of nose-poke holes with a modified serial reaction time task. In each trial, a sequence was randomly selected and the position of the first illuminated hole, which functioned as a cue stimulus, informed the rats whether the following sequence was a predictable one or not, based on prior training. The rats responded predictively only when the cues of the predictable sequences were presented. They did not show predictive responses when the cues of unpredictable sequences were presented, even though the unpredictable sequences partially had the same order of holes as the predictable sequences. These results indicate that the rats can recall sequences on the basis of presentation of the first cue stimulus informing predictable or unpredictable sequences. Recording neuronal activity while rats perform this behavioral task would be useful to elucidate neuronal mechanisms that mediate sequence recall.     

Simulation modelling of ecological hierarchies in constructive dynamical systems

Organized complexity is a characteristic feature of ecological systems with heterogeneous components interacting at several spatio-temporal scales. The hierarchy theory is a powerful epistemological framework to describe such systems by decomposing them vertically into levels and horizontally into holons. It was at first developed in a temporal and functional perspective and then, in the context of landscape ecology, extended to a spatial and structural approach. So far, most ecological applications of this theory were restricted to observational purposes, using multi-scale analysis to describe hierarchies. In spite of an increasing attention to dynamics of hierarchically structured ecological systems, current simulation models are still very limited in their representation of self-organization in complex adaptive systems. An ontological conceptualization of the hierarchy theory is outlined, focusing on key concepts, such as levels of organization and the compound and component faces of the holons. Various existing formalisms are currently used in simulation modelling, such as system dynamics, discrete event and agent based paradigms. Their ability to express the hierarchical organization of dynamical ecological systems is discussed. It turns out that a multi-modelling approach linking all these formalisms and oriented toward the specification of a constructive dynamical system would be able to express the dynamical structure of the hierarchy (creation, destruction and change of holons) and the functional and structural links between levels of organization.