Somato-dendritic Synaptic Plasticity and Error-backpropagation in Active Dendrites

In the last decade dendrites of cortical neurons have been shown to nonlinearly combine synaptic inputs by evoking local dendritic spikes. It has been suggested that these nonlinearities raise the computational power of a single neuron, making it comparable to a 2-layer network of point neurons. But how these nonlinearities can be incorporated into the synaptic plasticity to optimally support learning remains unclear. We present a theoretically derived synaptic plasticity rule for supervised and reinforcement learning that depends on the timing of the presynaptic, the dendritic and the postsynaptic spikes. For supervised learning, the rule can be seen as a biological version of the classical error-backpropagation algorithm applied to the dendritic case. When modulated by a delayed reward signal, the same plasticity is shown to maximize the expected reward in reinforcement learning for various coding scenarios. Our framework makes specific experimental predictions and highlights the unique advantage of active dendrites for implementing powerful synaptic plasticity rules that have access to downstream information via backpropagation of action potentials.     

An FPGA hardware/software co-design towards evolvable spiking neural networks for robotics application

This paper presents an approach that permits the effective hardware realization of a novel Evolvable Spiking Neural Network (ESNN) paradigm on Field Programmable Gate Arrays (FPGAs). The ESNN possesses a hybrid learning algorithm that consists of a Spike Timing Dependent Plasticity (STDP) mechanism fused with a Genetic Algorithm (GA). The design and implementation direction utilizes the latest advancements in FPGA technology to provide a partitioned hardware/software co-design solution. The approach achieves the maximum FPGA flexibility obtainable for the ESNN paradigm. The algorithm was applied as an embedded intelligent system robotic controller to solve an autonomous navigation and obstacle avoidance problem.     

On filtering false positive transmembrane protein predictions

While helical transmembrane (TM) region prediction tools achieve high (>90%) success rates for real integral membrane proteins, they produce a considerable number of false positive hits in sequences of known nontransmembrane queries. We propose a modification of the dense alignment surface (DAS) method that achieves a substantial decrease in the false positive error rate. Essentially, a sequence that includes possible transmembrane regions is compared in a second step with TM segments in a sequence library of documented transmembrane proteins. If the performance of the query sequence against the library of documented TM segment-containing sequences in this test is lower than an empirical threshold, it is classified as a non-transmembrane protein. The probability of false positive prediction for trusted TM region hits is expressed in terms of E-values. The modified DAS method, the DAS-TMfilter algorithm, has an unchanged high sensitivity for TM segments ( approximately 95% detected in a learning set of 128 documented transmembrane proteins). At the same time, the selectivity measured over a non-redundant set of 526 soluble proteins with known 3D structure is approximately 99%, mainly because a large number of falsely predicted single membrane-pass proteins are eliminated by the DAS-TMfilter algorithm.     

Generative models for discovering sparse distributed representations.

We describe a hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connections to perform Bayesian perceptual inference correctly. Once perceptual inference has been performed the connection strengths can be updated using a very simple learning rule that only requires locally available information. We demonstrate that the network learns to extract sparse, distributed, hierarchical representations.     

An imperfect dopaminergic error signal can drive temporal-difference learning

An open problem in the field of computational neuroscience is how to link synaptic plasticity to system-level learning. A promising framework in this context is temporal-difference (TD) learning. Experimental evidence that supports the hypothesis that the mammalian brain performs temporal-difference learning includes the resemblance of the phasic activity of the midbrain dopaminergic neurons to the TD error and the discovery that cortico-striatal synaptic plasticity is modulated by dopamine. However, as the phasic dopaminergic signal does not reproduce all the properties of the theoretical TD error, it is unclear whether it is capable of driving behavior adaptation in complex tasks. Here, we present a spiking temporal-difference learning model based on the actor-critic architecture. The model dynamically generates a dopaminergic signal with realistic firing rates and exploits this signal to modulate the plasticity of synapses as a third factor. The predictions of our proposed plasticity dynamics are in good agreement with experimental results with respect to dopamine, pre- and post-synaptic activity. An analytical mapping from the parameters of our proposed plasticity dynamics to those of the classical discrete-time TD algorithm reveals that the biological constraints of the dopaminergic signal entail a modified TD algorithm with self-adapting learning parameters and an adapting offset. We show that the neuronal network is able to learn a task with sparse positive rewards as fast as the corresponding classical discrete-time TD algorithm. However, the performance of the neuronal network is impaired with respect to the traditional algorithm on a task with both positive and negative rewards and breaks down entirely on a task with purely negative rewards. Our model demonstrates that the asymmetry of a realistic dopaminergic signal enables TD learning when learning is driven by positive rewards but not when driven by negative rewards.     

Impact of active dendrites and structural plasticity on the memory capacity of neural tissue

We consider the combined effects of active dendrites and structural plasticity on the storage capacity of neural tissue. We compare capacity for two different modes of dendritic integration: (1) linear, where synaptic inputs are summed across the entire dendritic arbor, and (2) nonlinear, where each dendritic compartment functions as a separately thresholded neuron-like summing unit. We calculate much larger storage capacities for cells with nonlinear subunits and show that this capacity is accessible to a structural learning rule that combines random synapse formation with activity-dependent stabilization/elimination. In a departure from the common view that memories are encoded in the overall connection strengths between neurons, our results suggest that long-term information storage in neural tissue could reside primarily in the selective addressing of synaptic contacts onto dendritic subunits.     

Learning generative models of molecular dynamics

We introduce three algorithms for learning generative models of molecular structures from molecular dynamics simulations. The first algorithm learns a Bayesian-optimal undirected probabilistic model over user-specified covariates (e.g., fluctuations, distances, angles, etc). L1 regularization is used to ensure sparse models and thus reduce the risk of over-fitting the data. The topology of the resulting model reveals important couplings between different parts of the protein, thus aiding in the analysis of molecular motions. The generative nature of the model makes it well-suited to making predictions about the global effects of local structural changes (e.g., the binding of an allosteric regulator). Additionally, the model can be used to sample new conformations. The second algorithm learns a time-varying graphical model where the topology and parameters change smoothly along the trajectory, revealing the conformational sub-states. The last algorithm learns a Markov Chain over undirected graphical models which can be used to study and simulate kinetics. We demonstrate our algorithms on multiple molecular dynamics trajectories.     

Generating adaptive behaviour within a memory-prediction framework

The Memory-Prediction Framework (MPF) and its Hierarchical-Temporal Memory implementation (HTM) have been widely applied to unsupervised learning problems, for both classification and prediction. To date, there has been no attempt to incorporate MPF/HTM in reinforcement learning or other adaptive systems; that is, to use knowledge embodied within the hierarchy to control a system, or to generate behaviour for an agent. This problem is interesting because the human neocortex is believed to play a vital role in the generation of behaviour, and the MPF is a model of the human neocortex.We propose some simple and biologically-plausible enhancements to the Memory-Prediction Framework. These cause it to explore and interact with an external world, while trying to maximize a continuous, time-varying reward function. All behaviour is generated and controlled within the MPF hierarchy. The hierarchy develops from a random initial configuration by interaction with the world and reinforcement learning only. Among other demonstrations, we show that a 2-node hierarchy can learn to successfully play "rocks, paper, scissors" against a predictable opponent.     

How strongly do sequence conservation patterns and empirical scales correlate with exposure patterns of transmembrane helices of membrane proteins?

Given the difficulty in determining high-resolution structures of helical membrane proteins, sequence-based prediction methods can be useful in elucidating diverse physiological processes mediated by this important class of proteins. Predicting the angular orientations of transmembrane (TM) helices about the helix axes, based on the helix parameters from electron microscopy data, is a classical problem in this regard. This problem has triggered the development of a number of different empirical scales. Recently, sequence conservation patterns were also made use of for improved predictions. Empirical scales and sequence conservation patterns (collectively termed as "prediction scales") have also found frequent applications in other research areas of membrane proteins: for example, in structure modeling and in prediction of buried TM helices. This trend is expected to grow in the near future unless there are revolutionary developments in the experimental characterization of membrane proteins. Thus, it is timely and imperative to carry out a comprehensive benchmark test over the prediction scales proposed so far to determine their pros and cons. In the current analysis, we use exposure patterns of TM helices as a golden standard, because if one develops a prediction scale that correlates perfectly with exposure patterns of TM helices, it will enable one to predict buried residues (or buried faces) of TM helices with an accuracy of 100%. Our analysis reveals several important points. (1) It demonstrates that sequence conservation patterns are much more strongly correlated with exposure patterns of TM helices than empirical scales. (2) Scales that were specifically parameterized using structure data (structure-based scales) display stronger correlation than hydrophobicity-based scales, as expected. (3) A nonnegligible difference is observed among the structure-based scales in their correlational property, suggesting that not every learning algorithm is equally effective. (4) A straightforward framework of optimally combining sequence conservation patterns and empirical scales is proposed, which reveals that improvements gained from combining the two sources of information are not dramatic in almost all cases. In turn, this calls for the development of fundamentally different scales that capture the essentials of membrane protein folding for substantial improvements.     

Self-selected modular recurrent neural networks with postural and inertial subnetworks applied to complex movements

 It has been shown that dynamic recurrent neural networks are successful in identifying the complex mapping relationship between full-wave-rectified electromyographic (EMG) signals and limb trajectories during complex movements. These connectionist models include two types of adaptive parameters: the interconnection weights between the units and the time constants associated to each neuron-like unit; they are governed by continuous-time equations. Due to their internal structure, these models are particularly appropriate to solve dynamical tasks (with time-varying input and output signals). We show in this paper that the introduction of a modular organization dedicated to different aspects of the dynamical mapping including privileged communication channels can refine the architecture of these recurrent networks. We first divide the initial individual network into two communicating subnetworks. These two modules receive the same EMG signals as input but are involved in different identification tasks related to position and acceleration. We then show that the introduction of an artificial distance in the model (using a Gaussian modulation factor of weights) induces a reduced modular architecture based on a self-elimination of null synaptic weights. Moreover, this self-selected reduced model based on two subnetworks performs the identification task better than the original single network while using fewer free parameters (better learning curve and better identification quality). We also show that this modular network exhibits several features that can be considered as biologically plausible after the learning process: self-selection of a specific inhibitory communicating path between both subnetworks after the learning process, appearance of tonic and phasic neurons, and coherent distribution of the values of the time constants within each subnetwork. Received: 17 September 2001 / Accepted in revised form: 15 January 2002     

A neuromorphic architecture for object recognition and motion anticipation using burst-STDP

In this work we investigate the possibilities offered by a minimal framework of artificial spiking neurons to be deployed in silico. Here we introduce a hierarchical network architecture of spiking neurons which learns to recognize moving objects in a visual environment and determine the correct motor output for each object. These tasks are learned through both supervised and unsupervised spike timing dependent plasticity (STDP). STDP is responsible for the strengthening (or weakening) of synapses in relation to pre- and post-synaptic spike times and has been described as a Hebbian paradigm taking place both in vitro and in vivo. We utilize a variation of STDP learning, called burst-STDP, which is based on the notion that, since spikes are expensive in terms of energy consumption, then strong bursting activity carries more information than single (sparse) spikes. Furthermore, this learning algorithm takes advantage of homeostatic renormalization, which has been hypothesized to promote memory consolidation during NREM sleep. Using this learning rule, we design a spiking neural network architecture capable of object recognition, motion detection, attention towards important objects, and motor control outputs. We demonstrate the abilities of our design in a simple environment with distractor objects, multiple objects moving concurrently, and in the presence of noise. Most importantly, we show how this neural network is capable of performing these tasks using a simple leaky-integrate-and-fire (LIF) neuron model with binary synapses, making it fully compatible with state-of-the-art digital neuromorphic hardware designs. As such, the building blocks and learning rules presented in this paper appear promising for scalable fully neuromorphic systems to be implemented in hardware chips.     

Sleep-Dependent Reactivation of Ensembles in Motor Cortex Promotes Skill Consolidation

Despite many prior studies demonstrating offline behavioral gains in motor skills after sleep, the underlying neural mechanisms remain poorly understood. To investigate the neurophysiological basis for offline gains, we performed single-unit recordings in motor cortex as rats learned a skilled upper-limb task. We found that sleep improved movement speed with preservation of accuracy. These offline improvements were linked to both replay of task-related ensembles during non-rapid eye movement (NREM) sleep and temporal shifts that more tightly bound motor cortical ensembles to movements; such offline gains and temporal shifts were not evident with sleep restriction. Interestingly, replay was linked to the coincidence of slow-wave events and bursts of spindle activity. Neurons that experienced the most consistent replay also underwent the most significant temporal shift and binding to the motor task. Significantly, replay and the associated performance gains after sleep only occurred when animals first learned the skill; continued practice during later stages of learning (i.e., after motor kinematics had stabilized) did not show evidence of replay. Our results highlight how replay of synchronous neural activity during sleep mediates large-scale neural plasticity and stabilizes kinematics during early motor learning.     

Mirrored STDP Implements Autoencoder Learning in a Network of Spiking Neurons

The autoencoder algorithm is a simple but powerful unsupervised method for training neural networks. Autoencoder networks can learn sparse distributed codes similar to those seen in cortical sensory areas such as visual area V1, but they can also be stacked to learn increasingly abstract representations. Several computational neuroscience models of sensory areas, including Olshausen & Field’s Sparse Coding algorithm, can be seen as autoencoder variants, and autoencoders have seen extensive use in the machine learning community. Despite their power and versatility, autoencoders have been difficult to implement in a biologically realistic fashion. The challenges include their need to calculate differences between two neuronal activities and their requirement for learning rules which lead to identical changes at feedforward and feedback connections. Here, we study a biologically realistic network of integrate-and-fire neurons with anatomical connectivity and synaptic plasticity that closely matches that observed in cortical sensory areas. Our choice of synaptic plasticity rules is inspired by recent experimental and theoretical results suggesting that learning at feedback connections may have a different form from learning at feedforward connections, and our results depend critically on this novel choice of plasticity rules. Specifically, we propose that plasticity rules at feedforward versus feedback connections are temporally opposed versions of spike-timing dependent plasticity (STDP), leading to a symmetric combined rule we call Mirrored STDP (mSTDP). We show that with mSTDP, our network follows a learning rule that approximately minimizes an autoencoder loss function. When trained with whitened natural image patches, the learned synaptic weights resemble the receptive fields seen in V1. Our results use realistic synaptic plasticity rules to show that the powerful autoencoder learning algorithm could be within the reach of real biological networks.     

Protein kinase D promotes plasticity-induced F-actin stabilization in dendritic spines and regulates memory formation

Actin turnover in dendritic spines influences spine development, morphology, and plasticity, with functional consequences on learning and memory formation. In nonneuronal cells, protein kinase D (PKD) has an important role in stabilizing F-actin via multiple molecular pathways. Using in vitro models of neuronal plasticity, such as glycine-induced chemical long-term potentiation (LTP), known to evoke synaptic plasticity, or long-term depolarization block by KCl, leading to homeostatic morphological changes, we show that actin stabilization needed for the enlargement of dendritic spines is dependent on PKD activity. Consequently, impaired PKD functions attenuate activity-dependent changes in hippocampal dendritic spines, including LTP formation, cause morphological alterations in vivo, and have deleterious consequences on spatial memory formation. We thus provide compelling evidence that PKD controls synaptic plasticity and learning by regulating actin stability in dendritic spines.     

Data mining and predictive modeling of biomolecular network from biomedical literature databases

In this paper, we present a novel approach Bio-IEDM (biomedical information extraction and data mining) to integrate text mining and predictive modeling to analyze biomolecular network from biomedical literature databases. Our method consists of two phases. In phase 1, we discuss a semisupervised efficient learning approach to automatically extract biological relationships such as protein-protein interaction, protein-gene interaction from the biomedical literature databases to construct the biomolecular network. Our method automatically learns the patterns based on a few user seed tuples and then extracts new tuples from the biomedical literature based on the discovered patterns. The derived biomolecular network forms a large scale-free network graph. In phase 2, we present a novel clustering algorithm to analyze the biomolecular network graph to identify biologically meaningful subnetworks (communities). The clustering algorithm considers the characteristics of the scale-free network graphs and is based on the local density of the vertex and its neighborhood functions that can be used to find more meaningful clusters with different density level. The experimental results indicate our approach is very effective in extracting biological knowledge from a huge collection of biomedical literature. The integration of data mining and information extraction provides a promising direction for analyzing the biomolecular network     

Biomechanics - Review of approaches for performance training in spinal manipulation

Motor skills development is an inherent part of clinical training in health disciplines. The conscious use of educational theory to ground learning is receiving increasing attention across health care education. There are three distinct, yet overlapping, stages of motor skill learning; the cognitive, the integrative or associative, and the autonomous; in which a contextual framework for learning content may be structured. The learning is associated with a mapping of changes within the central nervous system by the interactive mechanisms of adaptation, use-dependent plasticity and operant reinforcement. Successful skill learning requires a sufficient amount of practice and the implementation of relevant feedback strategies in the form of knowledge of performance (KP) or knowledge of results (KR). There is a natural maturation of skills that may be accelerated by feedback. Several factors contribute to stronger skills development. "Mixture-of-experts" models systematically sequence tasks into logical blocks of theory, practice and student reflection on performance. Feedback should involve both KP and KR that compares performance to a tangible standard. Rehearsals should balance use of simulators and volunteer simulated patients to provide the full range of safe and effective learning opportunities prior to students accepting a role as care givers to the public in any clinical setting.     

Associative memory model with temporal spike coding and active dendrite

We propose a neuron model whose internal state is integrated on a two dimensional phase space composed of time and dendritic space. Here, the postsynaptic potential is given as a curved surface on the phase space. Using the proposed neuron model, we introduce a continuous-time associative memory model with spike propagation delay and multiple synaptic sites on the dendrite. We show by numerical simulation that the memory capacity is doubled due to the effects of temporal spike coding and active dendrite compared to the sparse coding associative memory model.     

TM Finder: a prediction program for transmembrane protein segments using a combination of hydrophobicity and nonpolar phase helicity scales

Based on the principle of dual prediction by segment hydrophobicity and nonpolar phase helicity, in concert with imposed threshold values of these two parameters, we developed the automated prediction program TM Finder that can successfully locate most transmembrane (TM) segments in proteins. The program uses the results of experiments on a series of host-guest TM segment mimic peptides of prototypic sequence KK AAAXAAAAAXAAWAAXAAAKKKK-amide (where X = each of the 20 commonly occurring amino acids) through which an HPLC-derived hydropathy scale, a hydrophobicity threshold for spontaneous membrane insertion, and a nonpolar phase helical propensity scale were determined. Using these scales, the optimized prediction algorithm of TM Finder defines TM segments by first searching for competent core segments using the combination of hydrophobicity and helicity scales, and then performs a gap-joining operation, which minimizes prediction bias caused by local hydrophilic residues and/or the choice of window size. In addition, the hydrophobicity threshold requirement enables TM Finder to distinguish reliably between membrane proteins and globular proteins, thereby adding an important dimension to the program. A full web version of the TM Finder program can be accessed at http://www.bioinformatics-canada.org/TM/.