A model for sensorimotor control and learning

A model for motor learning, generalization, and adaptation is presented. It is shown that the equations of motion of a limb can be expressed in a parametric form that facilitates transformation of desired trajectories into plans. These parametric equations are used in conjunction with a quantized multidimensional memory organized by state variables. The memory is supplied with data derived from the analysis of practice movements. A small computer and mechanical arm are used to implement the model and study its properties. Results verify the ability to acquire new movements, adapt to mechanical loads, and generalize between similar movements.This research was done while the author was a graduate student at the Massachusetts Institute of Technology in the Artificial Intelligence Laboratory and Department of Psychology. It was supported in part by training grant NGMS 5-T01-GM01064-15     

Modeling perceptual learning with multiple interacting elements: A neural network model describing early visual perceptual learning

We introduce a neural network model of an early visual cortical area, in order to understand better results of psychophysical experiments concerning perceptual learning during odd element (pop-out) detection tasks (Ahissar and Hochstein, 1993, 1994a).The model describes a network, composed of orientation selective units, arranged in a hypercolumn structure, with receptive field properties modeled from real monkey neurons. Odd element detection is a final pattern of activity with one (or a few) salient units active. The learning algorithm used was the Associative reward-penalty (Ar-p) algorithm of reinforcement learning (Barto and Anandan, 1985), following physiological data indicating the role of supervision in cortical plasticity.Simulations show that network performance improves dramatically as the weights of inter-unit connections reach a balance between lateral iso-orientation inhibition, and facilitation from neighboring neurons with different preferred orientations. The network is able to learn even from chance performance, and in the presence of a large amount of noise in the response function. As additional tests of the model, we conducted experiments with human subjects in order to examine learning strategy and test model predictions.     

A neural model for category learning

We present a general neural model for supervised learning of pattern categories which can resolve pattern classes separated by nonlinear, essentially arbitrary boundaries. The concept of a pattern class develops from storing in memory a limited number of class elements (prototypes). Associated with each prototype is a modifiable scalar weighting factor () which effectively defines the threshold for categorization of an input with the class of the given prototype. Learning involves (1) commitment of prototypes to memory and (2) adjustment of the various factors to eliminate classification errors. In tests, the model ably defined classification boundaries that largely separated complicated pattern regions. We discuss the role which divisive inhibition might play in a possible implementation of the model by a network of neurons.This work was supported in part by the Alfred P. Sloan Foundation and the Ittleson Foundation, Inc.     

Contemporary approaches to neural circuit manipulation and mapping: focus on reward and addiction

Tying complex psychological processes to precisely defined neural circuits is a major goal of systems and behavioural neuroscience. This is critical for understanding adaptive behaviour, and also how neural systems are altered in states of psychopathology, such as addiction. Efforts to relate psychological processes relevant to addiction to activity within defined neural circuits have been complicated by neural heterogeneity. Recent advances in technology allow for manipulation and mapping of genetically and anatomically defined neurons, which when used in concert with sophisticated behavioural models, have the potential to provide great insight into neural circuit bases of behaviour. Here we discuss contemporary approaches for understanding reward and addiction, with a focus on midbrain dopamine and cortico-striato-pallidal circuits.     

A neural circuit model of emotional learning using two pathways with different granularity and speed of information processing

We propose a neural circuit model of emotional learning using two pathways with different granularity and speed of information processing. In order to derive a precise time process, we utilized a spiking model neuron proposed by Izhikevich and spike-timing-dependent synaptic plasticity (STDP) of both excitatory and inhibitory synapses. We conducted computer simulations to evaluate the proposed model. We demonstrate some aspects of emotional learning from the perspective of the time process. The agreement of the results with the previous behavioral experiments suggests that the structure and learning process of the proposed model are appropriate.     

A plausible neural circuit for decision making and its formation based on reinforcement learning

A human’s, or lower insects’, behavior is dominated by its nervous system. Each stable behavior has its own inner steps and control rules, and is regulated by a neural circuit. Understanding how the brain influences perception, thought, and behavior is a central mandate of neuroscience. The phototactic flight of insects is a widely observed deterministic behavior. Since its movement is not stochastic, the behavior should be dominated by a neural circuit. Based on the basic firing characteristics of biological neurons and the neural circuit’s constitution, we designed a plausible neural circuit for this phototactic behavior from logic perspective. The circuit’s output layer, which generates a stable spike firing rate to encode flight commands, controls the insect’s angular velocity when flying. The firing pattern and connection type of excitatory and inhibitory neurons are considered in this computational model. We simulated the circuit’s information processing using a distributed PC array, and used the real-time average firing rate of output neuron clusters to drive a flying behavior simulation. In this paper, we also explored how a correct neural decision circuit is generated from network flow view through a bee’s behavior experiment based on the reward and punishment feedback mechanism. The significance of this study: firstly, we designed a neural circuit to achieve the behavioral logic rules by strictly following the electrophysiological characteristics of biological neurons and anatomical facts. Secondly, our circuit’s generality permits the design and implementation of behavioral logic rules based on the most general information processing and activity mode of biological neurons. Thirdly, through computer simulation, we achieved new understanding about the cooperative condition upon which multi-neurons achieve some behavioral control. Fourthly, this study aims in understanding the information encoding mechanism and how neural circuits achieve behavior control. Finally, this study also helps establish a transitional bridge between the microscopic activity of the nervous system and macroscopic animal behavior.     

A decision-making model based on a spiking neural circuit and synaptic plasticity

To adapt to the environment and survive, most animals can control their behaviors by making decisions. The process of decision-making and responding according to cues in the environment is stable, sustainable, and learnable. Understanding how behaviors are regulated by neural circuits and the encoding and decoding mechanisms from stimuli to responses are important goals in neuroscience. From results observed in Drosophila experiments, the underlying decision-making process is discussed, and a neural circuit that implements a two-choice decision-making model is proposed to explain and reproduce the observations. Compared with previous two-choice decision making models, our model uses synaptic plasticity to explain changes in decision output given the same environment. Moreover, biological meanings of parameters of our decision-making model are discussed. In this paper, we explain at the micro-level (i.e., neurons and synapses) how observable decision-making behavior at the macro-level is acquired and achieved.     

A neural model for conditioned avoidance learning

A mathematical model for learning of a conditioned avoidance behavior is presented. An identification of the net excitation of a neural model (Rashevsky, N., 1960.Mathematical Biophysics. Vol. II. New York: Dover Publications, Inc.) with the instantaneous probability of response is introduced and its usefulness in discussing block-trial learning performances in the conditioned avoidance situation is outlined for normal and brain-operated animals, using experimental data collected by the author. Later, the model is applied to consecutive trial learning and connection is made with the approach of H. D. Landahl (1964. “An Avoidance Learning Situation. A Neural Net Model.”Bull. Math. Biophysics,26, 83–89; and 1965, “A Neural Net Model for Escape Learning.”Bull. Math. Biophysics,27, Special Edition, 317–328) wherein lie further data with which the model can be compared.     

A neural net model for escape learning

An escape learning situation is discussed in terms of a neural model in which a stimulus can result in a conditioned excitement and a specific conditioned response. By using the simplest relations between the strengths of conditioning and the number of reinforcements and by introducing a distribution of fluctuations occurring regularly in time, one can calculate the probabilities of various responses, as well as the various latencies, in successive trials. The results are in moderately satisfactory agreement with the data of R. L. Solomon and L. C. Wynne (Psychol. Monogr.,67, No. 4, 1953). Consequences of the model for various experimental situations are discussed. This research was supported in part by the United States Public Health Service Grant RCA GM K6 18,420 and in part by the United States Air Force through the Air Force Office of Scientific Research of the Air Research Development Command under Grant No. AF AFOSR 370-64.     

A flexible B-spline model for multiple longitudinal biomarkers and survival

Often when jointly modeling longitudinal and survival data, we are interested in a multivariate longitudinal measure that may not fit well by linear models. To overcome this problem, we propose a joint longitudinal and survival model that has a nonparametric model for the longitudinal markers. We use cubic B-splines to specify the longitudinal model and a proportional hazards model to link the longitudinal measures to the hazard. To fit the model, we use a Markov chain Monte Carlo algorithm. We select the number of knots for the cubic B-spline model using the Conditional Predictive Ordinate (CPO) and the Deviance Information Criterion (DIC). The method and model selection approach are validated in a simulation. We apply this method to examine the link between viral load, CD4 count, and time to event in data from an AIDS clinical trial. The cubic B-spline model provides a good fit to the longitudinal data that could not be obtained with simple parametric models.     

Specificity of sensorimotor learning and the neural code: Neuronal representations in the primary motor cortex

Human studies show that the learning of a new sensorimotor mapping that requires adaptation to directional errors is local and generalizes poorly to untrained directions. We trained monkeys to learn new visuomotor rotations for only one target in space and recorded neuronal activity in the primary motor cortex before, during and after learning. Similar to humans, the monkeys showed poor transfer of learning to other directions, as observed by behavioral aftereffects for untrained directions. To test for internal representations underlying these changes, we compared two features of neuronal activity before and after learning: changes in firing rates and changes in information content. Specific elevations of firing rate were only observed in a subpopulation of cells in the motor cortex with directional properties corresponding to the locally learned rotation; namely cells only showed plasticity if their preferred direction was near the training one. We applied measures from information theory to probe for learning-related changes in the neuronal code. Single cells conveyed more information about the direction of movement and this specific improvement in encoding was correlated with an increase in the slope of the neurons' tuning curve. Further, the improved information after learning enabled a more accurate reconstruction of movement direction from neuronal populations. Our findings suggest a neural mechanism for the confined generalization of a newly acquired internal model by showing a tight relationship between the locality of learning and the properties of neurons. They also provide direct evidence for improvement in the neural code as a result of learning.     

Purposive behavior and cognitive mapping: a neural network model

This study presents a real-time, biologically plausible neural network approach to purposive behavior and cognitive mapping. The system is composed of (a) an action system, consisting of a goal-seeking neural mechanism controlled by a motivational system; and (b) a cognitive system, involving a neural cognitive map. The goal-seeking mechanism displays exploratory behavior until either (a) the goal is found or (b) an adequate prediction of the goal is generated. The cognitive map built by the network is a top logical map, i.e., it represents only the adjacency, but not distances or directions, between places. The network has recurrent and non-recurrent properties that allow the reading of the cognitive map without modifying it. Two types of predictions are introduced: fast-time and real-time predictions. Fast-time predictions are produced in advance of what occurs in real time, when the information stored in the cognitive map is used to predict the remote future. Real-time predictions are generated simultaneously with the occurrence of environmental events, when the information stored in the cognitive map is being updated. Computer simulations show that the network successfully describes latent learning and detour behavior in rats. In addition, simulations demonstrate that the network can be applied to problem-solving paradigms such as the Tower of Hanoi puzzle.     

A neural net model for multiple memory domains

Previous studies with neural nets constructed of discrete populations of formal neurons have assumed that all neurons have the same probability of connection with any other neuron in the net. However, in this new study we incorporate the behavior of the neural systems in which the neural connections can be set up by means of chemical markers carried by the individual cells. With this new approach we studied the dynamics of isolated neural nets again as well as the dynamics of neural nets with sustained inputs. Results obtained with this approach show simple and multiple hysteresis phenomena. Such hysteresis loops may be considered to represent the basis for short-term memory.     

An avoidance learning situation. A neural net model

A simple avoidance situation is considered in terms of a neural net learning model. Data for the control situation can be represented by an expression having three parameters which determine the initial and the steady state activities together with the transient aspects. The introduction of a learning parameter then allows one to calculate satisfactorily the results obtained in the experimental situation in which shock is applied. This research was supported in part by the United States Air Force through the Air Force Office of Scientific Research of the Air Research Development Command under Grant No. AF AFOSR 370-63 and in part by the United States Public Health Service Grant RCA GM K6 18,420.     

A neural model for generating and learning a rapid movement sequence

 In this article, a neural model for generating and learning a rapid ballistic movement sequence in two-dimensional (2D) space is presented and evaluated in the light of some considerations about handwriting generation. The model is based on a central nucleus (called a planning space) consisting of a fully connected grid of leaky integrators simulating neurons, and reading an input vector Ξ (t) which represents the external movement of the end effector. The movement sequencing results in a succession of motor strokes whose instantiation is controlled by the global activation of the planning space as defined by a competitive interaction between the neurons of the grid. Constraints such as spatial accuracy and movement time are exploited for the correct synchronization of the impulse commands. These commands are then fed into a neuromuscular synergy whose output is governed by a delta lognormal equation. Each movement sequence is memorized originally as a symbolic engram representing the sequence of the principal reference points of the 2D movement. These points, called virtual targets, correspond to the targets of each single rapid motor stroke composing the movement sequence. The task during the learning phase is to detect the engram corresponding to a new observed movement; the process is controlled by the dynamics of the neural grid. Received: 16 March 1995/Accepted in revised form: 25 July 1995     

A model for complex sequence learning and reproduction in neural populations

Temporal patterns of activity which repeat above chance level in the brains of vertebrates and in the mammalian neocortex have been reported experimentally. This temporal structure is thought to subserve functions such as movement, speech, and generation of rhythms. Several studies aim to explain how particular sequences of activity are learned, stored, and reproduced. The learning of sequences is usually conceived as the creation of an excitation pathway within a homogeneous neuronal population, but models embodying the autonomous function of such a learning mechanism are fraught with concerns about stability, robustness, and biological plausibility. We present two related computational models capable of learning and reproducing sequences which come from external stimuli. Both models assume that there exist populations of densely interconnected excitatory neurons, and that plasticity can occur at the population level. The first model uses temporally asymmetric Hebbian plasticity to create excitation pathways between populations in response to activation from an external source. The transition of the activity from one population to the next is permitted by the interplay of excitatory and inhibitory populations, which results in oscillatory behavior that seems to agree with experimental findings in the mammalian neocortex. The second model contains two layers, each one like the network used in the first model, with unidirectional excitatory connections from the first to the second layer experiencing Hebbian plasticity. Input sequences presented in the second layer become associated with the ongoing first layer activity, so that this activity can later elicit the the presented sequence in the absence of input. We explore the dynamics of these models, and discuss their potential implications, particularly to working memory, oscillations, and rhythm generation.     

Probability-based model of protein-protein interactions on biological timescales

Background  

Simulation methods can assist in describing and understanding complex networks of interacting proteins, providing fresh insights into the function and regulation of biological systems. Recent studies have investigated such processes by explicitly modelling the diffusion and interactions of individual molecules. In these approaches, two entities are considered to have interacted if they come within a set cutoff distance of each other.     

Thunderstruck: The ACDC model of flexible sequences and rhythms in recurrent neural circuits

Adaptive sequential behavior is a hallmark of human cognition. In particular, humans can learn to produce precise spatiotemporal sequences given a certain context. For instance, musicians can not only reproduce learned action sequences in a context-dependent manner, they can also quickly and flexibly reapply them in any desired tempo or rhythm without overwriting previous learning. Existing neural network models fail to account for these properties. We argue that this limitation emerges from the fact that sequence information (i.e., the position of the action) and timing (i.e., the moment of response execution) are typically stored in the same neural network weights. Here, we augment a biologically plausible recurrent neural network of cortical dynamics to include a basal ganglia-thalamic module which uses reinforcement learning to dynamically modulate action. This “associative cluster-dependent chain” (ACDC) model modularly stores sequence and timing information in distinct loci of the network. This feature increases computational power and allows ACDC to display a wide range of temporal properties (e.g., multiple sequences, temporal shifting, rescaling, and compositionality), while still accounting for several behavioral and neurophysiological empirical observations. Finally, we apply this ACDC network to show how it can learn the famous “Thunderstruck” song intro and then flexibly play it in a “bossa nova” rhythm without further training.