A model for sensorimotor control and learning

A model for motor learning, generalization, and adaptation is presented. It is shown that the equations of motion of a limb can be expressed in a parametric form that facilitates transformation of desired trajectories into plans. These parametric equations are used in conjunction with a quantized multidimensional memory organized by state variables. The memory is supplied with data derived from the analysis of practice movements. A small computer and mechanical arm are used to implement the model and study its properties. Results verify the ability to acquire new movements, adapt to mechanical loads, and generalize between similar movements.This research was done while the author was a graduate student at the Massachusetts Institute of Technology in the Artificial Intelligence Laboratory and Department of Psychology. It was supported in part by training grant NGMS 5-T01-GM01064-15     

Modeling perceptual learning with multiple interacting elements: A neural network model describing early visual perceptual learning

We introduce a neural network model of an early visual cortical area, in order to understand better results of psychophysical experiments concerning perceptual learning during odd element (pop-out) detection tasks (Ahissar and Hochstein, 1993, 1994a).The model describes a network, composed of orientation selective units, arranged in a hypercolumn structure, with receptive field properties modeled from real monkey neurons. Odd element detection is a final pattern of activity with one (or a few) salient units active. The learning algorithm used was the Associative reward-penalty (Ar-p) algorithm of reinforcement learning (Barto and Anandan, 1985), following physiological data indicating the role of supervision in cortical plasticity.Simulations show that network performance improves dramatically as the weights of inter-unit connections reach a balance between lateral iso-orientation inhibition, and facilitation from neighboring neurons with different preferred orientations. The network is able to learn even from chance performance, and in the presence of a large amount of noise in the response function. As additional tests of the model, we conducted experiments with human subjects in order to examine learning strategy and test model predictions.     

A neural model for category learning

We present a general neural model for supervised learning of pattern categories which can resolve pattern classes separated by nonlinear, essentially arbitrary boundaries. The concept of a pattern class develops from storing in memory a limited number of class elements (prototypes). Associated with each prototype is a modifiable scalar weighting factor () which effectively defines the threshold for categorization of an input with the class of the given prototype. Learning involves (1) commitment of prototypes to memory and (2) adjustment of the various factors to eliminate classification errors. In tests, the model ably defined classification boundaries that largely separated complicated pattern regions. We discuss the role which divisive inhibition might play in a possible implementation of the model by a network of neurons.This work was supported in part by the Alfred P. Sloan Foundation and the Ittleson Foundation, Inc.     

Contemporary approaches to neural circuit manipulation and mapping: focus on reward and addiction

Tying complex psychological processes to precisely defined neural circuits is a major goal of systems and behavioural neuroscience. This is critical for understanding adaptive behaviour, and also how neural systems are altered in states of psychopathology, such as addiction. Efforts to relate psychological processes relevant to addiction to activity within defined neural circuits have been complicated by neural heterogeneity. Recent advances in technology allow for manipulation and mapping of genetically and anatomically defined neurons, which when used in concert with sophisticated behavioural models, have the potential to provide great insight into neural circuit bases of behaviour. Here we discuss contemporary approaches for understanding reward and addiction, with a focus on midbrain dopamine and cortico-striato-pallidal circuits.     

A neural model for conditioned avoidance learning

A mathematical model for learning of a conditioned avoidance behavior is presented. An identification of the net excitation of a neural model (Rashevsky, N., 1960.Mathematical Biophysics. Vol. II. New York: Dover Publications, Inc.) with the instantaneous probability of response is introduced and its usefulness in discussing block-trial learning performances in the conditioned avoidance situation is outlined for normal and brain-operated animals, using experimental data collected by the author. Later, the model is applied to consecutive trial learning and connection is made with the approach of H. D. Landahl (1964. “An Avoidance Learning Situation. A Neural Net Model.”Bull. Math. Biophysics,26, 83–89; and 1965, “A Neural Net Model for Escape Learning.”Bull. Math. Biophysics,27, Special Edition, 317–328) wherein lie further data with which the model can be compared.     

A neural net model for escape learning

An escape learning situation is discussed in terms of a neural model in which a stimulus can result in a conditioned excitement and a specific conditioned response. By using the simplest relations between the strengths of conditioning and the number of reinforcements and by introducing a distribution of fluctuations occurring regularly in time, one can calculate the probabilities of various responses, as well as the various latencies, in successive trials. The results are in moderately satisfactory agreement with the data of R. L. Solomon and L. C. Wynne (Psychol. Monogr.,67, No. 4, 1953). Consequences of the model for various experimental situations are discussed. This research was supported in part by the United States Public Health Service Grant RCA GM K6 18,420 and in part by the United States Air Force through the Air Force Office of Scientific Research of the Air Research Development Command under Grant No. AF AFOSR 370-64.     

A decision-making model based on a spiking neural circuit and synaptic plasticity

To adapt to the environment and survive, most animals can control their behaviors by making decisions. The process of decision-making and responding according to cues in the environment is stable, sustainable, and learnable. Understanding how behaviors are regulated by neural circuits and the encoding and decoding mechanisms from stimuli to responses are important goals in neuroscience. From results observed in Drosophila experiments, the underlying decision-making process is discussed, and a neural circuit that implements a two-choice decision-making model is proposed to explain and reproduce the observations. Compared with previous two-choice decision making models, our model uses synaptic plasticity to explain changes in decision output given the same environment. Moreover, biological meanings of parameters of our decision-making model are discussed. In this paper, we explain at the micro-level (i.e., neurons and synapses) how observable decision-making behavior at the macro-level is acquired and achieved.     

A flexible B-spline model for multiple longitudinal biomarkers and survival

Often when jointly modeling longitudinal and survival data, we are interested in a multivariate longitudinal measure that may not fit well by linear models. To overcome this problem, we propose a joint longitudinal and survival model that has a nonparametric model for the longitudinal markers. We use cubic B-splines to specify the longitudinal model and a proportional hazards model to link the longitudinal measures to the hazard. To fit the model, we use a Markov chain Monte Carlo algorithm. We select the number of knots for the cubic B-spline model using the Conditional Predictive Ordinate (CPO) and the Deviance Information Criterion (DIC). The method and model selection approach are validated in a simulation. We apply this method to examine the link between viral load, CD4 count, and time to event in data from an AIDS clinical trial. The cubic B-spline model provides a good fit to the longitudinal data that could not be obtained with simple parametric models.     

Specificity of sensorimotor learning and the neural code: Neuronal representations in the primary motor cortex

Human studies show that the learning of a new sensorimotor mapping that requires adaptation to directional errors is local and generalizes poorly to untrained directions. We trained monkeys to learn new visuomotor rotations for only one target in space and recorded neuronal activity in the primary motor cortex before, during and after learning. Similar to humans, the monkeys showed poor transfer of learning to other directions, as observed by behavioral aftereffects for untrained directions. To test for internal representations underlying these changes, we compared two features of neuronal activity before and after learning: changes in firing rates and changes in information content. Specific elevations of firing rate were only observed in a subpopulation of cells in the motor cortex with directional properties corresponding to the locally learned rotation; namely cells only showed plasticity if their preferred direction was near the training one. We applied measures from information theory to probe for learning-related changes in the neuronal code. Single cells conveyed more information about the direction of movement and this specific improvement in encoding was correlated with an increase in the slope of the neurons' tuning curve. Further, the improved information after learning enabled a more accurate reconstruction of movement direction from neuronal populations. Our findings suggest a neural mechanism for the confined generalization of a newly acquired internal model by showing a tight relationship between the locality of learning and the properties of neurons. They also provide direct evidence for improvement in the neural code as a result of learning.     

Purposive behavior and cognitive mapping: a neural network model

This study presents a real-time, biologically plausible neural network approach to purposive behavior and cognitive mapping. The system is composed of (a) an action system, consisting of a goal-seeking neural mechanism controlled by a motivational system; and (b) a cognitive system, involving a neural cognitive map. The goal-seeking mechanism displays exploratory behavior until either (a) the goal is found or (b) an adequate prediction of the goal is generated. The cognitive map built by the network is a top logical map, i.e., it represents only the adjacency, but not distances or directions, between places. The network has recurrent and non-recurrent properties that allow the reading of the cognitive map without modifying it. Two types of predictions are introduced: fast-time and real-time predictions. Fast-time predictions are produced in advance of what occurs in real time, when the information stored in the cognitive map is used to predict the remote future. Real-time predictions are generated simultaneously with the occurrence of environmental events, when the information stored in the cognitive map is being updated. Computer simulations show that the network successfully describes latent learning and detour behavior in rats. In addition, simulations demonstrate that the network can be applied to problem-solving paradigms such as the Tower of Hanoi puzzle.     

A neural net model for multiple memory domains

Previous studies with neural nets constructed of discrete populations of formal neurons have assumed that all neurons have the same probability of connection with any other neuron in the net. However, in this new study we incorporate the behavior of the neural systems in which the neural connections can be set up by means of chemical markers carried by the individual cells. With this new approach we studied the dynamics of isolated neural nets again as well as the dynamics of neural nets with sustained inputs. Results obtained with this approach show simple and multiple hysteresis phenomena. Such hysteresis loops may be considered to represent the basis for short-term memory.     

A neural model for generating and learning a rapid movement sequence

 In this article, a neural model for generating and learning a rapid ballistic movement sequence in two-dimensional (2D) space is presented and evaluated in the light of some considerations about handwriting generation. The model is based on a central nucleus (called a planning space) consisting of a fully connected grid of leaky integrators simulating neurons, and reading an input vector Ξ (t) which represents the external movement of the end effector. The movement sequencing results in a succession of motor strokes whose instantiation is controlled by the global activation of the planning space as defined by a competitive interaction between the neurons of the grid. Constraints such as spatial accuracy and movement time are exploited for the correct synchronization of the impulse commands. These commands are then fed into a neuromuscular synergy whose output is governed by a delta lognormal equation. Each movement sequence is memorized originally as a symbolic engram representing the sequence of the principal reference points of the 2D movement. These points, called virtual targets, correspond to the targets of each single rapid motor stroke composing the movement sequence. The task during the learning phase is to detect the engram corresponding to a new observed movement; the process is controlled by the dynamics of the neural grid. Received: 16 March 1995/Accepted in revised form: 25 July 1995     

A model for complex sequence learning and reproduction in neural populations

Temporal patterns of activity which repeat above chance level in the brains of vertebrates and in the mammalian neocortex have been reported experimentally. This temporal structure is thought to subserve functions such as movement, speech, and generation of rhythms. Several studies aim to explain how particular sequences of activity are learned, stored, and reproduced. The learning of sequences is usually conceived as the creation of an excitation pathway within a homogeneous neuronal population, but models embodying the autonomous function of such a learning mechanism are fraught with concerns about stability, robustness, and biological plausibility. We present two related computational models capable of learning and reproducing sequences which come from external stimuli. Both models assume that there exist populations of densely interconnected excitatory neurons, and that plasticity can occur at the population level. The first model uses temporally asymmetric Hebbian plasticity to create excitation pathways between populations in response to activation from an external source. The transition of the activity from one population to the next is permitted by the interplay of excitatory and inhibitory populations, which results in oscillatory behavior that seems to agree with experimental findings in the mammalian neocortex. The second model contains two layers, each one like the network used in the first model, with unidirectional excitatory connections from the first to the second layer experiencing Hebbian plasticity. Input sequences presented in the second layer become associated with the ongoing first layer activity, so that this activity can later elicit the the presented sequence in the absence of input. We explore the dynamics of these models, and discuss their potential implications, particularly to working memory, oscillations, and rhythm generation.     

Probability-based model of protein-protein interactions on biological timescales

Background  

Simulation methods can assist in describing and understanding complex networks of interacting proteins, providing fresh insights into the function and regulation of biological systems. Recent studies have investigated such processes by explicitly modelling the diffusion and interactions of individual molecules. In these approaches, two entities are considered to have interacted if they come within a set cutoff distance of each other.     

A computational neural model of orientation detection based on multiple guesses: comparison of geometrical and algebraic models

The implementation of Hubel-Wiesel hypothesis that orientation selectivity of a simple cell is based on ordered arrangement of its afferent cells has some difficulties. It requires the receptive fields (RFs) of those ganglion cells (GCs) and LGN cells to be similar in size and sub-structure and highly arranged in a perfect order. It also requires an adequate number of regularly distributed simple cells to match ubiquitous edges. However, the anatomical and electrophysiological evidence is not strong enough to support this geometry-based model. These strict regularities also make the model very uneconomical in both evolution and neural computation. We propose a new neural model based on an algebraic method to estimate orientations. This approach synthesizes the guesses made by multiple GCs or LGN cells and calculates local orientation information subject to a group of constraints. This algebraic model need not obey the constraints of Hubel-Wiesel hypothesis, and is easily implemented with a neural network. By using the idea of a satisfiability problem with constraints, we also prove that the precision and efficiency of this model are mathematically practicable. The proposed model makes clear several major questions which Hubel-Wiesel model does not account for. Image-rebuilding experiments are conducted to check whether this model misses any important boundary in the visual field because of the estimation strategy. This study is significant in terms of explaining the neural mechanism of orientation detection, and finding the circuit structure and computational route in neural networks. For engineering applications, our model can be used in orientation detection and as a simulation platform for cell-to-cell communications to develop bio-inspired eye chips.     

A learning network model of the neural integrator of the oculomotor system

Certain premotor neurons of the oculomotor system fire at a rate proportional to desired eye velocity. Their output is integrated by a network of neurons to supply an eye positon command to the motoneurons of the extraocular muscles. This network, known as the neural integrator, is calibrated during infancy and then maintained through development and trauma with remarkable precision. We have modeled this system with a self-organizing neural network that learns to integrate vestibular velocity commands to generate appropriate eye movements. It learns by using current eye movement on any given trial to calculate the amount of retinal image slip and this is used as the error signal. The synaptic weights are then changed using a straightforward algorithm that is independent of the network configuration and does not necessitate backwards propagation of information. Minimization of the error in this fashion causes the network to develop multiple positive feedback loops that enable it to integrate a push-pull signal without integrating the background rate on which it rides. The network is also capable of recovering from various lesions and of generating more complicated signals to simulate induced postsaccadic drift and compensation for eye muscle mechanics.     

Expressions of multiple neuronal dynamics during sensorimotor learning in the motor cortex of behaving monkeys

Previous studies support the notion that sensorimotor learning involves multiple processes. We investigated the neuronal basis of these processes by recording single-unit activity in motor cortex of non-human primates (Macaca fascicularis), during adaptation to force-field perturbations. Perturbed trials (reaching to one direction) were practiced along with unperturbed trials (to other directions). The number of perturbed trials relative to the unperturbed ones was either low or high, in two separate practice schedules. Unsurprisingly, practice under high-rate resulted in faster learning with more pronounced generalization, as compared to the low-rate practice. However, generalization and retention of behavioral and neuronal effects following practice in high-rate were less stable; namely, the faster learning was forgotten faster. We examined two subgroups of cells and showed that, during learning, the changes in firing-rate in one subgroup depended on the number of practiced trials, but not on time. In contrast, changes in the second subgroup depended on time and practice; the changes in firing-rate, following the same number of perturbed trials, were larger under high-rate than low-rate learning. After learning, the neuronal changes gradually decayed. In the first subgroup, the decay pace did not depend on the practice rate, whereas in the second subgroup, the decay pace was greater following high-rate practice. This group shows neuronal representation that mirrors the behavioral performance, evolving faster but also decaying faster at learning under high-rate, as compared to low-rate. The results suggest that the stability of a new learned skill and its neuronal representation are affected by the acquisition schedule.