Shaping embodied neural networks for adaptive goal-directed behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.     

Role of basal ganglia circuits in resisting interference by distracters: a swLORETA study

Background

The selection of task-relevant information requires both the focalization of attention on the task and resistance to interference from irrelevant stimuli. Both mechanisms rely on a dorsal frontoparietal network, while focalization additionally involves a ventral frontoparietal network. The role of subcortical structures in attention is less clear, despite the fact that the striatum interacts significantly with the frontal cortex via frontostriatal loops. One means of investigating the basal ganglia''s contributions to attention is to examine the features of P300 components (i.e. amplitude, latency, and generators) in patients with basal ganglia damage (such as in Parkinson''s disease (PD), in which attention is often impaired). Three-stimulus oddball paradigms can be used to study distracter-elicited and target-elicited P300 subcomponents.

Methodology/Principal Findings

In order to compare distracter- and target-elicited P300 components, high-density (128-channel) electroencephalograms were recorded during a three-stimulus visual oddball paradigm in 15 patients with early PD and 15 matched healthy controls. For each subject, the P300 sources were localized using standardized weighted low-resolution electromagnetic tomography (swLORETA). Comparative analyses (one-sample and two-sample t-tests) were performed using SPM5® software. The swLORETA analyses showed that PD patients displayed fewer dorsolateral prefrontal (DLPF) distracter-P300 generators but no significant differences in target-elicited P300 sources; this suggests dysfunction of the DLPF cortex when the executive frontostriatal loop is disrupted by basal ganglia damage.

Conclusions/Significance

Our results suggest that the cortical attention frontoparietal networks (mainly the dorsal one) are modulated by the basal ganglia. Disruption of this network in PD impairs resistance to distracters, which results in attention disorders.     

Sequential neural processes in abacus mental addition: an EEG and FMRI case study

Abacus experts are able to mentally calculate multi-digit numbers rapidly. Some behavioral and neuroimaging studies have suggested a visuospatial and visuomotor strategy during abacus mental calculation. However, no study up to now has attempted to dissociate temporally the visuospatial neural process from the visuomotor neural process during abacus mental calculation. In the present study, an abacus expert performed the mental addition tasks (8-digit and 4-digit addends presented in visual or auditory modes) swiftly and accurately. The 100% correct rates in this expert's task performance were significantly higher than those of ordinary subjects performing 1-digit and 2-digit addition tasks. ERPs, EEG source localizations, and fMRI results taken together suggested visuospatial and visuomotor processes were sequentially arranged during the abacus mental addition with visual addends and could be dissociated from each other temporally. The visuospatial transformation of the numbers, in which the superior parietal lobule was most likely involved, might occur first (around 380 ms) after the onset of the stimuli. The visuomotor processing, in which the superior/middle frontal gyri were most likely involved, might occur later (around 440 ms). Meanwhile, fMRI results suggested that neural networks involved in the abacus mental addition with auditory stimuli were similar to those in the visual abacus mental addition. The most prominently activated brain areas in both conditions included the bilateral superior parietal lobules (BA 7) and bilateral middle frontal gyri (BA 6). These results suggest a supra-modal brain network in abacus mental addition, which may develop from normal mental calculation networks.     

Possible mechanisms of firing patterns reorganization of the neurons of the output nuclei basal ganglia

Previously proposed unitary modification rules and known experimental data were used for understanding possible mechanisms of reorganization of firing patterns of neurons in the output basal ganglia nuclei. According to the suggested mechanism, a switch from regular-spiking to bursting activity evoked by systemic inactivation of N-methyl-d-aspartate (NMDA) receptors or dopaminic receptors mainly depends on modifications of cortico-striatal synapses whereas the opposite effect of inactivation of the same receptors directly on the output basal ganglia cells is less effective. We hypothesized that some of the output basal ganglia nuclei neurons which can generate bursting discharges due to inactivation of NMDA and dopaminic receptors are glutamatergic or cholinergic cells.     

Symmetry-breaking bifurcation: A possible mechanism for 2:1 frequency-locking in animal locomotion

The generation and control of animal locomotion is believed to involve central pattern generators — networks of neurons which are capable of producing oscillatory behavior. In the present work, the quadrupedal locomotor central pattern generator is modelled as four distinct but symmetrically coupled nonlinear oscillators. We show that the typical patterns for two such networks of oscillators include 2:1 frequency-locked oscillations. These patterns, which arise through symmetry-breaking Hopf bifurcation, correspond in part to observed patterns of 2:1 frequency-locking of limb movements during electrically elicited locomotion of decerebrate and spinal quadrupeds. We briefly describe how our theoretical predictions could be tested experimentally.     

Identification of Neural Networks That Contribute to Motion Sickness through Principal Components Analysis of Fos Labeling Induced by Galvanic Vestibular Stimulation

Motion sickness is a complex condition that includes both overt signs (e.g., vomiting) and more covert symptoms (e.g., anxiety and foreboding). The neural pathways that mediate these signs and symptoms are yet to identified. This study mapped the distribution of c-fos protein (Fos)-like immunoreactivity elicited during a galvanic vestibular stimulation paradigm that is known to induce motion sickness in felines. A principal components analysis was used to identify networks of neurons activated during this stimulus paradigm from functional correlations between Fos labeling in different nuclei. This analysis identified five principal components (neural networks) that accounted for greater than 95% of the variance in Fos labeling. Two of the components were correlated with the severity of motion sickness symptoms, and likely participated in generating the overt signs of the condition. One of these networks included neurons in locus coeruleus, medial, inferior and lateral vestibular nuclei, lateral nucleus tractus solitarius, medial parabrachial nucleus and periaqueductal gray. The second included neurons in the superior vestibular nucleus, precerebellar nuclei, periaqueductal gray, and parabrachial nuclei, with weaker associations of raphe nuclei. Three additional components (networks) were also identified that were not correlated with the severity of motion sickness symptoms. These networks likely mediated the covert aspects of motion sickness, such as affective components. The identification of five statistically independent component networks associated with the development of motion sickness provides an opportunity to consider, in network activation dimensions, the complex progression of signs and symptoms that are precipitated in provocative environments. Similar methodology can be used to parse the neural networks that mediate other complex responses to environmental stimuli.     

Mechanisms of neural integration in the parietal lobe for visual attention.

The impulse discharges of neurons in the inferior parietal association cortex (area 7) were studied in the alert, behaving rhesus monkey, trained to fixate and follow visual targets. Four classes of cells related to visual or visuomotor function were found. Cells of one of these are sensitive to visual stimuli and have large, contralateral receptive fields with maximal sensitivity in the far temporal quadrants. Cells of the other three classes are related to visuomotor functions: visual fixation, tracking, and saccades. They are neither sensory nor motor in the usual sense for they are activated only by interested fixation of gaze or tracking, or before visually evoked saccadic eye movements. They are not activated during the spontaneous saccades and fixations that the monkey makes while casually exploring his environment. It is hypothesized that the light-sensitive neurons provide the visual input to the visuomotor cells that, in turn, produce a command signal for the direction of visual attention and for shifting the focus of attention from one target to another.     

Coding of communication calls in the subcortical and cortical structures of the auditory system

The processing of species-specific communication signals in the auditory system represents an important aspect of animal behavior and is crucial for its social interactions, reproduction, and survival. In this article the neuronal mechanisms underlying the processing of communication signals in the higher centers of the auditory system--inferior colliculus (IC), medial geniculate body (MGB) and auditory cortex (AC)--are reviewed, with particular attention to the guinea pig. The selectivity of neuronal responses for individual calls in these auditory centers in the guinea pig is usually low--most neurons respond to calls as well as to artificial sounds; the coding of complex sounds in the central auditory nuclei is apparently based on the representation of temporal and spectral features of acoustical stimuli in neural networks. Neuronal response patterns in the IC reliably match the sound envelope for calls characterized by one or more short impulses, but do not exactly fit the envelope for long calls. Also, the main spectral peaks are represented by neuronal firing rates in the IC. In comparison to the IC, response patterns in the MGB and AC demonstrate a less precise representation of the sound envelope, especially in the case of longer calls. The spectral representation is worse in the case of low-frequency calls, but not in the case of broad-band calls. The emotional content of the call may influence neuronal responses in the auditory pathway, which can be demonstrated by stimulation with time-reversed calls or by measurements performed under different levels of anesthesia. The investigation of the principles of the neural coding of species-specific vocalizations offers some keys for understanding the neural mechanisms underlying human speech perception.     

History influences signal recognition: neural network models of túngara frogs

Animals often attend to only a few of the cues provided by the complex displays of conspecifics. We suggest that these perceptual biases are influenced by mechanisms of signal recognition inherited from antecedent species. We tested this hypothesis by manipulating the evolutionary history of artificial neural networks, observing how the resulting networks respond to many novel stimuli and comparing these responses to the behaviour of females in phonotaxis experiments. Networks with different evolutionary histories proved equally capable of evolving to recognize the call of the túngara frog, Physalaemus pustulosus, but exhibited distinct responses to novel stimuli. History influenced the ability of networks to predict known responses of túngara frogs; network accuracy was determined by how closely the network history approximated the hypothesized history of the túngara frog. Our findings emphasize the influence of past selection pressures on current perceptual mechanisms, and demonstrate how neural network models can be used to address behavioural questions that are intractable through traditional methods.     

Gene expression profiling of cranial sensory ganglia that transmit food intake stimuli

Peripheral cranial sensory nerves projecting into the oral cavity receive food intake stimuli and transmit sensory signals to the central nervous system. They are derived from four cranial sensory ganglia, trigeminal, geniculate, petrosal, and nodose ganglia, each of which contains multiple kinds of sensory neurons with different cell morphologies and neuronal properties. We investigated the complex properties of these neurons from the viewpoint of gene expression using DNA microarrays. The 498 genes were selected from a total of 8,740 genes as showing tissue-dependent expression on the microarray by hierarchical cluster analysis, in which several genes known to be differentially expressed in cranial sensory ganglia are included. This suggests that DNA microarray cluster analysis revealed a number of characteristic genes for sensory neurons in these ganglia. Among the selected 498 genes, 44 genes are associated with neurotransmission, such as neuropeptides, their receptors, and vesicle transport, and 26 are ion channels regulating membrane potentials. The identification of a number of genes related directly to neural properties indicates that these sensory ganglia contain heterogeneous types of neurons with different neural properties.     

Simulating vestibular compensation using recurrent back-propagation

Vestibular compensation is simulated as learning in a dynamic neural network model of the horizontal vestibulo-ocular reflex (VOR). The bilateral, three-layered VOR model consists of nonlinear units representing horizontal canal afferents, vestibular nuclei (VN) neurons and eye muscle motoneurons. Dynamic processing takes place via commissural connections that link the VN bilaterally. The intact network is trained, using recurrent back-propagation, to produce the VOR with velocity storage integration. Compensation is simulated by removing vestibular afferent input from one side and retraining the network. The time course of simulated compensation matches that observed experimentally. The behavior of model VN neurons in the compensated network also matches real data, but only if connections at the motoneurons, as well as at the VN, are allowed to be plastic. The dynamic properties of real VN neurons in compensated and normal animals are found to differ when tested with sinusoidal but not with step stimuli. The model reproduces these conflicting data, and suggests that the disagreement may be due to VN neuron nonlinearity.     

Physiology and morphology of olfactory neurons associating with the protocerebral lobe of the honeybee brain

Physiology and morphology of olfactory neurons associated with the protocerebral lobe around the alpha-lobe of the mushroom body were studied in the brain of the honeybee Apis mellifera using intracellular recording and staining techniques. The responses of neurons to behaviorally relevant odorants (a blend, and components of the Nasonov pheromone, and some other non-pheromonal odors) were recorded. Different response patterns were observed within different neurons, and often within the same neuron, in response to different stimuli. All the neurons stained had innervations in the protocerebral lobe. The cell profiles varied from cells connecting the antennal lobe with both the protocerebral and lateral protocerebral lobes (projection neurons), cells linking the pedunculus of the mushroom body with both the protocerebral and lateral protocerebral lobes (PE1 neurons), cells linking the alpha-lobe and protocerebral lobe with the calyces of the mushroom body (feedback neurons), and cells linking the alpha-lobe and protocerebral lobe with the antennal lobe (recurrent neurons), to cells connecting the protocerebral lobe with the contralateral protocerebrum (bilateral neurons). These findings suggest that the protocerebral lobe acts as an olfactory center associating with other centers, and provides multi-layered recurrent networks within the protocerebrum and between the deutocerebrum and the protocerebrum in honeybee olfactory pathways.     

Central neuronal pathways of the cardioinhibitory reflex in the isopod crustacean Bathynomus doederleini

We have already identified central neurons for cardioinhibition and cardioacceleration in Bathynomus, an isopod crustacean. The 1st thoracic ganglion (TG1) has cardioinhibitory neurons, which we call CIs, while the 2nd and 3rd thoracic ganglia (TG2 and TG3) have cardioacceleratory neurons, which we call CA1s and CA2s. We examined neuronal pathways for cardioinhibitory reflexes in whole animal preparations, using intracellular and extracellular recording methods. Cardiac inhibition in response to a variety of external stimuli was mediated by activation of CIs and inhibition of both CAs. When preparations had the ventral nerve cord intact, CIs were activated by excitatory postsynaptic potentials and CAs were inhibited by inhibitory postsynaptic potentials in response to tactile stimuli applied to sensilla setae on appendages and afferent stimuli applied to ganglionic roots of the thoracic ganglia. However, stimulation of ganglionic nerve roots of TG2 and TG3, or tactile stimulation of the body surface, failed to evoke inhibition of CAs in preparations in which both the cerebral ganglion and TG1 had been excised. These results suggest that TG1 is an indispensable central region for the excitation of CI and for inhibition of CA neurons, induced by tactile stimuli and by stimuli applied to nerve roots of TG2 and TG3.     

Rana computatrix: progress report, 1984

This paper describes an approach to analyzing mechanisms of visuomotor coordination that is twofold: top-down, to offer a coordinated control program of interacting schemata (functional units, underlying behavior, which can be activated in different combinations) to achieve behavior noted by neuroethologists; and bottom-up, to provide detailed models of neural networks that are consistent with known anatomy and physiology, but that involve additional assumptions, amenable to experimental testing, to yield a network capable of exhibiting appropriate behavior. Rana computatrix, an evolving series of models of frog and toad visuomotor coordination, is described.     

Plasticity and predetermination of mesencephalic and trunk neural crest transplanted into the region of the cardiac neural crest

The cardiac neural crest contains ectomesenchymal and neural anlagen that are necessary for normal heart development. It is not known whether other regions of the neural crest are capable of supporting normal heart development. In the experiments reported herein, quail donor embryos provided cardiac, trunk, or mesencephalic neural crest to replace or add to the chick host cardiac neural crest. Neither trunk nor mesencephalic neural crest was capable of generating ectomesenchyme competent to effect truncal septation. Addition of mesencephalic neural crest resulted in a high incidence of persistent truncus arteriosus, suggesting that ectomesenchyme derived from the mesencephalic region interferes with ectomesenchyme derived from the cardiac neural crest. Derivatives from the trunk neural crest, on the other hand, did not result in abnormal development of the truncal septum. While mesencephalic neural crest seeded the cardiac ganglia with both neurons and supporting cells, this capability was limited in the trunk neural crest to the more mature regions. These studies indicate a predetermination of the ectomesenchymal derivatives of the cranial neural crest and a possible competition of neural anlagen to form neurons and supporting cells in the cardiac ganglia.     

A role of dopamine-dependent activity reorganizations in the cortico-basal ganglia-thalamocortical loops in visual attention (hypothetical mechanism)

A mechanism of attention is proposed according to which its influence on visual processing is switched on by release of dopamine into the striatum. A dopamine release during involuntary attention is promoted by visual activation of striatonigral cells via the thalamus and subsequent disinhibition through the basal ganglia of the superior colliculus. A dopamine release during voluntary attention is promoted by activation of prefrontal cortex. The strengthening of responses of neocortical neurons to attended stimulus, and suppression of responses to other stimuli is the result of opposite modulatory action of dopamine on the efficacy of strong and weak corticostriatal inputs. This leads to changes in the output basal ganglia signals ("attentional filter") that exert disinhibitory and inhibitory influence (via the thalamus) on neocortical cells that initially were strongly and weakly activated by a stimulus, respectively. From proposed mechanism follows, that attention modulates only those components of responses of cortical neurons which latency exceeds the latency of reactions of dopaminergic cells (80-100 ms).     

The Mechanism of Saccade Motor Pattern Generation Investigated by a Large-Scale Spiking Neuron Model of the Superior Colliculus

The subcortical saccade-generating system consists of the retina, superior colliculus, cerebellum and brainstem motoneuron areas. The superior colliculus is the site of sensory-motor convergence within this basic visuomotor loop preserved throughout the vertebrates. While the system has been extensively studied, there are still several outstanding questions regarding how and where the saccade eye movement profile is generated and the contribution of respective parts within this system. Here we construct a spiking neuron model of the whole intermediate layer of the superior colliculus based on the latest anatomy and physiology data. The model consists of conductance-based spiking neurons with quasi-visual, burst, buildup, local inhibitory, and deep layer inhibitory neurons. The visual input is given from the superficial superior colliculus and the burst neurons send the output to the brainstem oculomotor nuclei. Gating input from the basal ganglia and an integral feedback from the reticular formation are also included.We implement the model in the NEST simulator and show that the activity profile of bursting neurons can be reproduced by a combination of NMDA-type and cholinergic excitatory synaptic inputs and integrative inhibitory feedback. The model shows that the spreading neural activity observed in vivo can keep track of the collicular output over time and reset the system at the end of a saccade through activation of deep layer inhibitory neurons. We identify the model parameters according to neural recording data and show that the resulting model recreates the saccade size-velocity curves known as the saccadic main sequence in behavioral studies. The present model is consistent with theories that the superior colliculus takes a principal role in generating the temporal profiles of saccadic eye movements, rather than just specifying the end points of eye movements.     

Basal ganglia neural mechanisms of natural movement sequences

Natural rodent grooming and other instinctive behavior serves as a natural model of complex movement sequences. Rodent grooming has syntactic (rule-driven) sequences and more random movement patterns. Both incorporate the same movements--only the serial structure differs. Recordings of neural activity in the dorsolateral striatum and the substantia nigra pars reticulata indicate preferential activation during syntactic sequences over more random sequences. Neurons that are responsive during syntactic grooming sequences are often unresponsive or have reverse activation profiles during kinematically similar movements that occur in flexible or random grooming sequences. Few neurons could be categorized as strictly movement related--instead they were activated only in the context of particular sequential patterns of movements. Particular sequential patterns included "syntactic chain" grooming sequences of paw, head, and body movements and also "warm-up" sequences, which consist of head and body/limb movements that precede locomotion after a period of quiet resting (Golani 1992). Activation during warm-up was less intense and less frequent than during grooming sequences, but both sequences activated neurons above baseline levels, and the same neurons sometimes responded to both sequences. The fact that striatal neurons code 2 natural sequences which are made up of different constituent movements suggests that the basal ganglia may have a generalized role in sequence control. The basal ganglia are modulated by the context of the sequence and may play an executive function in the complex natural patterns of sequenced behaviour.     

Neural mechanisms underlying motivation of mental versus physical effort

Mental and physical efforts, such as paying attention and lifting weights, have been shown to involve different brain systems. These cognitive and motor systems, respectively, include cortical networks (prefronto-parietal and precentral regions) as well as subregions of the dorsal basal ganglia (caudate and putamen). Both systems appeared sensitive to incentive motivation: their activity increases when we work for higher rewards. Another brain system, including the ventral prefrontal cortex and the ventral basal ganglia, has been implicated in encoding expected rewards. How this motivational system drives the cognitive and motor systems remains poorly understood. More specifically, it is unclear whether cognitive and motor systems can be driven by a common motivational center or if they are driven by distinct, dedicated motivational modules. To address this issue, we used functional MRI to scan healthy participants while performing a task in which incentive motivation, cognitive, and motor demands were varied independently. We reasoned that a common motivational node should (1) represent the reward expected from effort exertion, (2) correlate with the performance attained, and (3) switch effective connectivity between cognitive and motor regions depending on task demand. The ventral striatum fulfilled all three criteria and therefore qualified as a common motivational node capable of driving both cognitive and motor regions of the dorsal striatum. Thus, we suggest that the interaction between a common motivational system and the different task-specific systems underpinning behavioral performance might occur within the basal ganglia.     

Identified neuronal individuals in the buccal ganglia of Helix pomatia.

The buccal ganglia of Helix pomatia are used as model nervous structures in neurophysiological and in epileptological studies. Many basic problems concerning membrane physics, functioning of the single neurons and of neuronal networks can be studied easily using these ganglia. The model character mainly comes from the relative simplicity of this nervous system and that it contains large visually identifiable neurons. As in other invertebrate nervous systems, the large neurons have proved to be individuals showing the same functional and structural properties from one animal to the next.