Adaptive timing in neural networks: The conditioned response

A conditioned response not only reflects knowledge of an association between two events, a CS and a US, it also reflects knowledge about the timing of these events. A neural network and set of learning rules that generates appropriately timed conditioned response waveforms is presented. The model is capable of simulating some of the basic temporal properties of conditioned responses exhibited in biological systems, including (1) decreasing onset latency during acquisition training, (2) peak amplitude accurring at the temporal locus of the US, (3) inhibition of delay, and (4) trace conditioning. The model is also capable of simulating complex CR waveforms under certain conditions, and these simulations are compared with the results of behavioral experiments. The temporally adaptive responses are achieved by virtue of stimulus trace processes that are built into the network architecture.     

Predictive Coding: A Possible Explanation of Filling-In at the Blind Spot

Filling-in at the blind spot is a perceptual phenomenon in which the visual system fills the informational void, which arises due to the absence of retinal input corresponding to the optic disc, with surrounding visual attributes. It is known that during filling-in, nonlinear neural responses are observed in the early visual area that correlates with the perception, but the knowledge of underlying neural mechanism for filling-in at the blind spot is far from complete. In this work, we attempted to present a fresh perspective on the computational mechanism of filling-in process in the framework of hierarchical predictive coding, which provides a functional explanation for a range of neural responses in the cortex. We simulated a three-level hierarchical network and observe its response while stimulating the network with different bar stimulus across the blind spot. We find that the predictive-estimator neurons that represent blind spot in primary visual cortex exhibit elevated non-linear response when the bar stimulated both sides of the blind spot. Using generative model, we also show that these responses represent the filling-in completion. All these results are consistent with the finding of psychophysical and physiological studies. In this study, we also demonstrate that the tolerance in filling-in qualitatively matches with the experimental findings related to non-aligned bars. We discuss this phenomenon in the predictive coding paradigm and show that all our results could be explained by taking into account the efficient coding of natural images along with feedback and feed-forward connections that allow priors and predictions to co-evolve to arrive at the best prediction. These results suggest that the filling-in process could be a manifestation of the general computational principle of hierarchical predictive coding of natural images.     

Artificial neural networks and the study of evolution of prey coloration

In this paper, I investigate the use of artificial neural networks in the study of prey coloration. I briefly review the anti-predator functions of prey coloration and describe both in general terms and with help of two studies as specific examples the use of neural network models in the research on prey coloration. The first example investigates the effect of visual complexity of background on evolution of camouflage. The second example deals with the evolutionary choice of defence strategy, crypsis or aposematism. I conclude that visual information processing by predators is central in evolution of prey coloration. Therefore, the capability to process patterns as well as to imitate aspects of predator's information processing and responses to visual information makes neural networks a well-suited modelling approach for the study of prey coloration. In addition, their suitability for evolutionary simulations is an advantage when complex or dynamic interactions are modelled. Since not all behaviours of neural network models are necessarily biologically relevant, it is important to validate a neural network model with empirical data. Bringing together knowledge about neural networks with knowledge about topics of prey coloration would provide a potential way to deepen our understanding of the specific appearances of prey coloration.     

Stimulus detection after interruption of the feedforward response in a backward masking paradigm

In backward masking, a target stimulus is rendered invisible by the presentation of a second stimulus, the mask. When the mask is effective, neural responses to the target are suppressed. Nevertheless, weak target responses sometimes may produce a behavioural response. It remains unclear whether the reduced target response is a purely feedforward response or that it includes recurrent activity. Using a feedforward neural network of biological plausible spiking neurons, we tested whether a transient spike burst is sufficient for face categorization. After training the network, the system achieved face/non-face categorization for sets of grayscale images. In a backward masking paradigm, the transient burst response was cut off thereby reducing the feedforward target response. Despite the suppressed feedforward responses stimulus classification remained robust. Thus according to our model data stimulus detection is possible with purely, suppressed feedforward responses.     

Linear systems analysis of the relationship between firing of deep cerebellar neurons and the classically conditioned nictitating membrane response in rabbits

The correlation of the activity of neurons in the interposed and dentate nuclei of the cerebellum with conditioned movements of the nictitating membrane was investigated using linear systems analysis. The activity of single deep cerebellar nuclear cells was assumed to be the input to a linear system that produced nictitating membrane movement. Data were initially analyzed with a causal model to assess the degree to which past neural activity predicted the conditioned response. 55 of 165 cells had correlation coefficients of 0.50 or greater between the model's moment-to-moment output and the actual output, with two interpositus cells having correlation coefficients of greater than 0.90. Double-sided impulse responses indicated that afference from the face and efference copy probably affect deep cerebellar neural activity. Nonlinearities were also found in the relationship between neuronal activity and conditioned movement. It was concluded that cerebellar deep nuclear firing is highly correlated with future nictitating membrane movements but that the firing-movement relationship contains noncausal and nonlinear components.     

An oscillator network exhibiting a long-lasting response to an external signal

This study proposes an oscillator network to model the long-lasting responses observed in neural circuits. The responses of the proposed network model are represented by the temporal synchronization of the oscillators. The response duration does not depend on the natural frequency of the oscillators, which allows the responses to last much longer than the oscillation period of the oscillators. We can control the response duration by tuning the connection strengths between the oscillators and the external signal that triggers the responses. It is possible to break and restart the responses regardless of the way in which the oscillators are connected.     

A Predictive Reinforcement Model of Dopamine Neurons for Learning Approach Behavior

A neural network model of how dopamine and prefrontal cortex activity guides short- and long-term information processing within the cortico-striatal circuits during reward-related learning of approach behavior is proposed. The model predicts two types of reward-related neuronal responses generated during learning: (1) cell activity signaling errors in the prediction of the expected time of reward delivery and (2) neural activations coding for errors in the prediction of the amount and type of reward or stimulus expectancies. The former type of signal is consistent with the responses of dopaminergic neurons, while the latter signal is consistent with reward expectancy responses reported in the prefrontal cortex. It is shown that a neural network architecture that satisfies the design principles of the adaptive resonance theory of Carpenter and Grossberg (1987) can account for the dopamine responses to novelty, generalization, and discrimination of appetitive and aversive stimuli. These hypotheses are scrutinized via simulations of the model in relation to the delivery of free food outside a task, the timed contingent delivery of appetitive and aversive stimuli, and an asymmetric, instructed delay response task.     

Behavioral and neural analysis of extinction

The neural mechanisms by which fear is inhibited are poorly understood at the present time. Behaviorally, a conditioned fear response may be reduced in intensity through a number of means. Among the simplest of these is extinction, a form of learning characterized by a decrease in the amplitude and frequency of a conditioned response when the conditioned stimulus that elicits it is repeatedly nonreinforced. Because clinical interventions for patients suffering from fear dysregulation seek to inhibit abnormal, presumably learned fear responses, an understanding of fear extinction is likely to inform and increase the efficacy of these forms of treatment. This review considers the behavioral, cellular, and molecular literatures on extinction and presents the most recent advances in our understanding while identifying issues that require considerable further research.     

On looking for neural networks and “cell assemblies” that underlie behavior: II. Neural realization of the mathematical model

Part I (P. H. Greene,Bull. Math. Biophysics,24, 247–275, 1962) discussed a number of formal properties of animal behavior, and presented evidence that these properties would follow naturally from a model in which patterns of neural activity in perception or motor action constituted the resonant responses of linear neural networks. Equations were derived for parameters characterizing networks which would possess desired resonant responses. These equations expressed purely mathematical requirements. The present paper shows that a simple neural model would be entirely adequate to meet these requirements. According to this model, an input locus may become functionally connected to a particular resonant response mode by firing at a frequency which comes to approach the resonant frequency of that mode. The information in a complicated “cell assembly” of the type considered could be transmitted through a nerve tract by a very simple frequency code. One neurological guess is that frequency-coded inputs excite the transients in dendritic networks. If the amplitude of the pattern becomes large, as it would near resonance, the all-or-none axonal response would become excited. This axonal response would tend to augment resonant patterns and disrupt other patterns, for a reason inherent in any linear network. Since resonant responses are automatically present in any linear network, unless special processes suppress them, they must have led to overt behavior in animals first possessing such networks. Evolution either suppressed this feature or exploited it. Since its properties resemble those of animal behavior, the latter might be suspected. Some implications are presented regarding what a physiologist might have to look for when he studies a neural system. This research was supported by the Office of Naval Research under Contract No. Nonr 2121(17) NR 049-148. Reproduction in whole or in part is permitted for any purpose of the United States Government.     

A theory for the use of visual orientation information which exploits the columnar structure of striate cortex

A neural model is constructed based on the structure of a visual orientation hypercolumn in mammalian striate cortex. It is then assumed that the perceived orientation of visual contours is determined by the pattern of neuronal activity across orientation columns. Using statistical estimation theory, limits on the precision of orientation estimation and discrimination are calculated. These limits are functions of single unit response properties such as orientation tuning width, response amplitude and response variability, as well as the degree of organization in the neural network. It is shown that a network of modest size, consisting of broadly orientation selective units, can reliably discriminate orientation with a precision equivalent to human performance. Of the various network parameters, the discrimination threshold depends most critically on the number of cells in the hypercolumn. The form of the dependence on cell number correctly predicts the results of psychophysical studies of orientation discrimination. The model system's performance is also consistent with psychophysical data in two situations in which human performance is not optimal. First, interference with orientation discrimination occurs when multiple stimuli activate cells in the same hypercolumn. Second, systematic errors in the estimation of orientation can occur when a stimulus is composed of intersecting lines. The results demonstrate that it is possible to relate neural activity to visual performance by an examination of the pattern of activity across orientation columns. This provides support for the hypothesis that perceived orientation is determined by the distributed pattern of neural activity. The results also encourage the view of neural activity. The results also are determined by the responses of many neurons rather than the sensitivity of individual cells.     

Temporal coding in the guinea-pig auditory cortex as revealed by optical imaging and its pattern-time-series analysis

The neural network structure of a guinea-pig's primary auditory cortex is estimated by applying pattern-time-series analysis to the auditory evoked responses. Spatiotemporal patterns in click-evoked responses, observed by optical recording with voltage-sensitive dye, are analyzed by time series analysis using a multivariable autoregressive (MAR) model. Oscillatory neural activities with a distribution of about 10 40 Hz in the click-induced evoked responses are found in the cortical response field. The cortical regions where the distributed neural oscillations are generated are identified by pattern-time-series analysis. In addition, two types of cortico-cortical connections, unilateral and bilateral connections between the cortical points, are speculated to be the causes of oscillatory neural activity transfer. It can be said that the so-called synchronized neural oscillation, in the sense of coherency or correlation between the two evoked responses at the oscillatory frequency, does not necessarily represent real corticocortical neural connections at the evoked response points.     

Language models based on Hebbian cell assemblies.

This paper demonstrates how associative neural networks as standard models for Hebbian cell assemblies can be extended to implement language processes in large-scale brain simulations. To this end the classical auto- and hetero-associative paradigms of attractor nets and synfire chains (SFCs) are combined and complemented by conditioned associations as a third principle which allows for the implementation of complex graph-like transition structures between assemblies. We show example simulations of a multiple area network for object-naming, which categorises objects in a visual hierarchy and generates different specific syntactic motor sequences ("words") in response. The formation of cell assemblies due to ongoing plasticity in a multiple area network for word learning is studied afterwards. Simulations show how assemblies can form by means of percolating activity across auditory and motor-related language areas, a process supported by rhythmic, synchronized propagating waves through the network. Simulations further reproduce differences in own EEG&MEG experiments between responses to word- versus non-word stimuli in human subjects.     

Modeling the spinal pudendo-vesical reflex for bladder control by pudendal afferent stimulation

Electrical stimulation of the pudendal nerve (PN) is a promising approach to restore continence and micturition following bladder dysfunction resulting from neurological disease or injury. Although the pudendo-vesical reflex and its physiological properties are well established, there is limited understanding of the specific neural mechanisms that mediate this reflex. We sought to develop a computational model of the spinal neural network that governs the reflex bladder response to PN stimulation. We implemented and validated a neural network architecture based on previous neuroanatomical and electrophysiological studies. Using synaptically-connected integrate and fire model neurons, we created a network model with realistic spiking behavior. The model produced expected sacral parasympathetic nucleus (SPN) neuron firing rates from prescribed neural inputs and predicted bladder activation and inhibition with different frequencies of pudendal afferent stimulation. In addition, the model matched experimental results from previous studies of temporal patterns of pudendal afferent stimulation and selective pharmacological blockade of inhibitory neurons. The frequency- and pattern-dependent effects of pudendal afferent stimulation were determined by changes in firing rate of spinal interneurons, suggesting that neural network interactions at the lumbosacral level can mediate the bladder response to different frequencies or temporal patterns of pudendal afferent stimulation. Further, the anatomical structure of excitatory and inhibitory interneurons in the network model was necessary and sufficient to reproduce the critical features of the pudendo-vesical reflex, and this model may prove useful to guide development of novel, more effective electrical stimulation techniques for bladder control.     

Social buffering ameliorates conditioned fear responses in female rats

The stress experienced by an animal is ameliorated when the animal is exposed to distressing stimuli along with a conspecific animal(s). This is known as social buffering. Previously, we found that the presence of an unfamiliar male rat induced social buffering and ameliorated conditioned fear responses of a male rat subjected to an auditory conditioned stimulus (CS). However, because our knowledge of social buffering is highly biased towards findings in male subjects, analyses using female subjects are crucial for comprehensively understanding the social buffering phenomenon. In the present studies, we assessed social buffering of conditioned fear responses in female rats. We found that the estrus cycle did not affect the intensity of the rats' fear responses to the CS or their degree of vigilance due to the presence of a conspecific animal. Based on these findings, we then assessed whether social buffering ameliorated conditioned fear responses in female rats without taking into account their estrus cycles. When fear conditioned female rats were exposed to the CS without the presence of a conspecific, they exhibited behavioral responses, including freezing, and elevated corticosterone levels. By contrast, the presence of an unfamiliar female rat suppressed these responses. Based on these findings, we conclude that social buffering can ameliorate conditioned fear responses in female rats.     

The Wisdom of Networks: A General Adaptation and Learning Mechanism of Complex Systems

I hypothesize that re‐occurring prior experience of complex systems mobilizes a fast response, whose attractor is encoded by their strongly connected network core. In contrast, responses to novel stimuli are often slow and require the weakly connected network periphery. Upon repeated stimulus, peripheral network nodes remodel the network core that encodes the attractor of the new response. This “core‐periphery learning” theory reviews and generalizes the heretofore fragmented knowledge on attractor formation by neural networks, periphery‐driven innovation, and a number of recent reports on the adaptation of protein, neuronal, and social networks. The core‐periphery learning theory may increase our understanding of signaling, memory formation, information encoding and decision‐making processes. Moreover, the power of network periphery‐related “wisdom of crowds” inventing creative, novel responses indicates that deliberative democracy is a slow yet efficient learning strategy developed as the success of a billion‐year evolution. Also see the video abstract here: https://youtu.be/IIjP7zWGjVE .     

Dual Reward Prediction Components Yield Pavlovian Sign- and Goal-Tracking

Reinforcement learning (RL) has become a dominant paradigm for understanding animal behaviors and neural correlates of decision-making, in part because of its ability to explain Pavlovian conditioned behaviors and the role of midbrain dopamine activity as reward prediction error (RPE). However, recent experimental findings indicate that dopamine activity, contrary to the RL hypothesis, may not signal RPE and differs based on the type of Pavlovian response (e.g. sign- and goal-tracking responses). In this study, we address this discrepancy by introducing a new neural correlate for learning reward predictions; the correlate is called “cue-evoked reward”. It refers to a recall of reward evoked by the cue that is learned through simple cue-reward associations. We introduce a temporal difference learning model, in which neural correlates of the cue itself and cue-evoked reward underlie learning of reward predictions. The animal''s reward prediction supported by these two correlates is divided into sign and goal components respectively. We relate the sign and goal components to approach responses towards the cue (i.e. sign-tracking) and the food-tray (i.e. goal-tracking) respectively. We found a number of correspondences between simulated models and the experimental findings (i.e. behavior and neural responses). First, the development of modeled responses is consistent with those observed in the experimental task. Second, the model''s RPEs were similar to dopamine activity in respective response groups. Finally, goal-tracking, but not sign-tracking, responses rapidly emerged when RPE was restored in the simulated models, similar to experiments with recovery from dopamine-antagonist. These results suggest two complementary neural correlates, corresponding to the cue and its evoked reward, form the basis for learning reward predictions in the sign- and goal-tracking rats.     

Mechanism of gain modulation at single neuron and network levels

Gain modulation, in which the sensitivity of a neural response to one input is modified by a second input, is studied at single-neuron and network levels. At the single neuron level, gain modulation can arise if the two inputs are subject to a direct multiplicative interaction. Alternatively, these inputs can be summed in a linear manner by the neuron and gain modulation can arise, instead, from a nonlinear input–output relationship. We derive a mathematical constraint that can distinguish these two mechanisms even though they can look very similar, provided sufficient data of the appropriate type are available. Previously, it has been shown in coordinate transformation studies that artificial neurons with sigmoid transfer functions can acquire a nonlinear additive form of gain modulation through learning-driven adjustment of synaptic weights. We use the constraint derived for single-neuron studies to compare responses in this network with those of another network model based on a biologically inspired transfer function that can support approximately multiplicative interactions.     

Conditioning and time representation in long short-term memory networks

Dopaminergic models based on the temporal-difference learning algorithm usually do not differentiate trace from delay conditioning. Instead, they use a fixed temporal representation of elapsed time since conditioned stimulus onset. Recently, a new model was proposed in which timing is learned within a long short-term memory (LSTM) artificial neural network representing the cerebral cortex (Rivest et al. in J Comput Neurosci 28(1):107–130, 2010). In this paper, that model’s ability to reproduce and explain relevant data, as well as its ability to make interesting new predictions, are evaluated. The model reveals a strikingly different temporal representation between trace and delay conditioning since trace conditioning requires working memory to remember the past conditioned stimulus while delay conditioning does not. On the other hand, the model predicts no important difference in DA responses between those two conditions when trained on one conditioning paradigm and tested on the other. The model predicts that in trace conditioning, animal timing starts with the conditioned stimulus offset as opposed to its onset. In classical conditioning, it predicts that if the conditioned stimulus does not disappear after the reward, the animal may expect a second reward. Finally, the last simulation reveals that the buildup of activity of some units in the networks can adapt to new delays by adjusting their rate of integration. Most importantly, the paper shows that it is possible, with the proposed architecture, to acquire discharge patterns similar to those observed in dopaminergic neurons and in the cerebral cortex on those tasks simply by minimizing a predictive cost function.     

Fear Conditioning in an Abdominal Pain Model: Neural Responses during Associative Learning and Extinction in Healthy Subjects

Fear conditioning is relevant for elucidating the pathophysiology of anxiety, but may also be useful in the context of chronic pain syndromes which often overlap with anxiety. Thus far, no fear conditioning studies have employed aversive visceral stimuli from the lower gastrointestinal tract. Therefore, we implemented a fear conditioning paradigm to analyze the conditioned response to rectal pain stimuli using fMRI during associative learning, extinction and reinstatement.In N = 21 healthy humans, visual conditioned stimuli (CS⁺) were paired with painful rectal distensions as unconditioned stimuli (US), while different visual stimuli (CS⁻) were presented without US. During extinction, all CSs were presented without US, whereas during reinstatement, a single, unpaired US was presented. In region-of-interest analyses, conditioned anticipatory neural activation was assessed along with perceived CS-US contingency and CS unpleasantness.Fear conditioning resulted in significant contingency awareness and valence change, i.e., learned unpleasantness of a previously neutral stimulus. This was paralleled by anticipatory activation of the anterior cingulate cortex, the somatosensory cortex and precuneus (all during early acquisition) and the amygdala (late acquisition) in response to the CS⁺. During extinction, anticipatory activation of the dorsolateral prefrontal cortex to the CS⁻ was observed. In the reinstatement phase, a tendency for parahippocampal activation was found.Fear conditioning with rectal pain stimuli is feasible and leads to learned unpleasantness of previously neutral stimuli. Within the brain, conditioned anticipatory activations are seen in core areas of the central fear network including the amygdala and the anterior cingulate cortex. During extinction, conditioned responses quickly disappear, and learning of new predictive cue properties is paralleled by prefrontal activation. A tendency for parahippocampal activation during reinstatement could indicate a reactivation of the old memory trace. Together, these findings contribute to our understanding of aversive visceral learning and memory processes relevant to the pathophysiology of chronic abdominal pain.