An oscillatory neural network model of sparse distributed memory and novelty detection

A model of sparse distributed memory is developed that is based on phase relations between the incoming signals and an oscillatory mechanism for information processing. This includes phase-frequency encoding of input information, natural frequency adaptation among the network oscillators for storage of input signals, and a resonance amplification mechanism that responds to familiar stimuli. Simulations of this model show different types of dynamics in response to new and familiar stimuli. The application of the model to hippocampal working memory is discussed.     

Object selection by an oscillatory neural network

We describe a new solution to the problem of consecutive selection of objects in a visual scene by an oscillatory neural network with the global interaction realised through a central executive element (central oscillator). The frequency coding is used to represent greyscale images in the network. The functioning of the network is based on three main principles: (1) the synchronisation of oscillators via phase-locking, (2) adaptation of the natural frequency of the central oscillator, and (3) resonant increase of the amplitudes of the oscillators which work in-phase with the central oscillator. Examples of network simulations are presented to show the reliability of the results of consecutive selection of objects under conditions of constant and varying brightness of the objects.     

Hebbian mechanisms help explain development of multisensory integration in the superior colliculus: a neural network model

The superior colliculus (SC) integrates relevant sensory information (visual, auditory, somatosensory) from several cortical and subcortical structures, to program orientation responses to external events. However, this capacity is not present at birth, and it is acquired only through interactions with cross-modal events during maturation. Mathematical models provide a quantitative framework, valuable in helping to clarify the specific neural mechanisms underlying the maturation of the multisensory integration in the SC. We extended a neural network model of the adult SC (Cuppini et?al., Front Integr Neurosci 4:1?C15, 2010) to describe the development of this phenomenon starting from an immature state, based on known or suspected anatomy and physiology, in which: (1) AES afferents are present but weak, (2) Responses are driven from non-AES afferents, and (3) The visual inputs have a marginal spatial tuning. Sensory experience was modeled by repeatedly presenting modality-specific and cross-modal stimuli. Synapses in the network were modified by simple Hebbian learning rules. As a consequence of this exposure, (1) Receptive fields shrink and come into spatial register, and (2) SC neurons gained the adult characteristic integrative properties: enhancement, depression, and inverse effectiveness. Importantly, the unique architecture of the model guided the development so that integration became dependent on the relationship between the cortical input and the SC. Manipulations of the statistics of the experience during the development changed the integrative profiles of the neurons, and results matched well with the results of physiological studies.     

A learning network model of the neural integrator of the oculomotor system

Certain premotor neurons of the oculomotor system fire at a rate proportional to desired eye velocity. Their output is integrated by a network of neurons to supply an eye positon command to the motoneurons of the extraocular muscles. This network, known as the neural integrator, is calibrated during infancy and then maintained through development and trauma with remarkable precision. We have modeled this system with a self-organizing neural network that learns to integrate vestibular velocity commands to generate appropriate eye movements. It learns by using current eye movement on any given trial to calculate the amount of retinal image slip and this is used as the error signal. The synaptic weights are then changed using a straightforward algorithm that is independent of the network configuration and does not necessitate backwards propagation of information. Minimization of the error in this fashion causes the network to develop multiple positive feedback loops that enable it to integrate a push-pull signal without integrating the background rate on which it rides. The network is also capable of recovering from various lesions and of generating more complicated signals to simulate induced postsaccadic drift and compensation for eye muscle mechanics.     

Respiration modulates oscillatory neural network activity at rest

Despite recent advances in understanding how respiration affects neural signalling to influence perception, cognition, and behaviour, it is yet unclear to what extent breathing modulates brain oscillations at rest. We acquired respiration and resting state magnetoencephalography (MEG) data from human participants to investigate if, where, and how respiration cyclically modulates oscillatory amplitudes (2 to 150 Hz). Using measures of phase–amplitude coupling, we show respiration-modulated brain oscillations (RMBOs) across all major frequency bands. Sources of these modulations spanned a widespread network of cortical and subcortical brain areas with distinct spectrotemporal modulation profiles. Globally, delta and gamma band modulations varied with distance to the head centre, with stronger modulations at distal (versus central) cortical sites. Overall, we provide the first comprehensive mapping of RMBOs across the entire brain, highlighting respiration–brain coupling as a fundamental mechanism to shape neural processing within canonical resting state and respiratory control networks (RCNs).

Despite recent advances, it remains unclear to what extent breathing modulates brain oscillations at rest. This magnetoencephalography study in human participants identifies a widespread brain network of neural oscillations that are coupled to the respiratory rhythm.     

An unsupervised hyperspheric multi-layer feedforward neural network model

This paper introduces a novel Neural Network model intended for classification of patterns into distinct categories. Arbitrary accurate category formation in a predefined feature space is asymptotically achieved by means of an unsupervised learning algorithm. Learning takes place by assignment of labeled neurons to unrecognized input exemplars and subsequent labels merging subject to similarity constraints. Two model variants, one under category separability assumption, and a second under a more general category probability density unimodality (and non-separability) assumption are suggested. The hyperspheric nature (as opposed to hyperplanar — typical of some current classifiers) of this model and its multilayer feedforward architecture are explained. Simulation results demonstrating asymptotic convergence and excellent classification accuracy are provided.     

An artificial hysteresis binary neuron: a model suppressing the oscillatory behaviors of neural dynamics

A hysteresis binary McCulloch-Pitts neuron model is proposed in order to suppress the complicated oscillatory behaviors of neural dynamics. The artificial hysteresis binary neural network is used for scheduling time-multiplex crossbar switches in order to demonstrate the effects of hysteresis. Time-multiplex crossbar switching systems must control traffic on demand such that packet blocking probability and packet waiting time are minimized. The system using n×n processing elements solves an n×n crossbar-control problem with O(1) time, while the best existing parallel algorithm requires O(n) time. The hysteresis binary neural network maximizes the throughput of packets through a crossbar switch. The solution quality of our system does not degrade with the problem size.     

An avoidance learning situation. A neural net model

A simple avoidance situation is considered in terms of a neural net learning model. Data for the control situation can be represented by an expression having three parameters which determine the initial and the steady state activities together with the transient aspects. The introduction of a learning parameter then allows one to calculate satisfactorily the results obtained in the experimental situation in which shock is applied. This research was supported in part by the United States Air Force through the Air Force Office of Scientific Research of the Air Research Development Command under Grant No. AF AFOSR 370-63 and in part by the United States Public Health Service Grant RCA GM K6 18,420.     

An unsupervised neural network model for the development of reflex co-ordination

In this paper, we present a model for the development of connections between muscle afferents and motoneurones in the human spinal cord. The model consists of a limb with six muscles, one motoneurone pool, one pooled (Ia-like) afferent for each muscle and a central programme generator. The weights of the connections between the afferents and the motoneurone pools are adapted during centrally induced movements of the limb. The connections between the afferents and the motoneurone pools adapt in a hebbian way, using only local information present at the synapses. This neural network is tested in two examples of a limb with two degrees of freedom and six muscles. Despite the simplifications, the model predicts the pattern of autogenic and heterogenic monosynaptic reflexes quite realistically.     

A neural network model of the vestibulo-ocular reflex using a local synaptic learning rule.

Vertebrates use the vestibulo-ocular reflex to maintain clear vision during head movements. This reflex requires eye-velocity commands from the semicircular canals to be integrated (mathematically) to produce eye-position commands for the extraocular muscles. This is accomplished by a neural network in the caudal pons. A model of this network is proposed using positive feedback via lateral inhibition. The model has been adapted to a learning network. We have developed a synaptic learning rule using only local information to make the model more physiological.     

An associative memory that can form hypotheses: a phase-coded neural network

 Nonlinear associative memories as realized, e.g., by Hopfield nets are characterized by attractor-type dynamics. When fed with a starting pattern, they converge to exactly one of the stored patterns which is supposed to be most similar. These systems cannot render hypotheses of classification, i.e., render several possible answers to a given classification problem. Inspired by von der Malsburg’s correlation theory of brain function, we extend conventional neural network architectures by introducing additional dynamical variables. Assuming an oscillatory time structure of neural firing, i.e., the existence of neural clocks, we assign a so-called phase to each formal neuron. The phases explicitly describe detailed correlations of neural activities neglected in conventional neural network architectures. Implementing this extension into a simple self-organizing network based on a feature map, we present an associative memory that actually is capable of forming hypotheses of classification. Received: 6 December 1993/Accepted in revised form: 14 July 1994     

A spiking neural network model of the midbrain superior colliculus that generates saccadic motor commands

Single-unit recordings suggest that the midbrain superior colliculus (SC) acts as an optimal controller for saccadic gaze shifts. The SC is proposed to be the site within the visuomotor system where the nonlinear spatial-to-temporal transformation is carried out: the population encodes the intended saccade vector by its location in the motor map (spatial), and its trajectory and velocity by the distribution of firing rates (temporal). The neurons’ burst profiles vary systematically with their anatomical positions and intended saccade vectors, to account for the nonlinear main-sequence kinematics of saccades. Yet, the underlying collicular mechanisms that could result in these firing patterns are inaccessible to current neurobiological techniques. Here, we propose a simple spiking neural network model that reproduces the spike trains of saccade-related cells in the intermediate and deep SC layers during saccades. The model assumes that SC neurons have distinct biophysical properties for spike generation that depend on their anatomical position in combination with a center–surround lateral connectivity. Both factors are needed to account for the observed firing patterns. Our model offers a basis for neuronal algorithms for spatiotemporal transformations and bio-inspired optimal controllers.     

An oscillatory correlation model of auditory streaming

We present a neurocomputational model for auditory streaming, which is a prominent phenomenon of auditory scene analysis. The proposed model represents auditory scene analysis by oscillatory correlation, where a perceptual stream corresponds to a synchronized assembly of neural oscillators and different streams correspond to desynchronized oscillator assemblies. The underlying neural architecture is a two-dimensional network of relaxation oscillators with lateral excitation and global inhibition, where one dimension represents time and another dimension frequency. By employing dynamic connections along the frequency dimension and a random element in global inhibition, the proposed model produces a temporal coherence boundary and a fissure boundary that closely match those from the psychophysical data of auditory streaming. Several issues are discussed, including how to represent physical time and how to relate shifting synchronization to auditory attention.     

上海市景观格局的人工神经网络(ANN)模型

定量分析城市景观的空间格局,深入研究景观格局的形成机制,将有助于理解城市景观的格局与过程,分析城市化的社会、经济和生态学后果以及制定更有效的景观管理策略。研究以城市景观生态学途径,应用基于GIS的景观格局分析与人工神经网络(ANN)相结合的方法定量分析上海市城市景观格局(1994年)及其变化规律,建立了能够较好地模拟上海市景观格局对居住区用地、道路密度、人口密度、城市发展历史与黄浦江等自然、社会、经济因素响应的人工神经网络。结果表明,人工神经网络方法适于研究城市化驱动因素与城市景观格局的非线性对应关系,为景观格局形成机制和景观空间结构与生态学过程相互关系的深入研究提供了一条有效、实用的研究途径。     

Anti-Hebbian learning in a non-linear neural network

The Hebbian rule (Hebb 1949), coupled with an appropriate mechanism to limit the growth of synaptic weights, allows a neuron to learn to respond to the first principal component of the distribution of its input signals (Oja 1982). Rubner and Schulten (1990) have recently suggested the use of an anti-Hebbian rule in a network with hierarchical lateral connections. When applied to neurons with linear response functions, this model allows additional neurons to learn to respond to additional principal components (Rubner and Tavan 1989). Here we apply the model to neurons with non-linear response functions characterized by a threshold and a transition width. We propose local, unsupervised learning rules for the threshold and the transition width, and illustrate the operation of these rules with some simple examples. A network using these rules sorts the input patterns into classes, which it identifies by a binary code, with the coarser structure coded by the earlier neurons in the hierarchy.     

Modeling perceptual learning with multiple interacting elements: A neural network model describing early visual perceptual learning

We introduce a neural network model of an early visual cortical area, in order to understand better results of psychophysical experiments concerning perceptual learning during odd element (pop-out) detection tasks (Ahissar and Hochstein, 1993, 1994a).The model describes a network, composed of orientation selective units, arranged in a hypercolumn structure, with receptive field properties modeled from real monkey neurons. Odd element detection is a final pattern of activity with one (or a few) salient units active. The learning algorithm used was the Associative reward-penalty (Ar-p) algorithm of reinforcement learning (Barto and Anandan, 1985), following physiological data indicating the role of supervision in cortical plasticity.Simulations show that network performance improves dramatically as the weights of inter-unit connections reach a balance between lateral iso-orientation inhibition, and facilitation from neighboring neurons with different preferred orientations. The network is able to learn even from chance performance, and in the presence of a large amount of noise in the response function. As additional tests of the model, we conducted experiments with human subjects in order to examine learning strategy and test model predictions.     

A neural network model of the McCollough effect

A self-organizing, feature-extracting network (von der Malsburg, 1973) is extended to two feature dimensions to encompass line orientation and color. It is applied to McCollough effects, particularly longlasting, contingent-aftereffects. McCollough effects are thought to involve low-level associative memory in the form of synaptic modification. The McCollough-Malsburg Model (MMM) embodies positive synaptic modification with correlated firing of units in an input layer and an excitatory cortical layer. Computer simulation of MMM reproduces orientation-contingent color aftereffects. The model embodies many of the mechanisms thought to be operating in developmental plasticity, suggesting that equivalent mechanisms may be involved in adult long-term adaptation as well.This work was supported in part by NIH Grant No. 5 R01 NS09755-4 COM of the National Institute of Neurological Diseases and Stroke (M.A. Arbib, Principal Investigator)     

A neural model for category learning

We present a general neural model for supervised learning of pattern categories which can resolve pattern classes separated by nonlinear, essentially arbitrary boundaries. The concept of a pattern class develops from storing in memory a limited number of class elements (prototypes). Associated with each prototype is a modifiable scalar weighting factor () which effectively defines the threshold for categorization of an input with the class of the given prototype. Learning involves (1) commitment of prototypes to memory and (2) adjustment of the various factors to eliminate classification errors. In tests, the model ably defined classification boundaries that largely separated complicated pattern regions. We discuss the role which divisive inhibition might play in a possible implementation of the model by a network of neurons.This work was supported in part by the Alfred P. Sloan Foundation and the Ittleson Foundation, Inc.     

Abstractive morphological learning with a recurrent neural network

In traditional word-and-paradigm models of morphology, an inflectional system is represented via a set of exemplary paradigms. Novel wordforms are produced by analogy with previously encountered forms. This paper describes a recurrent neural network which can use this strategy to learn the paradigms of a morphologically complex language based on incomplete and randomized input. Results are given which show good performance for a range of typologically diverse languages.