A model of cortical memory processing based on columnar organization

Recent neurophysiological experiments using mammalian brains indicated that some cortical neurons exhibit oscillatory activities which can be of functional importance in visual perception. These findings suggest that the oscillation is an ubiquitous feature of cortical information processing carried out by columns which are receiving growing attention as functional subdivisions of cortical circuitry. On the assumption that a basic functional unit is a column comprising excitatory and inhibitory neurons, a network model of cortical memory processing which can account for these oscillations is proposed. Numerical simulations revealed that for appropriately determined parameters the network can attain memory-pattern retrieval resulting from fixed-point behaviour despite the fact that columns have the characteristic of oscillators. Received: 19 March 1993/Accepted in revised form: 23 September 1993     

Cluster associative memory formation in a three-layer network of phase oscillators

A three-layer network model of oscillatory associative memory is proposed. The network is capable of storing binary images, which can be retrieved upon presenting an appropriate stimulus. Binary images are encoded in the form of the spatial distribution of oscillatory phase clusters in-phase and anti-phase relative to a reference periodic signal. The information is loaded into the network using a set of interlayer connection weights. A condition for error-free pattern retrieval is formulated, delimiting the maximal number of patterns to be stored in the memory (storage capacity). It is shown that the capacity can be significantly increased by generating an optimal alphabet (basis pattern set). The number of stored patterns can reach values of the network size (the number of oscillators in each layer), which is significantly higher than the capacity of conventional oscillatory memory models. The dynamical and information characteristics of the retrieval process based on the optimal alphabet, including the size of “attraction basins“ and the input pattern distortion admissible for error-free retrieval, are investigated.     

Extremely Dilute Modular Neuronal Networks: Neocortical Memory Retrieval Dynamics

A model of columnar networks of neocortical association areas is studied. The neuronal network is composed of many Hebbian autoassociators, or modules, each of which interacts with a relatively small number of the others, randomly chosen. Any module encodes and stores a number of elementary percepts, or features. Memory items, or patterns, are peculiar combinations of features sparsely distributed over the multi-modular network. Any feature stored in any module can be involved in several of the stored patterns; feature-sharing is in fact source of local ambiguities and, consequently, a potential cause of erroneous memory retrieval spreading through the model network in pattern completion tasks.The memory retrieval dynamics of the large modular autoassociator is investigated by combining mathematical analysis and numerical simulations. An oscillatory retrieval process is proposed that is very efficient in overcoming feature-sharing drawbacks; it requires a mechanism that modulates the robustness of local attractors to noise, and neuronal activity sparseness such that quiescent and active modules are about equally noisy to any post-synaptic module.Moreover, it is shown that statistical correlation between 'kinds' of features across the set of memory patterns can be exploited to obtain a more efficient achievement of memory retrieval capabilities.It is also shown that some spots of the network cannot be reached by retrieval activity spread if they are not directly cued by the stimulus. The locations of these activity isles depend on the pattern to retrieve, while their extension only depends (in large networks) on statistics of inter-modular connections and stored patterns. The existence of activity isles determines an upper-bound to retrieval quality that does not depend on the specific retrieval dynamics adopted, nor on whether feature-sharing is permitted. The oscillatory retrieval process nearly saturates this bound.     

Spontaneous Local Gamma Oscillation Selectively Enhances Neural Network Responsiveness

Synchronized oscillation is very commonly observed in many neuronal systems and might play an important role in the response properties of the system. We have studied how the spontaneous oscillatory activity affects the responsiveness of a neuronal network, using a neural network model of the visual cortex built from Hodgkin-Huxley type excitatory (E-) and inhibitory (I-) neurons. When the isotropic local E-I and I-E synaptic connections were sufficiently strong, the network commonly generated gamma frequency oscillatory firing patterns in response to random feed-forward (FF) input spikes. This spontaneous oscillatory network activity injects a periodic local current that could amplify a weak synaptic input and enhance the network's responsiveness. When E-E connections were added, we found that the strength of oscillation can be modulated by varying the FF input strength without any changes in single neuron properties or interneuron connectivity. The response modulation is proportional to the oscillation strength, which leads to self-regulation such that the cortical network selectively amplifies various FF inputs according to its strength, without requiring any adaptation mechanism. We show that this selective cortical amplification is controlled by E-E cell interactions. We also found that this response amplification is spatially localized, which suggests that the responsiveness modulation may also be spatially selective. This suggests a generalized mechanism by which neural oscillatory activity can enhance the selectivity of a neural network to FF inputs.     

Izhikevich神经元网络的同步与联想记忆

联想记忆是人脑的一项重要功能。以Izhikevich神经元模型为节点,构建神经网络,神经元之间采用全连结的方式;以神经元群体的时空编码（spatio-temporal coding）理论研究所构建神经网络的联想记忆功能。在加入高斯白噪声的情况下,调节网络中神经元之间的连接强度的大小,当连接强度和噪声强度达到一个阈值时网络中部分神经元同步放电,实现了存储模式的联想记忆与恢复。仿真结果表明,神经元之间的连接强度在联想记忆的过程中发挥了重要的作用,噪声可以促使神经元间的同步放电,有助于神经网络实现存储模式的联想记忆与恢复。     

Bistable,Irregular Firing and Population Oscillations in a Modular Attractor Memory Network

Attractor neural networks are thought to underlie working memory functions in the cerebral cortex. Several such models have been proposed that successfully reproduce firing properties of neurons recorded from monkeys performing working memory tasks. However, the regular temporal structure of spike trains in these models is often incompatible with experimental data. Here, we show that the in vivo observations of bistable activity with irregular firing at the single cell level can be achieved in a large-scale network model with a modular structure in terms of several connected hypercolumns. Despite high irregularity of individual spike trains, the model shows population oscillations in the beta and gamma band in ground and active states, respectively. Irregular firing typically emerges in a high-conductance regime of balanced excitation and inhibition. Population oscillations can produce such a regime, but in previous models only a non-coding ground state was oscillatory. Due to the modular structure of our network, the oscillatory and irregular firing was maintained also in the active state without fine-tuning. Our model provides a novel mechanistic view of how irregular firing emerges in cortical populations as they go from beta to gamma oscillations during memory retrieval.     

A complex-valued associative memory for storing patterns as oscillatory states

A neuron model in which the neuron state is described by a complex number is proposed. A network of these neurons, which can be used as an associative memory, operates in two distinct modes: (i) fixed point mode and (ii) oscillatory mode. Mode selection can be done by varying a continuous mode parameter, , between and . At one extreme value of (), the network has conservative dynamics, and at the other (), the dynamics are dissipative and governed by a Lyapunov function. Patterns can be stored and retrieved at any value of by, (i) a one-step outer product rule or (ii) adaptive Hebbian learning. In the fixed point mode patterns are stored as fixed points, whereas in the oscillatory mode they are encoded as phase relations among individual oscillations. By virtue of an instability in the oscillatory mode, the retrieval pattern is stable over a finite interval, the stability interval, and the pattern gradually deteriorates with time beyond this interval. However, at certain values of sparsely distributed over -space the instability disappears. The neurophysiological significance of the instability is briefly discussed. The possibility of physically interpreting dissipativity and conservativity is explored by noting that while conservativity leads to energy savings, dissipativity leads to stability and reliable retrieval. Received: 4 December 1995 / Accepted in revised form: 18 June 1996     

Transitory memory retrieval in a biologically plausible neural network model

A number of memory models have been proposed. These all have the basic structure that excitatory neurons are reciprocally connected by recurrent connections together with the connections with inhibitory neurons, which yields associative memory (i.e., pattern completion) and successive retrieval of memory. In most of the models, a simple mathematical model for a neuron in the form of a discrete map is adopted. It has not, however, been clarified whether behaviors like associative memory and successive retrieval of memory appear when a biologically plausible neuron model is used. In this paper, we propose a network model for associative memory and successive retrieval of memory based on Pinsky-Rinzel neurons. The state of pattern completion in associative memory can be observed with an appropriate balance of excitatory and inhibitory connection strengths. Increasing of the connection strength of inhibitory interneurons changes the state of memory retrieval from associative memory to successive retrieval of memory. We investigate this transition.     

记忆水平依赖的海马-前额叶神经节律交互

海马(HPC)和前额叶皮层(PFC)的协同作用是记忆加工过程的关键,其相互作用对学习和记忆功能至关重要.大量证据表明,情景记忆的形成、巩固与检索依赖于特征神经节律在PFC和HPC脑区间的同步作用,这些节律包括theta节律、gamma节律和sharp wave ripples (SWRs)节律等.在精神类疾病中患者往往伴随出现学习记忆功能障碍,基于人类和动物的脑电研究均发现以上3种神经节律在HPC和PFC之间的同步性下降,可能作为反映精神病理下认知功能障碍的重要指标.本文从HPC-PFC网络中的神经节律研究出发,总结了theta节律、gamma节律和SWRs节律在两脑区间的协调交互模式在情景记忆中的作用,以及精神分裂症和抑郁症状态下HPC-PFC通路上神经节律的异常表现及其潜在损伤机制,为今后精神疾病的快速诊断提供客观依据.     

Spiking neural network model for memorizing sequences with forward and backward recall

We present an oscillatory network of conductance based spiking neurons of Hodgkin–Huxley type as a model of memory storage and retrieval of sequences of events (or objects). The model is inspired by psychological and neurobiological evidence on sequential memories. The building block of the model is an oscillatory module which contains excitatory and inhibitory neurons with all-to-all connections. The connection architecture comprises two layers. A lower layer represents consecutive events during their storage and recall. This layer is composed of oscillatory modules. Plastic excitatory connections between the modules are implemented using an STDP type learning rule for sequential storage. Excitatory neurons in the upper layer project star-like modifiable connections toward the excitatory lower layer neurons. These neurons in the upper layer are used to tag sequences of events represented in the lower layer. Computer simulations demonstrate good performance of the model including difficult cases when different sequences contain overlapping events. We show that the model with STDP type or anti-STDP type learning rules can be applied for the simulation of forward and backward replay of neural spikes respectively.     

Roles of dynamic linkage of stable attractors across cortical networks in recalling long-term memory

 We propose a neural network model for a category-association task. By simulating the model, neuronal relevance of cortical interactions to recalling long-term memory was investigated. The model consists of the left and right hemispheres, each of which has IT (inferotemporal cortex) and PC (prefrontal cortex) networks. Information about visual features and their categories were encoded into point attractors of the IT and PC networks, respectively. In the task, the IT network of the right hemisphere was stimulated with a cue feature. After a delay period, the IT network of the left hemisphere was simultaneously stimulated with the choice feature and an irrelevant feature. The cue and choice features belong to the same category, while the irrelevant feature belongs to another category. To complete the task, the IT network must select the point attractor corresponding to the choice feature. We demonstrate that the top-down pathway (PC-to-IT) triggers the retrieval of long-term memory of the choice feature from the IT, and the bottom-up pathway (IT-to-PC) contributes to the maintenance of the retrieved memory during the delay period. The key mechanism for the retrieval and maintenance of that memory is the dynamic linkage of attractors across separate cortical networks. We show that a single hemisphere is sufficient for the memory retrieval, but it is advantageous to use the two hemispheres because the retrieved memory is thereby retained with greater reliability until the brain chooses the choice feature. Received: 4 April 2001 / Accepted in revised form: 17 September 2002 / Published online: 20 January 2003 Correspondence to: O. Hoshino (e-mail: hoshino@cc.oita-u.ac.jp, Tel.: +81-97-554-7301, Fax: +81-97-554-7507)     

A Graph Theoretical Approach to Study the Organization of the Cortical Networks during Different Mathematical Tasks

The two core systems of mathematical processing (subitizing and retrieval) as well as their functionality are already known and published. In this study we have used graph theory to compare the brain network organization of these two core systems in the cortical layer during difficult calculations. We have examined separately all the EEG frequency bands in healthy young individuals and we found that the network organization at rest, as well as during mathematical tasks has the characteristics of Small World Networks for all the bands, which is the optimum organization required for efficient information processing. The different mathematical stimuli provoked changes in the graph parameters of different frequency bands, especially the low frequency bands. More specific, in Delta band the induced network increases it’s local and global efficiency during the transition from subitizing to retrieval system, while results suggest that difficult mathematics provoke networks with higher cliquish organization due to more specific demands. The network of the Theta band follows the same pattern as before, having high nodal and remote organization during difficult mathematics. Also the spatial distribution of the network’s weights revealed more prominent connections in frontoparietal regions, revealing the working memory load due to the engagement of the retrieval system. The cortical networks of the alpha brainwaves were also more efficient, both locally and globally, during difficult mathematics, while the fact that alpha’s network was more dense on the frontparietal regions as well, reveals the engagement of the retrieval system again. Concluding, this study gives more evidences regarding the interaction of the two core systems, exploiting the produced functional networks of the cerebral cortex, especially for the difficult mathematics.     

Modelling studies on the computational function of fast temporal structure in cortical circuit activity

The interplay between modelling and experimental studies can support the exploration of the function of neuronal circuits in the cortex. We exemplify such an approach with a study on the role of spike timing and gamma-oscillations in associative memory in strongly connected circuits of cortical neurones. It is demonstrated how associative memory studies on different levels of abstraction can specify the functionality to be expected in real cortical neuronal circuits. In our model overlapping random configurations of sparse cell populations correspond to memory items that are stored by simple Hebbian coincidence learning. This associative memory task will be implemented with biophysically well tested compartmental neurones developed by Pinsky and Rinzel . We ran simulation experiments to study memory recall in two network architectures: one interconnected pool of cells, and two reciprocally connected pools. When recalling a memory by stimulating a spatially overlapping set of cells, the completed pattern is coded by an event of synchronized single spikes occurring after 25-60 ms. These fast associations are performed even at a memory load corresponding to the memory capacity of optimally tuned formal associative networks (>0.1 bit/synapse). With tonic stimulation or feedback loops in the network the neurones fire periodically in the gamma-frequency range (20-80 Hz). With fast changing inputs memory recall can be switched between items within a single gamma cycle. Thus, oscillation is not a primary coding feature necessary for associative memory. However, it accompanies reverberatory feedback providing an improved iterative memory recall completed after a few gamma cycles (60-260 ms). In the bidirectional architecture reverberations do not express in a rigid phase locking between the pools. For small stimulation sets bursting occurred in these cells acting as a supportive mechanism for associative memory.     

Gamma神经振荡产生机制及其功能研究进展

Gamma神经振荡的频率在30～100 Hz之间,存在于动物和人类大脑的多个区域,如丘脑、体感皮层以及海马等部位,在各个尺度水平上都可被检测到.抑制性中间神经元组成的神经网络是产生此高频节律性活动的主要条件之一.皮层的gamma神经振荡与丘脑-皮层系统有关.Gamma神经振荡具有易化突触可塑性和调节神经网络的作用,主要参与感觉特征绑定、选择性注意以及记忆等高级功能.     

Modelling of cortical and thalamic 600 Hz activity by means of oscillatory networks

The purpose of this study is to investigate information processing in the primary somatosensory system with the help of oscillatory network modelling. Specifically, we consider interactions in the oscillatory 600 Hz activity between the thalamus and the cortical Brodmann areas 3b and 1. This type of cortical activity occurs after electrical stimulation of peripheral nerves such as the median nerve. Our measurements consist of simultaneous 31-channel MEG and 32-channel EEG recordings and individual 3D MRI data. We perform source localization by means of a multi-dipole model. The dipole activation time courses are then modelled by a set of coupled oscillators, described by linear second-order ordinary delay differential equations (DDEs). In particular, a new model for the thalamic activity is included in the oscillatory network. The parameters of the DDE system are successfully fitted to the data by a nonlinear evolutionary optimization method. To activate the oscillatory network, an individual input function is used, based on measurements of the propagated stimulation signal at the biceps. A significant feedback from the cortex to the thalamus could be detected by comparing the network modelling with and without feedback connections. Our finding in humans is supported by earlier animal studies. We conclude that this type of rhythmic brain activity can be modelled by oscillatory networks in order to disentangle feed forward and feedback information transfer.     

Temporal coding in the guinea-pig auditory cortex as revealed by optical imaging and its pattern-time-series analysis

The neural network structure of a guinea-pig's primary auditory cortex is estimated by applying pattern-time-series analysis to the auditory evoked responses. Spatiotemporal patterns in click-evoked responses, observed by optical recording with voltage-sensitive dye, are analyzed by time series analysis using a multivariable autoregressive (MAR) model. Oscillatory neural activities with a distribution of about 10 40 Hz in the click-induced evoked responses are found in the cortical response field. The cortical regions where the distributed neural oscillations are generated are identified by pattern-time-series analysis. In addition, two types of cortico-cortical connections, unilateral and bilateral connections between the cortical points, are speculated to be the causes of oscillatory neural activity transfer. It can be said that the so-called synchronized neural oscillation, in the sense of coherency or correlation between the two evoked responses at the oscillatory frequency, does not necessarily represent real corticocortical neural connections at the evoked response points.     

A hybrid neural network of addressable and content-addressable memory

We investigate the memory structure and retrieval of the brain and propose a hybrid neural network of addressable and content-addressable memory which is a special database model and can memorize and retrieve any piece of information (a binary pattern) both addressably and content-addressably. The architecture of this hybrid neural network is hierarchical and takes the form of a tree of slabs which consist of binary neurons with the same array. Simplex memory neural networks are considered as the slabs of basic memory units, being distributed on the terminal vertexes of the tree. It is shown by theoretical analysis that the hybrid neural network is able to be constructed with Hebbian and competitive learning rules, and some other important characteristics of its learning and memory behavior are also consistent with those of the brain. Moreover, we demonstrate the hybrid neural network on a set of ten binary numeral patters     

A biologically motivated and analytically soluble model of collective oscillations in the cortex

A model of an associative network of spiking neurons with stationary states, globally locked oscillations, and weakly locked oscillatory states is presented and analyzed. The network is close to biology in the following sense. First, the neurons spike and our model includes an absolute refractory period after each spike. Second, we consider a distribution of axonal delay times. Finally, we describe synaptic signal transmission by excitatory and inhibitory potentials (EPSP and IPSP) with a realistic shape, that is, through a response kernel. During retrieval of a pattern, all active neurons exhibit periodic spike bursts which may or may not be synchronized (locked) into a coherent oscillation. We derive an analytical condition of locking and calculate the period of collective activity during oscillatory retrieval. In a stationary retrieval state, the overlap assumes a constant value proportional to the mean firing rate of the neurons. It is argued that in a biological network an intermediate scenario of weak locking is most likely.     

A neural network model of semantic memory linking feature-based object representation and words

Recent theories in cognitive neuroscience suggest that semantic memory is a distributed process, which involves many cortical areas and is based on a multimodal representation of objects. The aim of this work is to extend a previous model of object representation to realize a semantic memory, in which sensory-motor representations of objects are linked with words. The model assumes that each object is described as a collection of features, coded in different cortical areas via a topological organization. Features in different objects are segmented via γ-band synchronization of neural oscillators. The feature areas are further connected with a lexical area, devoted to the representation of words. Synapses among the feature areas, and among the lexical area and the feature areas are trained via a time-dependent Hebbian rule, during a period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from acoustic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits).