短时记忆的神经网络模型

提出一个带有指针环路的短时记忆神经网络模型,模型包含两个神经网络,其中一个是与长时记忆共有的存贮内容表达网络,另一个为短时指针神经元环路,由于指针环路仅作为记忆内容的临时指针,因此,仅用很少的存贮单元即可完成各种短时记忆任务,计算机仿真证明,本模型确能表现出短时记忆的存贮容量有限和组块编码两个基本特征。     

条件联想神经网络研究

在人脑的某些功能和神经系统中的突前抑制机制启发下,本文提出一个新型的神经网络模型——条件联想神经网络.模型是一个有突触前抑制的联想记忆神经网络.通过初步分析和计算机模拟,证明本模型具有一般联想记忆模型所未有的一些新的特性,如可以在不同条件下,对同一输入有不同的反应.对同一输入,在不同的条件下,又可以有相同的反应.这些特点将有助于人们对神经系统中信息处理过程的了解.此外,文中也指出可能实现本模型的神经结构.     

具有时滞的双向联想记忆神经网络的全局渐近稳定性

双向联想记忆模型是两层异联想网络,本文讨论了具有轴突信号传输时滞的双向联想记忆神经网络的全局渐近稳定性,得出了保证神经网络平衡点稳定的几个充分条件,所得到的结论对于具有时滞的连续双向联想记忆神经网络的设计和应用都是很有意义的。     

近存储饱和状态下联想学习记忆的神经网络模型

本文提出了神经网络在近饱和状态下的一种联想学习记忆模型.讨论了该模型的主要特性,对由100个神经元、记忆10个随机图样组成的网络系统给出并分析了计算机模拟结果,讨论了该模型的学习律与传统的Hebb学习律的区别,研究了网络在学习记忆和联想新态时初始噪声Pi和联想噪声Pa对新态恢复行为的影响,总结了在近饱和状态下该模型所具有的优势.     

联想记忆法初探

巴甫洛夫曾经指出:联想不是别的,只不过是由于两个或几个刺激同时地或连续地发生作用而产生了暂时的神经联系。即人们往往由当前的事物而想起有关另外的事物。它可以使人们的思路开阔、富有延伸性、灵活性。使建立神经联系的脑细胞产生兴奋,并在大脑皮层留下清晰的印迹,因而记忆十分牢固。坚持联想记忆,有利于培养学生的创造能力发散性思维能力。     

Izhikevich神经元网络的同步与联想记忆

联想记忆是人脑的一项重要功能。以Izhikevich神经元模型为节点,构建神经网络,神经元之间采用全连结的方式;以神经元群体的时空编码（spatio-temporal coding）理论研究所构建神经网络的联想记忆功能。在加入高斯白噪声的情况下,调节网络中神经元之间的连接强度的大小,当连接强度和噪声强度达到一个阈值时网络中部分神经元同步放电,实现了存储模式的联想记忆与恢复。仿真结果表明,神经元之间的连接强度在联想记忆的过程中发挥了重要的作用,噪声可以促使神经元间的同步放电,有助于神经网络实现存储模式的联想记忆与恢复。     

抑郁症模型大鼠学习记忆能力变化研究

为探讨抑郁症发生发展过程中学习记忆能力的变化模式及其可能机制．分别采用21天慢性非预见性刺激法和嗅球切除法建立的抑郁症模型大鼠．运用旷场行为实验（open—field behavior）检测大鼠主动性活动能力,用Morris水迷宫法检测大鼠空间学习记忆能力,HPLC—UV法测定大鼠血清皮质醇含量。电生理法记录海马CA1区LTP与LTD,观察海马神经元的突触可塑性。结果显示：与对照组相比,两种模型的自主活动性、空间探索兴趣和学习能力都明显降低,而记忆的反馈功能没有明显的变化。同时．两种模型大鼠海马神经细胞的突触可塑性显著下降,血清皮质醇的含量则明显上升。提示两种建模方法均导致大鼠产生抑郁症状和学习能力障碍．但对记忆反馈功能无明显影响。     

选择方向强化学习的神经网络模型

提出了神经网络模型的一种选择方向强化学习规则,定义并导出了新模型与Hopfield模型两种不同的筛选曲线,由此表明新模型对相关图样的分辨力优于Hopfield模型。在微机上模拟了由100个神经元构成的网络,结果显示新模型具有重复记忆这一神经生理学特点。定义并分行了记忆强度因子,模拟结果表明记忆强度因子愈大的记忆态,联想性能愈好,学习周期愈短。     

锥体神经元的连接模式对突触后局部电位的依赖

脑皮层的功能连接模式与突触可塑性密切相关,受突触空间分布和刺激模式等多种因素的影响。尽管越来越多的证据表明突触可塑性不仅受突触后动作电位而且还受突触后局部树突电位的影响,但是目前尚不清楚神经元的功能连接模式是否和怎样依赖于突触后局部电位的。为此,本文建立了一个无需硬边界设置的、突触后局部膜电位依赖的可塑性模型。该模型具有突触强度的自平衡能力并且能够再现多种突触可塑性实验结果。基于该模型对两个锥体神经元的功能连接模式进行仿真的结果表明,当突触后局部电位都处于亚阈值时两个神经元无功能连接,如果一个神经元的突触后膜电位高于阈值电位则产生向该神经元的单向连接,当两个神经元的突触后膜电位都超过阈值电位时则产生双向连接,说明突触后局部膜电位分布是神经元功能连接模式形成的关键。研究结果加深了神经网络连接模式形成机制的理解,对学习和记忆的研究具有重要意义。     

海马记忆功能的神经网络模型

综合神经心理学,神经生理学、解剖学与神经网络研究的成果,提出一个海马记忆功能的神经网络模型。模型由三个神经网络所组成;海马的ＣＡ１和ＣＡ３网络和大脑皮层联合区,ＣＡ３的功能是将不同感觉输入联合起来,而ＣＡ１的作用是将它们结成一个单一的记忆。而大脑皮层则是长期记忆的部位。在ＶＡＸ１１／７５０上进行计算机仿真,仿真证明模型有近期及长期记忆功能,破坏模拟海马的部分,模型显示出与顺行性遗忘症相似的特性。在     

Modelling studies on the computational function of fast temporal structure in cortical circuit activity

The interplay between modelling and experimental studies can support the exploration of the function of neuronal circuits in the cortex. We exemplify such an approach with a study on the role of spike timing and gamma-oscillations in associative memory in strongly connected circuits of cortical neurones. It is demonstrated how associative memory studies on different levels of abstraction can specify the functionality to be expected in real cortical neuronal circuits. In our model overlapping random configurations of sparse cell populations correspond to memory items that are stored by simple Hebbian coincidence learning. This associative memory task will be implemented with biophysically well tested compartmental neurones developed by Pinsky and Rinzel . We ran simulation experiments to study memory recall in two network architectures: one interconnected pool of cells, and two reciprocally connected pools. When recalling a memory by stimulating a spatially overlapping set of cells, the completed pattern is coded by an event of synchronized single spikes occurring after 25-60 ms. These fast associations are performed even at a memory load corresponding to the memory capacity of optimally tuned formal associative networks (>0.1 bit/synapse). With tonic stimulation or feedback loops in the network the neurones fire periodically in the gamma-frequency range (20-80 Hz). With fast changing inputs memory recall can be switched between items within a single gamma cycle. Thus, oscillation is not a primary coding feature necessary for associative memory. However, it accompanies reverberatory feedback providing an improved iterative memory recall completed after a few gamma cycles (60-260 ms). In the bidirectional architecture reverberations do not express in a rigid phase locking between the pools. For small stimulation sets bursting occurred in these cells acting as a supportive mechanism for associative memory.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

Construction of an associative memory using unstable periodic orbits of a chaotic attractor

Unstable periodic orbits are the skeleton of a chaotic attractor. We constructed an associative memory based on the chaotic attractor of an artificial neural network, which associates input patterns to unstable periodic orbits. By processing an input, the system is driven out of the ground state to one of the pre-defined disjunctive areas of the attractor. Each of these areas is associated with a different unstable periodic orbit. We call an input pattern learned if the control mechanism keeps the system on the unstable periodic orbit during the response. Otherwise, the system relaxes back to the ground state on a chaotic trajectory. The major benefits of this memory device are its high capacity and low-energy consumption. In addition, new information can be simply added by linking a new input to a new unstable periodic orbit.     

Cluster associative memory formation in a three-layer network of phase oscillators

A three-layer network model of oscillatory associative memory is proposed. The network is capable of storing binary images, which can be retrieved upon presenting an appropriate stimulus. Binary images are encoded in the form of the spatial distribution of oscillatory phase clusters in-phase and anti-phase relative to a reference periodic signal. The information is loaded into the network using a set of interlayer connection weights. A condition for error-free pattern retrieval is formulated, delimiting the maximal number of patterns to be stored in the memory (storage capacity). It is shown that the capacity can be significantly increased by generating an optimal alphabet (basis pattern set). The number of stored patterns can reach values of the network size (the number of oscillators in each layer), which is significantly higher than the capacity of conventional oscillatory memory models. The dynamical and information characteristics of the retrieval process based on the optimal alphabet, including the size of “attraction basins“ and the input pattern distortion admissible for error-free retrieval, are investigated.     

Internal representations for associative memory

We describe a class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data. In addition to the input and output layers, our networks incorporate an intermediate processing layer which serves to label each stored memory and to perform error correction and association. We study two classes of internal label representations: the unary representation and various sparse, distributed representations. Finally, we consider storage of sparse data and sparsification of data. These models are found to have advantages in terms of storage capacity, hardware efficiency, and recall reliability when compared to the Hopfield model, and to possess analogies to both biological neural networks and standard digital computer memories.     

Dynamics control of semantic processes in a hierarchical associative memory

A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks.     

Context-dependent associations in linear distributed memories

In this article we present a method that allows conditioning of the response of a linear distributed memory to a variable context. This method requires a system of two neural networks. The first net constructs the Kronecker product between the vector input and the vector context, and the second net supports a linear associative memory. This system is easily adaptable for different goals. We analyse here its capacity for the conditional extraction of features from a complex perceptual input, its capacity to perform quasi-logical operations (for instance, of the kind of “exclusive-or”), and its capacity to structurate a memory for temporal sequences which access is conditioned by the context. Finally, we evaluate the potential importance of the capacity to establish arbitrary contexts, for the evolution of biological cognitive systems. Part of this study has been presented in a preliminary version at the XVI Reunión Científica de la Sociedad Argentina de Biofísica, Tigre, Argentina, December 1987.     

Distinguishing spurious and nominal attractors applying unlearning to an asymmetric neural network

We study a neural network with asymmetric connections used as an associative memory. Asymmetry allows the nominal patterns to be stored in cycles. We apply an unlearning procedure, which modifies the synaptic connections. We analyze the global performance, including the network capacity, the attraction basin's size and also the relaxation time distribution. The latter shows a convenient bimodality that is used for discriminating between spurious and stored memory attractors. We show that unlearning in asymmetric networks allows enhancing the global performance of retrieval including retrieval of a sequence of correlated patterns.     

Optimal learning rules for familiarity detection

It has been suggested that the mammalian memory system has both familiarity and recollection components. Recently, a high-capacity network to store familiarity has been proposed. Here we derive analytically the optimal learning rule for such a familiarity memory using a signal- to-noise ratio analysis. We find that in the limit of large networks the covariance rule, known to be the optimal local, linear learning rule for pattern association, is also the optimal learning rule for familiarity discrimination. In the limit of large networks, the capacity is independent of the sparseness of the patterns and the corresponding information capacity is 0.057 bits per synapse, which is somewhat less than typically found for associative networks.     

Dynamics control of semantic processes in a hierarchical associative memory

 A neural mechanism for control of dynamics and function of associative processes in a hierarchical memory system is demonstrated. For the representation and processing of abstract knowledge, the semantic declarative memory system of the human brain is considered. The dynamics control mechanism is based on the influence of neuronal adaptation on the complexity of neural network dynamics. Different dynamical modes correspond to different levels of the ultrametric structure of the hierarchical memory being invoked during an associative process. The mechanism is deterministic but may also underlie free associative thought processes. The formulation of an abstract neural network model of hierarchical associative memory utilizes a recent approach to incorporate neuronal adaptation. It includes a generalized neuronal activation function recently derived by a Hodgkin-Huxley-type model. It is shown that the extent to which a hierarchically organized memory structure is searched is controlled by the neuronal adaptability, i.e. the strength of coupling between neuronal activity and excitability. In the brain, the concentration of various neuromodulators in turn can regulate the adaptability. An autonomously controlled sequence of bifurcations, from an initial exploratory to a final retrieval phase, of an associative process is shown to result from an activity-dependent release of neuromodulators. The dynamics control mechanism may be important in the context of various disorders of the brain and may also extend the range of applications of artificial neural networks. Received: 19 April 1995/Accepted in revised form: 8 August 1995