Elements for a general memory structure: properties of recurrent neural networks used to form situation models |
| |
Authors: | Valeri A Makarov Yongli Song Manuel G Velarde David Hübner Holk Cruse |
| |
Institution: | (1) Instituto Pluridisciplinar, Universidad Complutense, Paseo Juan XXIII, 1, 28040 Madrid, Spain;(2) Department de Matemática Aplicada, Facultad de Matemáticas, Universidad Complutense, Avda. Complutense s/n, 28040 Madrid, Spain;(3) Department of Mathematics, Tongji University, 200092 Shanghai, China;(4) Department of Biological Cybernetics, Faculty of Biology, University of Bielefeld, 33501 Bielefeld, Germany |
| |
Abstract: | We study how individual memory items are stored assuming that situations given in the environment can be represented in the
form of synaptic-like couplings in recurrent neural networks. Previous numerical investigations have shown that specific architectures
based on suppression or max units can successfully learn static or dynamic stimuli (situations). Here we provide a theoretical
basis concerning the learning process convergence and the network response to a novel stimulus. We show that, besides learning
“simple” static situations, a nD network can learn and replicate a sequence of up to n different vectors or frames. We find limits on the learning rate and show coupling matrices developing during training in
different cases including expansion of the network into the case of nonlinear interunit coupling. Furthermore, we show that
a specific coupling matrix provides low-pass-filter properties to the units, thus connecting networks constructed by static
summation units with continuous-time networks. We also show under which conditions such networks can be used to perform arithmetic
calculations by means of pattern completion. |
| |
Keywords: | Recurrent neural network Situation model Memory Learning |
本文献已被 PubMed SpringerLink 等数据库收录! |
|