Recollection ability of three-dimensional correlation matrix associative memory

This paper presents the approximate expression of the recollection probability of a three-dimensional correlation matrix autoassociative memory, in which each memorized pattern consists of binary (+1 or-1) elements, and discusses the recollection ability in comparison with that of the conventional two-dimensional correlation matrix associative memory. An associative memory using the correlation properties between memorized patterns desires the condition that the memorized patterns are mutually orthogonal or approximately orthogonal. The three-dimensional correlation matrix associative memory that inscribes the third order correlation of a memorized pattern heightens the recollection ability even though the above condition is not satisfied.     

A model of orthogonal auto-associative networks

Two types of auto-associative networks are well known, one based upon the correlation matrix and the other based upon the orthogonal projection matrix, both of which are calculated from the pattern vectors to be memorized. Although the latter type of networks have a desirable associative property compared to the former ones, they require, in conventional models, nonlocal calculation of the pattern vectors (i.e., pseudoinverse of a matrix) or some learning procedure based on the error correction paradigm. This paper proposes a new model of auto-associative networks in which the orthogonal projection is implemented in a special relation between the connections linking neuron-like elements. The connection weights can be determined by a Hebbian local learning, requiring no pseudoinverse calculation nor the error correction learning.     

On optimal associative recall by an incomplete key

A new association scheme is proposed. The fundamental principle of the conventional associative memory models is to solve the matrix equation which is made by the complete (memorized) keys and responses. Therefore, when an incomplete key pattern which is a fraction of memorized key is given to the models as a key, these models can not have an optimal association except for a special case. Analyzing the property of the incomplete key pattern, in this paper, we propose a new association model which behaves optimally for an incomplete key pattern. From the result of the computer simulation, we understand that this model has the expected ability.     

Stability and attractivity in associative memory networks

We focus on stable and attractive states in a network having two-state neuron-like elements. We calculate the connection matrix which guarantees the stability and the strongest attractivity of p memorized patterns. We present an analytical evaluation of the patterns' attractivity. These results are illustrated by some computer simulations.     

A Slantlet transform based intelligent system for magnetic resonance brain image classification

The present paper proposes the development of a new approach for automated diagnosis, based on classification of magnetic resonance (MR) human brain images. Wavelet transform based methods are a well-known tool for extracting frequency space information from non-stationary signals. In this paper, the proposed method employs an improved version of orthogonal discrete wavelet transform (DWT) for feature extraction, called Slantlet transform, which can especially be useful to provide improved time localization with simultaneous achievement of shorter supports for the filters. For each two-dimensional MR image, we have computed its intensity histogram and Slantlet transform has been applied on this histogram signal. Then a feature vector, for each image, is created by considering the magnitudes of Slantlet transform outputs corresponding to six spatial positions, chosen according to a specific logic. The features hence derived are used to train a neural network based binary classifier, which can automatically infer whether the image is that of a normal brain or a pathological brain, suffering from Alzheimer's disease. An excellent classification ratio of 100% could be achieved for a set of benchmark MR brain images, which was significantly better than the results reported in a very recent research work employing wavelet transform, neural networks and support vector machines.     

主动联想记忆神经网络模型

在信息编码能提高联想记忆的存贮能力和脑内存在主动活动机制的启发下,提出一个主动联想记忆模型。模型包括两个神经网络,其一为输入和输出网络,另一个为在学习时期能自主产生兴奋模式的主动网络。两个网络的神经元之间有突触联系。由于自主产生的兴奋模式与输入无关,并可能接近于相互正交,因此,本模型有较高的存贮能力。初步分析和计算机仿真证明：本模型确有比通常联想记忆模型高的存贮能力,特别是在输入模式间有高度相关情况下、最后,对提出的模型与双向自联想记忆和光学全息存贮机制的关系作了讨论。     

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns.     

Biophysical mechanism of the interaction between default mode network and working memory network

Default mode network (DMN) is a functional brain network with a unique neural activity pattern that shows high activity in resting states but low activity in task states. This unique pattern has been proved to relate with higher cognitions such as learning, memory and decision-making. But neural mechanisms of interactions between the default network and the task-related network are still poorly understood. In this paper, a theoretical model of coupling the DMN and working memory network (WMN) is proposed. The WMN and DMN both consist of excitatory and inhibitory neurons connected by AMPA, NMDA, GABA synapses, and are coupled with each other only by excitatory synapses. This model is implemented to demonstrate dynamical processes in a working memory task containing encoding, maintenance and retrieval phases. Simulated results have shown that: (1) AMPA channels could produce significant synchronous oscillations in population neurons, which is beneficial to change oscillation patterns in the WMN and DMN. (2) Different NMDA conductance between the networks could generate multiple neural activity modes in the whole network, which may be an important mechanism to switch states of the networks between three different phases of working memory. (3) The number of sequentially memorized stimuli was related to the energy consumption determined by the network''s internal parameters, and the DMN contributed to a more stable working memory process. (4) Finally, this model demonstrated that, in three phases of working memory, different memory phases corresponded to different functional connections between the DMN and WMN. Coupling strengths that measured these functional connections differed in terms of phase synchronization. Phase synchronization characteristics of the contained energy were consistent with the observations of negative and positive correlations between the WMN and DMN reported in referenced fMRI experiments. The results suggested that the coupled interaction between the WMN and DMN played important roles in working memory.Supplementary InformationThe online version contains supplementary material available at 10.1007/s11571-021-09674-1.     

Principles of the design of D-neuronal networks

A self-learning neuronal network called MACSIM (Mathematical Analyzer and Composer of Simple Melodies) for music pattern recognition is proposed. As universal network elements the D-neurons defined in the previous paper (Fedor and Majernik, 1977) are used. MACSIM is intended to model, at the level of elementar neuronal circuits, the processing of simple melodies which takes place in the human brain auditory system. Melody composing and music itself are considered in this study in order to verify the principles of design and functioning of the proposed basic D-neuronal networks (organs) of MACSIM, only. Two kinds of neuronal organs are incorporated in MACSIM, namely the so-called analyzers and the so-called neuro-effectors. The first ones carry out predicting, learning and decoding of the input sequences of tones. An analyzer consists of several neuronal trees that have the ability to specialize and respecialize so that every melody repeated several times can be recognized after a corresponding learning period. Such a self-learning process runs without structural changes within the mentioned networks. A melody which has been memorized in an analyzer can be memorized later in a neuro-effector, namely in a form suitable for its recall at the output of MACSIM. The neuronal organization of the neuro-effectors is similar, to a certain extent, to that of the cerebellar cortex. With the help of the staggered arrangement of D-neurons set up on the lines of the organization of Purkinje cells in the cerebellar cortex, a new reliable memory storage mechanism is proposed. This paper is not self-contained; the reader is assumed to be familiar with the above-mentioned paper.     

Learning shapes spontaneous activity itinerating over memorized states

Learning is a process that helps create neural dynamical systems so that an appropriate output pattern is generated for a given input. Often, such a memory is considered to be included in one of the attractors in neural dynamical systems, depending on the initial neural state specified by an input. Neither neural activities observed in the absence of inputs nor changes caused in the neural activity when an input is provided were studied extensively in the past. However, recent experimental studies have reported existence of structured spontaneous neural activity and its changes when an input is provided. With this background, we propose that memory recall occurs when the spontaneous neural activity changes to an appropriate output activity upon the application of an input, and this phenomenon is known as bifurcation in the dynamical systems theory. We introduce a reinforcement-learning-based layered neural network model with two synaptic time scales; in this network, I/O relations are successively memorized when the difference between the time scales is appropriate. After the learning process is complete, the neural dynamics are shaped so that it changes appropriately with each input. As the number of memorized patterns is increased, the generated spontaneous neural activity after learning shows itineration over the previously learned output patterns. This theoretical finding also shows remarkable agreement with recent experimental reports, where spontaneous neural activity in the visual cortex without stimuli itinerate over evoked patterns by previously applied signals. Our results suggest that itinerant spontaneous activity can be a natural outcome of successive learning of several patterns, and it facilitates bifurcation of the network when an input is provided.     

The Default Mode Network and the Working Memory Network Are Not Anti-Correlated during All Phases of a Working Memory Task

Introduction

The default mode network and the working memory network are known to be anti-correlated during sustained cognitive processing, in a load-dependent manner. We hypothesized that functional connectivity among nodes of the two networks could be dynamically modulated by task phases across time.

Methods

To address the dynamic links between default mode network and the working memory network, we used a delayed visuo-spatial working memory paradigm, which allowed us to separate three different phases of working memory (encoding, maintenance, and retrieval), and analyzed the functional connectivity during each phase within and between the default mode network and the working memory network networks.

Results

We found that the two networks are anti-correlated only during the maintenance phase of working memory, i.e. when attention is focused on a memorized stimulus in the absence of external input. Conversely, during the encoding and retrieval phases, when the external stimulation is present, the default mode network is positively coupled with the working memory network, suggesting the existence of a dynamically switching of functional connectivity between “task-positive” and “task-negative” brain networks.

Conclusions

Our results demonstrate that the well-established dichotomy of the human brain (anti-correlated networks during rest and balanced activation-deactivation during cognition) has a more nuanced organization than previously thought and engages in different patterns of correlation and anti-correlation during specific sub-phases of a cognitive task. This nuanced organization reinforces the hypothesis of a direct involvement of the default mode network in cognitive functions, as represented by a dynamic rather than static interaction with specific task-positive networks, such as the working memory network.     

Dynamic Causal Modeling and subspace identification methods

The main contribution of the paper is in formulating the problem of detection of brain regions structure within the framework of dynamic system theory. The motivation is to see if the mature domain of experimental identification of dynamic systems can provide a methodology alternative to Dynamic Causal Modeling (DCM) which is currently used as an exclusive tool to estimate the structure of interconnections among a given set of brain regions using the measured data from functional magnetic resonance imaging (fMRI). The key tool proposed for modeling the structure of brain interconnections in this paper is subspace identification methods which produce linear state-space model, thus neglecting the bilinear term from DCM. The procedure is illustrated using a simple two-region model with maximally simplified linearized hemodynamics. We assume that the underlying system can be modeled by a set of linear differential equations, and identify the parameters (in terms of state space matrices), without any a priori constraints. We then transform the hidden states so that the implicit state matrix has a form or structure that is consistent with the generation of (region-specific) hemodynamic signals by coupled neuronal states.     

The research of constructing dynamic cognition model based on brain network

Estimating the functional interactions and connections between brain regions to corresponding process in cognitive, behavioral and psychiatric domains is a central pursuit for understanding the human connectome. Few studies have examined the effects of dynamic evolution on cognitive processing and brain activation using brain network model in scalp electroencephalography (EEG) data. Aim of this study was to investigate the brain functional connectivity and construct dynamic programing model from EEG data and to evaluate a possible correlation between topological characteristics of the brain connectivity and cognitive evolution processing. Here, functional connectivity between brain regions is defined as the statistical dependence between EEG signals in different brain areas and is typically determined by calculating the relationship between regional time series using wavelet coherence. We present an accelerated dynamic programing algorithm to construct dynamic cognitive model that we found that spatially distributed regions coherence connection difference, the topologic characteristics with which they can transfer information, producing temporary network states. Our findings suggest that brain dynamics give rise to variations in complex network properties over time after variation audio stimulation, dynamic programing model gives the dynamic evolution processing at different time and frequency. In this paper, by applying a new construct approach to understand whole brain network dynamics, firstly, brain network is constructed by wavelet coherence, secondly, different time active brain regions are selected by network topological characteristics and minimum spanning tree. Finally, dynamic evolution model is constructed to understand cognitive process by dynamic programing algorithm, this model is applied to the auditory experiment, results showed that, quantitatively, more correlation was observed after variation audio stimulation, the EEG function connection dynamic evolution model on cognitive processing is feasible with wavelet coherence EEG recording.     

Learning and recall in a dynamic theory of coordination patterns

A dynamic theory of learning and recall of coordination patterns is developed in the context of relative timing skills. Characterizing the coordination patterns in such skills by the collective variable, relative phase, we choose a model system in which the intrinsic pattern dynamics as well as the influence of environmental and memorized information are well understood from previous experimental and theoretical work. To describe learning we endow memorized information with dynamics which is determined by a phenomenological strategy. Similarly, additional degrees of freedom must be introduced to understand recall. As such recall variables we choose the relative strengths with which each memorized pattern acts on the pattern dynamics and model their dynamics phenomenologically. The resulting dynamical system that resembles models used in pattern recognition theory is shown to adequately describe the learning and recall processes. Moreover, due to the operational character of the theory, several predictions emerge that are open to experimental test. In particular, we show under which conditions phase transitions occur in the dynamics of the coordination patterns during learning and during recall. Considering different time scales and their relations we demonstrate how these phase transitions can be identified and observed. Other predictions include the influence of the intrinsic pattern dynamics on the recall process and the existence of history and hysteresis effects in recall. We discuss different forms of forgetting and differentiation of memorized information. The results show how a new theoretical view of learning and recall as change of behavioral dynamics can lead to a different understanding of these processes by providing testable predictions.     

Influence of Choice of Null Network on Small-World Parameters of Structural Correlation Networks

In recent years, coordinated variations in brain morphology (e.g., volume, thickness) have been employed as a measure of structural association between brain regions to infer large-scale structural correlation networks. Recent evidence suggests that brain networks constructed in this manner are inherently more clustered than random networks of the same size and degree. Thus, null networks constructed by randomizing topology are not a good choice for benchmarking small-world parameters of these networks. In the present report, we investigated the influence of choice of null networks on small-world parameters of gray matter correlation networks in healthy individuals and survivors of acute lymphoblastic leukemia. Three types of null networks were studied: 1) networks constructed by topology randomization (TOP), 2) networks matched to the distributional properties of the observed covariance matrix (HQS), and 3) networks generated from correlation of randomized input data (COR). The results revealed that the choice of null network not only influences the estimated small-world parameters, it also influences the results of between-group differences in small-world parameters. In addition, at higher network densities, the choice of null network influences the direction of group differences in network measures. Our data suggest that the choice of null network is quite crucial for interpretation of group differences in small-world parameters of structural correlation networks. We argue that none of the available null models is perfect for estimation of small-world parameters for correlation networks and the relative strengths and weaknesses of the selected model should be carefully considered with respect to obtained network measures.     

湿地翅碱蓬生物量遥感估算模型

以黄河三角洲HJ-1A CCD遥感数据和滨海湿地翅碱蓬生物量实测数据为数据源,通过对比分析参数回归模型（单变量线性和非线性回归模型,多元线性逐步回归模型）和人工神经网络模型（BP网络、RBF网络、GRNN网络）,构建黄河三角洲湿地翅碱蓬生长初期的生物量湿重遥感估算最优模型。研究表明：基于遥感信息变量能够建立生长初期翅碱蓬生物量湿重估算模型。尽管基于RDVI、MSAVI和PC2的3个变量的多元线性回归模型的拟合效果较优,但是以SAVI、MSAVI、RVI、DVI、RDVI和PC2等7个遥感信息变量构建的BP神经网络模型的精度更高,平均相对误差为12.73%,估算效果最优,能够满足较高精度的生物量湿重估算需求。翅碱蓬生长初期生物量湿重最优估算模型的建立,为滨海地区植被生物量监测、区域翅碱蓬生物量季节动态模拟以及黄河三角洲生态系统功能评价提供技术支持与基础。     

Bottom�Cup attention: pulsed PCA transform and pulsed cosine transform

In this paper we propose a computational model of bottom–up visual attention based on a pulsed principal component analysis (PCA) transform, which simply exploits the signs of the PCA coefficients to generate spatial and motional saliency. We further extend the pulsed PCA transform to a pulsed cosine transform that is not only data-independent but also very fast in computation. The proposed model has the following biological plausibilities. First, the PCA projection vectors in the model can be obtained by using the Hebbian rule in neural networks. Second, the outputs of the pulsed PCA transform, which are inherently binary, simulate the neuronal pulses in the human brain. Third, like many Fourier transform-based approaches, our model also accomplishes the cortical center-surround suppression in frequency domain. Experimental results on psychophysical patterns and natural images show that the proposed model is more effective in saliency detection and predict human eye fixations better than the state-of-the-art attention models.     

基于频谱图像灰度共生矩阵的无损测温方法

本文提出了一种基于哈达玛变换的频谱图像灰度共生矩阵(Hadamard-GLCM)的高强度聚焦超声治疗无损测温方法。利用高强度聚焦超声辐照新鲜离体猪肉组织,获取辐照前后的B超图像的减影图像,采用Hadamard变换对其进行处理,获取频谱图像,将频谱图像的灰度共生矩阵惯性矩作为反应温度变化的信息参数。实验表明:不仅单组数据的Hadamard-GLCM惯性矩(HGMI)和温度能很好的线性拟合,而且多组数据的Hadamard-GLCM惯性矩与温度也成近似的线性关系,而且斜率非常接近,拟合度更接近1,误差小,对温度的分辨能力高,容错能力强,与传统的测温方法相比有着明显的优势,能为HIFU治疗过程中的无损测温提供有效的实时依据。     

Synchronization study in ring-like and grid-like neuronal networks

In this paper, we study the synchronization status of both two gap-junction coupled neurons and neuronal network with two different network connectivity patterns. One of the network connectivity patterns is a ring-like neuronal network, which only considers nearest-neighbor neurons. The other is a grid-like neuronal network, with all nearest neighbor couplings. We show that by varying some key parameters, such as the coupling strength and the external current injection, the neuronal network will exhibit various patterns of firing synchronization. Different types of firing synchronization are diagnosed by means of a mean field potential, a bifurcation diagram, a correlation coefficient and the ISI-distance method. Numerical simulations demonstrate that the synchronization status of multiple neurons is much dependent on the network patters, when the number of neurons is the same. It is also demonstrated that the synchronization status of two coupled neurons is similar with the grid-like neuronal network, but differs radically from that of the ring-like neuronal network. These results may be instructive in understanding synchronization transitions in neuronal systems.     

Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuron-like elements

A demonstration is given that an orthogonalizing filter for patterns is formed adaptively and very rapidly in a network of neuron-like elements with internal feedback connections. It is here assumed that the feedback gain is variable, and proportional to the correlation matrix of the output pattern vectors. The time-dependent signal transfer properties of the complete system are described by a system matrix which satisfies a matrix Bernoulli differential equation; solutions of this equation are outlined. The asymptotic value of the system matrix is shown to correspond to the orthogonal projection operator on the space that is complementary to the space spanned by all of the earlier input pattern vectors. Such a system then acts as a filter, which optimally extracts the amount that is new in an input pattern with respect to all old patterns. It also has features that are directly attributable to a distributed associative memory that is optimally selective.