On optimal associative recall by an incomplete key

A new association scheme is proposed. The fundamental principle of the conventional associative memory models is to solve the matrix equation which is made by the complete (memorized) keys and responses. Therefore, when an incomplete key pattern which is a fraction of memorized key is given to the models as a key, these models can not have an optimal association except for a special case. Analyzing the property of the incomplete key pattern, in this paper, we propose a new association model which behaves optimally for an incomplete key pattern. From the result of the computer simulation, we understand that this model has the expected ability.     

An algebraic model of an associative noise-like coding memory

A mathematical model of an associative memory is presented, sharing with the optical holography memory systems the properties which establish an analogy with biological memory. This memory system-developed from Gabor's model of memoryis based on a noise-like coding of the information by which it realizes a distributed, damage-tolerant, equipotential storage through simultaneous state changes of discrete substratum elements. Each two associated items being stored are coded by each other by means of two noise-like patterns obtained from them through a randomizing preprocessing. The algebraic braic transformations operating the information storage and retrieval are matrix-vector products involving Toeplitz type matrices. Several noise-like coded memory traces are superimposed on a common substratum without crosstalk interference; moreover, extraneous noise added to these memory traces does not injure the stored information. The main performances shown by this memory model are: i) the selective, complete recovering of stored information from incomplete keys, both mixed with extraneous information and translated from the position learnt; ii) a dynamic recollection where the information just recovered acts as a new key for a sequential retrieval process; iii) context-dependent responses. The hypothesis that the information is stored in the nervous system through a noise-like coding is suggested. The model has been simulated on a digital computer using bidimensional images.     

An after-shannon measure of the storage capacity of an associative noise-like coding memory

The maximum amount of information that can be stored, on the average, in each storage element, according to an associative scheme, has been measured for the memory model proposed by the author (Bottini 1980). In this model, the (binary) items being stored are coded by noise-like keys and the memory traces formed in this way are superimposed, by algebraic addition, on the same many-level storage elements. It is shown that the problem of measuring the information retrieved from the memory in a single recall and the problem — concerning the data-communication field —of measuring the information transmitted over a noisy channel are formally similar. In particular, the Shannon noisy-channel coding theorem can find an application also in our case of an associative memory. Finally, it is evidenced that the so-called matrix model of an associative memory has the same storage capacity as the model studied here.     

Transitory memory retrieval in a biologically plausible neural network model

A number of memory models have been proposed. These all have the basic structure that excitatory neurons are reciprocally connected by recurrent connections together with the connections with inhibitory neurons, which yields associative memory (i.e., pattern completion) and successive retrieval of memory. In most of the models, a simple mathematical model for a neuron in the form of a discrete map is adopted. It has not, however, been clarified whether behaviors like associative memory and successive retrieval of memory appear when a biologically plausible neuron model is used. In this paper, we propose a network model for associative memory and successive retrieval of memory based on Pinsky-Rinzel neurons. The state of pattern completion in associative memory can be observed with an appropriate balance of excitatory and inhibitory connection strengths. Increasing of the connection strength of inhibitory interneurons changes the state of memory retrieval from associative memory to successive retrieval of memory. We investigate this transition.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

A model of distributed type associative memory with quantized Hadamard transform

This paper proposes a new correlation matrix network model of associative memory in brain. Each memorized pattern which consists of binary (+1 or-1) elements is preprocessed by a quantized Hadamard transform to increase selectivity. The association ability of a correlation matrix network model depends on the orthogonality between key patterns by which the corresponding memorized patterns are associatively recalled. In a brain model, however, it is rare that the key patterns are mutually orthogonal since they are memorized patterns themselves. The quantized Hadamard transform, presented in this paper, renders the memorized patterns approximately orthogonal. The model is tested by computer simulation.     

Cluster associative memory formation in a three-layer network of phase oscillators

A three-layer network model of oscillatory associative memory is proposed. The network is capable of storing binary images, which can be retrieved upon presenting an appropriate stimulus. Binary images are encoded in the form of the spatial distribution of oscillatory phase clusters in-phase and anti-phase relative to a reference periodic signal. The information is loaded into the network using a set of interlayer connection weights. A condition for error-free pattern retrieval is formulated, delimiting the maximal number of patterns to be stored in the memory (storage capacity). It is shown that the capacity can be significantly increased by generating an optimal alphabet (basis pattern set). The number of stored patterns can reach values of the network size (the number of oscillators in each layer), which is significantly higher than the capacity of conventional oscillatory memory models. The dynamical and information characteristics of the retrieval process based on the optimal alphabet, including the size of “attraction basins“ and the input pattern distortion admissible for error-free retrieval, are investigated.     

Doubly robust estimates for binary longitudinal data analysis with missing response and missing covariates

Longitudinal studies often feature incomplete response and covariate data. Likelihood-based methods such as the expectation-maximization algorithm give consistent estimators for model parameters when data are missing at random (MAR) provided that the response model and the missing covariate model are correctly specified; however, we do not need to specify the missing data mechanism. An alternative method is the weighted estimating equation, which gives consistent estimators if the missing data and response models are correctly specified; however, we do not need to specify the distribution of the covariates that have missing values. In this article, we develop a doubly robust estimation method for longitudinal data with missing response and missing covariate when data are MAR. This method is appealing in that it can provide consistent estimators if either the missing data model or the missing covariate model is correctly specified. Simulation studies demonstrate that this method performs well in a variety of situations.     

Sequence learning recodes cortical representations instead of strengthening initial ones

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.     

Contextual taste cues modulate olfactory learning in C. elegans by an occasion-setting mechanism

Manipulations of context can affect learning and memory performance across species in many associative learning paradigms. Using taste cues to create distinct contexts for olfactory adaptation assays in the nematode Caenorhabditis elegans, we now show that performance in this associative learning paradigm is sensitive to context manipulations, and we investigate the mechanism(s) used for the integration of context cues in learning. One possibility is that the taste and olfactory stimuli are perceived as a combined, blended cue that the animals then associate with the unconditioned stimulus (US) in the same manner as with any other unitary conditioned stimuli (CS). Alternatively, an occasion-setting model suggests that the taste cues only define the appropriate context for olfactory memory retrieval without directly entering into the primary association. Analysis of genetic mutants demonstrated that the olfactory and context cues are sensed by distinct primary sensory neurons and that the animals' ability to use taste cues to modulate olfactory learning is independent from their ability to utilize these same taste cues for adaptation. We interpret these results as evidence for the occasion-setting mechanism in which context cues modulate primary Pavlovian association by functioning in a hierarchical manner to define the appropriate setting for memory recall.     

A latent-class mixture model for incomplete longitudinal Gaussian data

Summary .   In the analyses of incomplete longitudinal clinical trial data, there has been a shift, away from simple methods that are valid only if the data are missing completely at random, to more principled ignorable analyses, which are valid under the less restrictive missing at random assumption. The availability of the necessary standard statistical software nowadays allows for such analyses in practice. While the possibility of data missing not at random (MNAR) cannot be ruled out, it is argued that analyses valid under MNAR are not well suited for the primary analysis in clinical trials. Rather than either forgetting about or blindly shifting to an MNAR framework, the optimal place for MNAR analyses is within a sensitivity-analysis context. One such route for sensitivity analysis is to consider, next to selection models, pattern-mixture models or shared-parameter models. The latter can also be extended to a latent-class mixture model, the approach taken in this article. The performance of the so-obtained flexible model is assessed through simulations and the model is applied to data from a depression trial.     

Multiple imputation for multivariate data with missing and below-threshold measurements: time-series concentrations of pollutants in the Arctic

Many chemical and environmental data sets are complicated by the existence of fully missing values or censored values known to lie below detection thresholds. For example, week-long samples of airborne particulate matter were obtained at Alert, NWT, Canada, between 1980 and 1991, where some of the concentrations of 24 particulate constituents were coarsened in the sense of being either fully missing or below detection limits. To facilitate scientific analysis, it is appealing to create complete data by filling in missing values so that standard complete-data methods can be applied. We briefly review commonly used strategies for handling missing values and focus on the multiple-imputation approach, which generally leads to valid inferences when faced with missing data. Three statistical models are developed for multiply imputing the missing values of airborne particulate matter. We expect that these models are useful for creating multiple imputations in a variety of incomplete multivariate time series data sets.     

Multiple imputation methods for handling incomplete longitudinal and clustered data where the target analysis is a linear mixed effects model

Multiple imputation (MI) is increasingly popular for handling multivariate missing data. Two general approaches are available in standard computer packages: MI based on the posterior distribution of incomplete variables under a multivariate (joint) model, and fully conditional specification (FCS), which imputes missing values using univariate conditional distributions for each incomplete variable given all the others, cycling iteratively through the univariate imputation models. In the context of longitudinal or clustered data, it is not clear whether these approaches result in consistent estimates of regression coefficient and variance component parameters when the analysis model of interest is a linear mixed effects model (LMM) that includes both random intercepts and slopes with either covariates or both covariates and outcome contain missing information. In the current paper, we compared the performance of seven different MI methods for handling missing values in longitudinal and clustered data in the context of fitting LMMs with both random intercepts and slopes. We study the theoretical compatibility between specific imputation models fitted under each of these approaches and the LMM, and also conduct simulation studies in both the longitudinal and clustered data settings. Simulations were motivated by analyses of the association between body mass index (BMI) and quality of life (QoL) in the Longitudinal Study of Australian Children (LSAC). Our findings showed that the relative performance of MI methods vary according to whether the incomplete covariate has fixed or random effects and whether there is missingnesss in the outcome variable. We showed that compatible imputation and analysis models resulted in consistent estimation of both regression parameters and variance components via simulation. We illustrate our findings with the analysis of LSAC data.     

Internal representations for associative memory

We describe a class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data. In addition to the input and output layers, our networks incorporate an intermediate processing layer which serves to label each stored memory and to perform error correction and association. We study two classes of internal label representations: the unary representation and various sparse, distributed representations. Finally, we consider storage of sparse data and sparsification of data. These models are found to have advantages in terms of storage capacity, hardware efficiency, and recall reliability when compared to the Hopfield model, and to possess analogies to both biological neural networks and standard digital computer memories.     

主动联想记忆神经网络模型

在信息编码能提高联想记忆的存贮能力和脑内存在主动活动机制的启发下,提出一个主动联想记忆模型。模型包括两个神经网络,其一为输入和输出网络,另一个为在学习时期能自主产生兴奋模式的主动网络。两个网络的神经元之间有突触联系。由于自主产生的兴奋模式与输入无关,并可能接近于相互正交,因此,本模型有较高的存贮能力。初步分析和计算机仿真证明：本模型确有比通常联想记忆模型高的存贮能力,特别是在输入模式间有高度相关情况下、最后,对提出的模型与双向自联想记忆和光学全息存贮机制的关系作了讨论。     

Imputation methods for missing data for polygenic models

Methods to handle missing data have been an area of statistical research for many years. Little has been done within the context of pedigree analysis. In this paper we present two methods for imputing missing data for polygenic models using family data. The imputation schemes take into account familial relationships and use the observed familial information for the imputation. A traditional multiple imputation approach and multiple imputation or data augmentation approach within a Gibbs sampler for the handling of missing data for a polygenic model are presented.We used both the Genetic Analysis Workshop 13 simulated missing phenotype and the complete phenotype data sets as the means to illustrate the two methods. We looked at the phenotypic trait systolic blood pressure and the covariate gender at time point 11 (1970) for Cohort 1 and time point 1 (1971) for Cohort 2. Comparing the results for three replicates of complete and missing data incorporating multiple imputation, we find that multiple imputation via a Gibbs sampler produces more accurate results. Thus, we recommend the Gibbs sampler for imputation purposes because of the ease with which it can be extended to more complicated models, the consistency of the results, and the accountability of the variation due to imputation.     

Marginalized zero‐inflated Poisson models with missing covariates

Unlike zero‐inflated Poisson regression, marginalized zero‐inflated Poisson (MZIP) models for counts with excess zeros provide estimates with direct interpretations for the overall effects of covariates on the marginal mean. In the presence of missing covariates, MZIP and many other count data models are ordinarily fitted using complete case analysis methods due to lack of appropriate statistical methods and software. This article presents an estimation method for MZIP models with missing covariates. The method, which is applicable to other missing data problems, is illustrated and compared with complete case analysis by using simulations and dental data on the caries preventive effects of a school‐based fluoride mouthrinse program.     

Retrieving a mitogenomic mammal tree using composite taxa

Supermatrices are often characterized by a large amount of missing data. One possible approach to minimize such missing data is to create composite taxa. These taxa are formed by sampling sequences from different species in order to obtain a composite sequence that includes a maximum number of genes. Although this approach is increasingly used, its accuracy has rarely been tested and some authors prefer to analyze incomplete supermatrices by coding unavailable sequences as missing. To further validate the composite taxon approach, it was applied to complete mitochondrial matrices of 102 mammal species representing 93 families with varying amount of missing data. On average, missing data and composite matrices showed similar congruence to model trees obtained from the complete sequence matrix. As expected, the level of congruence to model trees decreased as missing data increased, with both approaches. We conclude that the composite taxon approach is worth considering in a phylogenomic context since it performs well and reduces computing time when compared to missing data matrices.     

Likelihood-based association analysis for nuclear families and unrelated subjects with missing genotype data

Missing data occur in genetic association studies for several reasons including missing family members and uncertain haplotype phase. Maximum likelihood is a commonly used approach to accommodate missing data, but it can be difficult to apply to family-based association studies, because of possible loss of robustness to confounding by population stratification. Here a novel likelihood for nuclear families is proposed, in which distinct sets of association parameters are used to model the parental genotypes and the offspring genotypes. This approach is robust to population structure when the data are complete, and has only minor loss of robustness when there are missing data. It also allows a novel conditioning step that gives valid analysis for multiple offspring in the presence of linkage. Unrelated subjects are included by regarding them as the children of two missing parents. Simulations and theory indicate similar operating characteristics to TRANSMIT, but with no bias with missing data in the presence of linkage. In comparison with FBAT and PCPH, the proposed model is slightly less robust to population structure but has greater power to detect strong effects. In comparison to APL and MITDT, the model is more robust to stratification and can accommodate sibships of any size. The methods are implemented for binary and continuous traits in software, UNPHASED, available from the author.