A neural net model for multiple memory domains

Previous studies with neural nets constructed of discrete populations of formal neurons have assumed that all neurons have the same probability of connection with any other neuron in the net. However, in this new study we incorporate the behavior of the neural systems in which the neural connections can be set up by means of chemical markers carried by the individual cells. With this new approach we studied the dynamics of isolated neural nets again as well as the dynamics of neural nets with sustained inputs. Results obtained with this approach show simple and multiple hysteresis phenomena. Such hysteresis loops may be considered to represent the basis for short-term memory.     

Cross-talk theory of memory capacity in neural networks

The present paper presents a theory for the mechanics of cross-talk among constituent neurons in networks in which multiple memory traces have been embedded, and develops criteria for memory capacity based on the disruptive influences of this cross-talk. The theory is based on interconnection patterns defined by the sequential configuration model of dynamic firing patterns. The theory accurately predicts the memory capacities observed in computer simulated nets, and predicts that cortical-like modules should be able to store up to about 300–900 selectively retrievable memory traces before disruption by cross-talk is likely. It also predicts that the cortex may has designed itself for modules of 30,000 neurons to at least in part to optimize memory capacity.     

Brain imaging and memory systems in humans: the contribution of PET methods

The development of neuroimaging methods such as PET, has provided a new impulse to the study of the neural basis of cognitive functions, and has extended the field of inquiry from the analysis of the consequences of brain lesions to the functional investigations of brain activity, either in patients with selective neuropsychological deficits or in normal subjects engaged in cognitive tasks. Specific patterns of hypometabolism in neurological patients are associated with different profiles of memory deficits. [¹⁸F]FDG PET studies have confirmed the association of episodic memory with the structures of Papez's circuit and have shown correlations between short-term and semantic memory and the language areas. The identification of anatomo-functional networks involved in specific components of memory function in normal subjects is the aim of several PET activation studies. The results are in agreement with ‘neural network’ models of the neural basis of memory, as complex functions subserved by multiple interconnected cortical and subcortical structures.     

Probabilistic computation by neuromine networks

In this paper, we address the question, can biologically feasible neural nets compute more than can be computed by deterministic polynomial time algorithms? Since we want to maintain a claim of plausibility and reasonableness we restrict ourselves to algorithmically easy to construct nets and we rule out infinite precision in parameters and in any analog parts of the computation. Our approach is to consider the recent advances in randomized algorithms and see if such randomized computations can be described by neural nets. We start with a pair of neurons and show that by connecting them with reciprocal inhibition and some tonic input, then the steady-state will be one neuron ON and one neuron OFF, but which neuron will be ON and which neuron will be OFF will be chosen at random (perhaps, it would be better to say that microscopic noise in the analog computation will be turned into a megascale random bit). We then show that we can build a small network that uses this random bit process to generate repeatedly random bits. This random bit generator can then be connected with a neural net representing the deterministic part of randomized algorithm. We, therefore, demonstrate that these neural nets can carry out probabilistic computation and thus be less limited than classical neural nets.     

A canonical form for neural nets without circles

By “neural net” will be meant “neural net without circles.” Every neural net effects a transformation from inputs (i.e., firing patterns of the input neurons) to outputs (firing patterns of the output neurons). Two neural nets will be calledequivalent if they effect the same transformation from inputs to outputs. A canonical form is found for neural nets with respect to equivalence; i.e., a class of neural nets is defined, no two of which are equivalent, and which contains a neural net equivalent to any given neural net. This research was supported by the U.S. Air Force under Contract AF 49(638)-414 monitored by the Air Force Office of Scientific Research.     

On problem solving with Hopfield neural networks

Hopfield and Tank have shown that neural networks can be used to solve certain computationally hard problems, in particular they studied the Traveling Salesman Problem (TSP). Based on network simulation results they conclude that analog VLSI neural nets can be promising in solving these problems. Recently, Wilson and Pawley presented the results of their simulations which contradict the original results and cast doubts on the usefulness of neural nets. In this paper we give the results of our simulations that clarify some of the discrepancies. We also investigate the scaling of TSP solutions found by neural nets as the size of the problem increases. Further, we consider the neural net solution of the Clustering Problem, also a computationally hard problem, and discuss the types of problems that appear to be well suited for a neural net approach.     

An associative memory that can form hypotheses: a phase-coded neural network

 Nonlinear associative memories as realized, e.g., by Hopfield nets are characterized by attractor-type dynamics. When fed with a starting pattern, they converge to exactly one of the stored patterns which is supposed to be most similar. These systems cannot render hypotheses of classification, i.e., render several possible answers to a given classification problem. Inspired by von der Malsburg’s correlation theory of brain function, we extend conventional neural network architectures by introducing additional dynamical variables. Assuming an oscillatory time structure of neural firing, i.e., the existence of neural clocks, we assign a so-called phase to each formal neuron. The phases explicitly describe detailed correlations of neural activities neglected in conventional neural network architectures. Implementing this extension into a simple self-organizing network based on a feature map, we present an associative memory that actually is capable of forming hypotheses of classification. Received: 6 December 1993/Accepted in revised form: 14 July 1994     

Individual differences in emotion processing

Recent functional brain imaging studies of the neurobiology of emotion have investigated how individual differences among subjects modulate neural responses during emotion processing. Differences in personality, dispositional affect, biological sex, and genotype can all substantially modulate the neural bases of emotion processing in prefrontal, limbic, and other brain regions, across a variety of domains including emotional reactions, emotional memory, and emotion perception. Analysis of individual differences provides a new window into the neurobiology of emotion processing that complements traditional approaches.     

A Potential Spatial Working Memory Training Task to Improve Both Episodic Memory and Fluid Intelligence

One current challenge in cognitive training is to create a training regime that benefits multiple cognitive domains, including episodic memory, without relying on a large battery of tasks, which can be time-consuming and difficult to learn. By giving careful consideration to the neural correlates underlying episodic and working memory, we devised a computerized working memory training task in which neurologically healthy participants were required to monitor and detect repetitions in two streams of spatial information (spatial location and scene identity) presented simultaneously (i.e. a dual n-back paradigm). Participants’ episodic memory abilities were assessed before and after training using two object and scene recognition memory tasks incorporating memory confidence judgments. Furthermore, to determine the generalizability of the effects of training, we also assessed fluid intelligence using a matrix reasoning task. By examining the difference between pre- and post-training performance (i.e. gain scores), we found that the trainers, compared to non-trainers, exhibited a significant improvement in fluid intelligence after 20 days. Interestingly, pre-training fluid intelligence performance, but not training task improvement, was a significant predictor of post-training fluid intelligence improvement, with lower pre-training fluid intelligence associated with greater post-training gain. Crucially, trainers who improved the most on the training task also showed an improvement in recognition memory as captured by d-prime scores and estimates of recollection and familiarity memory. Training task improvement was a significant predictor of gains in recognition and familiarity memory performance, with greater training improvement leading to more marked gains. In contrast, lower pre-training recollection memory scores, and not training task improvement, led to greater recollection memory performance after training. Our findings demonstrate that practice on a single working memory task can potentially improve aspects of both episodic memory and fluid intelligence, and that an extensive training regime with multiple tasks may not be necessary.     

Representational switching by dynamical reorganization of attractor structure in a network model of the prefrontal cortex

The prefrontal cortex (PFC) plays a crucial role in flexible cognitive behavior by representing task relevant information with its working memory. The working memory with sustained neural activity is described as a neural dynamical system composed of multiple attractors, each attractor of which corresponds to an active state of a cell assembly, representing a fragment of information. Recent studies have revealed that the PFC not only represents multiple sets of information but also switches multiple representations and transforms a set of information to another set depending on a given task context. This representational switching between different sets of information is possibly generated endogenously by flexible network dynamics but details of underlying mechanisms are unclear. Here we propose a dynamically reorganizable attractor network model based on certain internal changes in synaptic connectivity, or short-term plasticity. We construct a network model based on a spiking neuron model with dynamical synapses, which can qualitatively reproduce experimentally demonstrated representational switching in the PFC when a monkey was performing a goal-oriented action-planning task. The model holds multiple sets of information that are required for action planning before and after representational switching by reconfiguration of functional cell assemblies. Furthermore, we analyzed population dynamics of this model with a mean field model and show that the changes in cell assemblies' configuration correspond to those in attractor structure that can be viewed as a bifurcation process of the dynamical system. This dynamical reorganization of a neural network could be a key to uncovering the mechanism of flexible information processing in the PFC.     

Recognition of distantly related protein sequences using conserved motifs and neural networks.

A sensitive technique for protein sequence motif recognition based on neural networks has been developed. It involves three major steps. (1) At each appropriate alignment position of a set of N matched sequences, a set of N aligned oligopeptides is specified with preselected window length. N neural nets are subsequently and successively trained on N-1 amino acid spans after eliminating each ith oligopeptide. A test for recognition of each of the ith spans is performed. The average neural net recognition over N such trials is used as a measure of conservation for the particular windowed region of the multiple alignment. This process is repeated for all possible spans of given length in the multiple alignment. (2) The M most conserved regions are regarded as motifs and the oligopeptides within each are used to train intensively M individual neural networks. (3) The M networks are then applied in a search for related primary structures in a databank of known protein sequences. The oligopeptide spans in the database sequence with strongest neural net output for each of the M networks are saved and then scored according to the output signals and the proper combination that follows the expected N- to C-terminal sequence order. The motifs from the database with highest similarity scores can then be used to retrain the M neural nets, which can be subsequently utilized for further searches in the databank, thus providing even greater sensitivity to recognize distant familial proteins. This technique was successfully applied to the integrase, DNA-polymerase and immunoglobulin families.     

Preservation of musical memory in an amnesic professional cellist

Learning and memory of music involves a multitude of perceptual, motor, affective, and autobiographical memory processes [1]. Patient and imaging studies suggest that musical memory may involve distinct neural substrates [2,3]. However, the degree of independence of such a system from other memory domains is controversial [4]. We have investigated a 68-year-old professional cellist, patient PM, who developed severe amnesia following encephalitis. This case provided a unique opportunity to study musical memory in a patient with a precisely defined premorbid musical knowledge and well-demarcated focal lesions of the brain. Despite severe memory impairments, he performed like healthy musicians in various tests of recognition memory for music. These findings suggest that learning and retention of musical information depends on brain networks distinct from those involved in other types of episodic and semantic memory.     

Attentional selection in visual perception,memory and action: a quest for cross-domain integration

For decades, the cognitive and neural sciences have benefitted greatly from a separation of mind and brain into distinct functional domains. The tremendous success of this approach notwithstanding, it is self-evident that such a view is incomplete. Goal-directed behaviour of an organism requires the joint functioning of perception, memory and sensorimotor control. A prime candidate for achieving integration across these functional domains are attentional processes. Consequently, this Theme Issue brings together studies of attentional selection from many fields, both experimental and theoretical, that are united in their quest to find overreaching integrative principles of attention between perception, memory and action. In all domains, attention is understood as combination of competition and priority control (‘bias’), with the task as a decisive driving factor to ensure coherent goal-directed behaviour and cognition. Using vision as the predominant model system for attentional selection, many studies of this Theme Issue focus special emphasis on eye movements as a selection process that is both a fundamental action and serves a key function in perception. The Theme Issue spans a wide range of methods, from measuring human behaviour in the real word to recordings of single neurons in the non-human primate brain. We firmly believe that combining such a breadth in approaches is necessary not only for attentional selection, but also to take the next decisive step in all of the cognitive and neural sciences: to understand cognition and behaviour beyond isolated domains.     

A neural net model for epilepsy

A neural net model based in our previous studies with randomly interconnected neural nets is presented here capable of exhibiting epileptic features. These features can be explained in terms of the structural and dynamical properties of the model. In addition, apart from the fact that this model can imitate epileptic phenomena, it might also help to explain some poorly understood clinical phenomena from which general disturbances can produce focal seizures in the brain.     

The neurobiology of the human memory

Memory can be defined as the ability to acquire, process, store, and retrieve information. Memory is indispensable for learning, adaptation, and survival of every living organism. In humans, the remembering process has acquired great flexibility and complexity, reaching close links with other mental functions, such as thinking and emotions. Changes in synaptic connectivity and interactions among multiple neural networks provide the neurobiological substrates for memory encoding, retention, and consolidation. Memory may be categorized as short-term and long-term memory (according to the storage temporal duration), as implicit and explicit memory (with respect to the consciousness of remembering), as declarative (knowing that [fact]) and procedural (knowing how [skill]) memory, or as sensory (echoic, iconic and haptil), semantic, and episodic memory (according to the various remembering domains). Significant advances have been obtained in understanding memory neurobiology, but much remains to be learned in its cognitive, psychological, and phenomenological aspects.     

Prefrontal cortex and neural mechanisms of executive function

Executive function is a product of the coordinated operation of multiple neural systems and an essential prerequisite for a variety of cognitive functions. The prefrontal cortex is known to be a key structure for the performance of executive functions. To accomplish the coordinated operations of multiple neural systems, the prefrontal cortex must monitor the activities in other cortical and subcortical structures and control and supervise their operations by sending command signals, which is called top-down signaling. Although neurophysiological and neuroimaging studies have provided evidence that the prefrontal cortex sends top-down signals to the posterior cortices to control information processing, the neural correlate of these top-down signals is not yet known. Through use of the paired association task, it has been demonstrated that top-down signals are used to retrieve specific information stored in long-term memory. Therefore, we used a paired association task to examine the neural correlates of top-down signals in the prefrontal cortex. The preliminary results indicate that 32% of visual neurons exhibit pair-selectivity, which is similar to the characteristics of pair-coding activities in temporal neurons. The latency of visual responses in prefrontal neurons was longer than bottom-up signals but faster than top-down signals in inferior temporal neurons. These results suggest that pair-selective visual responses may be top-down signals that the prefrontal cortex provides to the temporal cortex, although further studies are needed to elucidate the neural correlates of top-down signals and their characteristics to understand the neural mechanism of executive control by the prefrontal cortex.     

When encoding yields remembering: insights from event-related neuroimaging.

To understand human memory, it is important to determine why some experiences are remembered whereas others are forgotten. Until recently, insights into the neural bases of human memory encoding, the processes by which information is transformed into an enduring memory trace, have primarily been derived from neuropsychological studies of humans with select brain lesions. The advent of functional neuroimaging methods, such as positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), has provided a new opportunity to gain additional understanding of how the brain supports memory formation. Importantly, the recent development of event-related fMRI methods now allows for examination of trial-by-trial differences in neural activity during encoding and of the consequences of these differences for later remembering. In this review, we consider the contributions of PET and fMRI studies to the understanding of memory encoding, placing a particular emphasis on recent event-related fMRI studies of the Dm effect: that is, differences in neural activity during encoding that are related to differences in subsequent memory. We then turn our attention to the rich literature on the Dm effect that has emerged from studies using event-related potentials (ERPs). It is hoped that the integration of findings from ERP studies, which offer higher temporal resolution, with those from event-related fMRI studies, which offer higher spatial resolution, will shed new light on when and why encoding yields subsequent remembering.     

Distribution of neural memory, loading factor, its regulation and optimization]

Recording and retrieving functions of the neural memory are simulated as a control of local conformational processes in neural synaptic fields. The localization of conformational changes is related to the afferent temporal-spatial pulse pattern flow, the microstructure of connections and a plurality of temporal delays in synaptic fields and afferent pathways. The loci of conformations are described by sets of afferent addresses named address domains. Being superimposed on each other, address domains form a multilayer covering of the address space of the neuron or the ensemble. The superposition factor determines the dissemination of the conformational process, and the fuzzing of memory, and its accuracy and reliability. The engram is formed as detects in the packing of the address space and hence can be retrieved in inverse form. The accuracy of the retrieved information depends on the threshold level of conformational transitions, the distribution of conformational changes in synaptic fields of the neuronal population, and the memory loading factor. The latter is represented in the model by a slow potential. It reflects total conformational changes and displaces the membrane potential to monostable conformational regimes, by governing the exit from the recording regime, the potentiation of the neurone, and the readiness to reproduction. A relative amplitude of the slow potential and the coefficient of postconformational modification of ionic conductivity, which provides maximum reliability, accuracy, and capacity of memory, are calculated.