On associative memory

The information storing capacity of certain associative and auto-associative memories is calculated. For example, in a 100×100 matrix of 1 bit storage elements more than 6,500 bits can be stored associatively, and more than 688,000 bits in a 1,000×1,000 matrix. Asymptotically, the storage capacity of an associative memory increases proportionally to the number of storage elements. The usefulness of associative memories, as opposed to conventional listing memories, is discussed — especially in connection with brain modelling.     

An after-shannon measure of the storage capacity of an associative noise-like coding memory

The maximum amount of information that can be stored, on the average, in each storage element, according to an associative scheme, has been measured for the memory model proposed by the author (Bottini 1980). In this model, the (binary) items being stored are coded by noise-like keys and the memory traces formed in this way are superimposed, by algebraic addition, on the same many-level storage elements. It is shown that the problem of measuring the information retrieved from the memory in a single recall and the problem — concerning the data-communication field —of measuring the information transmitted over a noisy channel are formally similar. In particular, the Shannon noisy-channel coding theorem can find an application also in our case of an associative memory. Finally, it is evidenced that the so-called matrix model of an associative memory has the same storage capacity as the model studied here.     

Asymmetric neural coding revealed by in vivo calcium imaging in the honey bee brain

Left–right asymmetries are common properties of nervous systems. Although lateralized sensory processing has been well studied, information is lacking about how asymmetries are represented at the level of neural coding. Using in vivo functional imaging, we identified a population-level left–right asymmetry in the honey bee''s primary olfactory centre, the antennal lobe (AL). When both antennae were stimulated via a frontal odour source, the inter-odour distances between neural response patterns were higher in the right than in the left AL. Behavioural data correlated with the brain imaging results: bees with only their right antenna were better in discriminating a target odour in a cross-adaptation paradigm. We hypothesize that the differences in neural odour representations in the two brain sides serve to increase coding capacity by parallel processing.     

Sequence learning recodes cortical representations instead of strengthening initial ones

We contrast two computational models of sequence learning. The associative learner posits that learning proceeds by strengthening existing association weights. Alternatively, recoding posits that learning creates new and more efficient representations of the learned sequences. Importantly, both models propose that humans act as optimal learners but capture different statistics of the stimuli in their internal model. Furthermore, these models make dissociable predictions as to how learning changes the neural representation of sequences. We tested these predictions by using fMRI to extract neural activity patterns from the dorsal visual processing stream during a sequence recall task. We observed that only the recoding account can explain the similarity of neural activity patterns, suggesting that participants recode the learned sequences using chunks. We show that associative learning can theoretically store only very limited number of overlapping sequences, such as common in ecological working memory tasks, and hence an efficient learner should recode initial sequence representations.     

Structural Synaptic Plasticity Has High Memory Capacity and Can Explain Graded Amnesia,Catastrophic Forgetting,and the Spacing Effect

Although already William James and, more explicitly, Donald Hebb''s theory of cell assemblies have suggested that activity-dependent rewiring of neuronal networks is the substrate of learning and memory, over the last six decades most theoretical work on memory has focused on plasticity of existing synapses in prewired networks. Research in the last decade has emphasized that structural modification of synaptic connectivity is common in the adult brain and tightly correlated with learning and memory. Here we present a parsimonious computational model for learning by structural plasticity. The basic modeling units are “potential synapses” defined as locations in the network where synapses can potentially grow to connect two neurons. This model generalizes well-known previous models for associative learning based on weight plasticity. Therefore, existing theory can be applied to analyze how many memories and how much information structural plasticity can store in a synapse. Surprisingly, we find that structural plasticity largely outperforms weight plasticity and can achieve a much higher storage capacity per synapse. The effect of structural plasticity on the structure of sparsely connected networks is quite intuitive: Structural plasticity increases the “effectual network connectivity”, that is, the network wiring that specifically supports storage and recall of the memories. Further, this model of structural plasticity produces gradients of effectual connectivity in the course of learning, thereby explaining various cognitive phenomena including graded amnesia, catastrophic forgetting, and the spacing effect.     

Convolution and correlation algebras

An algebraic characterization of convolution and correlation is outlined. The basic algebraic structures generated on a suitable vector space by the two operations are described. The convolution induces an associative Abelian algebra over the real field; the correlation induces a not-associative, not-commutative — but Lieadmissible algebra — with a left unity. The algebraic connection between the two algebras is found to coincide with the relation of isotopy, an extension of the concept of equivalence. The interest of these algebraic structures with respect to information processing is discussed.     

Recurrence plots of neuronal spike trains

The recently developed qualitative method of diagnosis of dynamical systems — recurrence plots has been applied to the analysis of dynamics of neuronal spike trains recorded from cerebellum and red nucleus of anesthetized cats. Recurrence plots revealed robust and common changes in the similarity structure of interspike interval sequences as well as significant deviations from randomness in serial ordering of intervals. Recurring episodes of alike, quasi-deterministic firing patterns suggest the spontaneous modulation of the dynamical complexity of the trajectories of observed neurons. These modulations are associated with changing dynamical properties of a neuronal spike-train-generating system. Their existence is compatible with the information processing paradigm of attractor neural networks.     

Neurokernel: An Open Source Platform for Emulating the Fruit Fly Brain

We have developed an open software platform called Neurokernel for collaborative development of comprehensive models of the brain of the fruit fly Drosophila melanogaster and their execution and testing on multiple Graphics Processing Units (GPUs). Neurokernel provides a programming model that capitalizes upon the structural organization of the fly brain into a fixed number of functional modules to distinguish between these modules’ local information processing capabilities and the connectivity patterns that link them. By defining mandatory communication interfaces that specify how data is transmitted between models of each of these modules regardless of their internal design, Neurokernel explicitly enables multiple researchers to collaboratively model the fruit fly’s entire brain by integration of their independently developed models of its constituent processing units. We demonstrate the power of Neurokernel’s model integration by combining independently developed models of the retina and lamina neuropils in the fly’s visual system and by demonstrating their neuroinformation processing capability. We also illustrate Neurokernel’s ability to take advantage of direct GPU-to-GPU data transfers with benchmarks that demonstrate scaling of Neurokernel’s communication performance both over the number of interface ports exposed by an emulation’s constituent modules and the total number of modules comprised by an emulation.     

Lateralization and unilateral transfer of spatial memory in marsh tits

Summary The results reported in this paper demonstrate lateralization and transfer of spatial memory processing in an adult, food-storig bird. The technique of monocular occlusion was used to investigate lateralization and memory transfer in food-storing marsh tits (Parus palustris) for two tasks, food-storing and one-trial associative learning, which rely on one-trial learning for the spatial location of hidden food items. In the food-storing task, marsh tits had to return to the sites where they had previously stored a seed; in the one-trial associative learning task, the birds had to return to sites where they had been allowed to eat some, but not all, of a piece of peanut. For both spatial memory tasks, it was demonstrated that although the visual systems fed by both eyes are involved in short-term storage, the right eye system is associated with long-term storage, and that memories are transferred from the left to the right eye system between 3 and 24 h after memory formation.     

The orientation of muscle fibres in the tail musculature ofRana temporaria tadpoles (Amphibia,Anura)

Summary Shape of the myosepts and arrangement of the muscle fibres were recorded in the lateral musculature of the tail ofRana temporaria embryos and larvae. Well developed myomeres are present as early as st. 18–19. The main characteristics—ie. those related to functional properties—of myoseptal shape as well as of muscle fibre arrangement, remain unchanged throughout further development until degeneration of the tail occurs during metamorphosis. The rather simple myoseptal shape observed inRana—as compared to the multiple cone-form observed in most fishes—shows a close agreement to hypothetical myosept models described in papers by Jarman (1961), van der Stelt (1968) and Willemse (1966). The muscle fibres in the m. lateralis ofRana are arranged in trajectorial patterns that show a close similarity to the trajectorial patterns observed in typical teleosts. Both arrangements agree with trajectorial models based on the mathematical analyses of Alexander (1968).Neurulas anaesthetized with 1:10000 MS-222 and exposed up two weeks to this anaesthetic developed the same shape of the myosepts and arrangement of muscle fibres as in controls. Thus even the details of the function-related features of the myomere structure develop without functioning. In this field possible feedback meachisms are either not affected by anaesthesia or do not exist at all.     

Structural Transformations of the Associative Cortex As the Morphological Base of the Development of Human Cognitive Functions from Birth to 20 Years of Age

The microstructure of the temporo-parieto-occipital subregion and the frontal area of the brain from birth to 20 years of age was studied using computer morphometry. These brain zones are involved in the higher integrative mechanisms of cognitive functioning in children, adolescents and young adults. Structural transformations of the cortex represent a stage-by-stage process. Each stage in the frontal and occipital associative zones has specific temporal limits and is characterized by the quantitative and qualitative specificity of the morphological changes at each of the system levels considered: neuronal, modular, and stratification. The structural modifications from birth to early adulthood are primarily associated with the final development of micro and macroassembles and their structural components, primarily, neurons of various types. The growth and differentiation of neurons involves heterochrony with respect to the terms and developmental rates in the frontal and occipital associative cortex. The terms of the most active synchronous postnatal structural modifications, occurring during the first year of life, during the years 2–3, 6–7, 9–10, and 13–14 were analyzed. It was shown, that local specialization of cellular ensembles at various levels is a consequence of the functional specialization of microensembles, involved in cortical information processing, including cognitive activity and other higher psychophysiological functions of the human brain.     

Associative brain systems as the behavioral control apparatus based on the dominant and conditioned reflex

The main statements of author's conception concerning the associative brain systems (thalamoparietal and thalamofrontal) as the behaviour control systems are presented. The participation of associative systems in performance of the high brain functions ensures due to the entrance of the whole information spectrum of biological and signal significance into them and to the presence of neuronal plastic mechanisms, the mechanisms for retrieval the whole behaviour programs from the long-term memory and the ability of short-term storing of behaviour programs and estimation of their adequacy on the ground of dominant and conditioning mechanisms.     

Pathways to elaboration of sexual dimorphism in bird plumage patterns

Patterns, such as bars and spots, are common in birds. Some patterns can function in camouflage and/or communication and can benefit both males and females, paving the way for elaboration in sexual dimorphism. Historically, sexual dichromatism was predominantly considered to be a consequence of mating systems. However, the distribution of traits between the sexes is not always indicative of function; genetic correlation may cause traits to evolve in both sexes and traits may serve a social function in males and/or females. In addition, sexual dichromatism in bird plumage patterns can be composed of multiple types of patterns within and/or between the sexes. Therefore, there can be more than one type of dimorphism and some are more elaborate than others. Under classical models of genetic correlation, patterns evolve in both sexes followed by a loss of patterning in one sex. Elaborate types of sexual dimorphism in plumage patterns may be due to selection acting on existing patterns and are perhaps derived. Waterfowl (Anseriformes) and gamebirds (Galliformes) arguably have the most striking plumage patterns. Using 288 species from these orders I reconstructed the evolutionary history of plumage pattern dimorphism. There was little support for genetic correlation but elaborate types of dimorphism are probably derived. Backward and forward evolutionary transitions between different types of dimorphism can occur by loss or elaboration. These results demonstrate that plumage patterns are evolutionary labile and current forms may represent shifting adaptations to a changing environment. © 2013 The Linnean Society of London, Biological Journal of the Linnean Society, 2014, 111 , 262–273.     

主动联想记忆神经网络模型

在信息编码能提高联想记忆的存贮能力和脑内存在主动活动机制的启发下,提出一个主动联想记忆模型。模型包括两个神经网络,其一为输入和输出网络,另一个为在学习时期能自主产生兴奋模式的主动网络。两个网络的神经元之间有突触联系。由于自主产生的兴奋模式与输入无关,并可能接近于相互正交,因此,本模型有较高的存贮能力。初步分析和计算机仿真证明：本模型确有比通常联想记忆模型高的存贮能力,特别是在输入模式间有高度相关情况下、最后,对提出的模型与双向自联想记忆和光学全息存贮机制的关系作了讨论。     

Analysis of interference in quasi-holographic neuronal models of memory]

Three-layer models with neural plasticity are considered. The weights of connections have both positive and negative values, which are chosen randomly with equal probability. The connections are symmetric relative to the layer of associative neurons. The models describe processes analogous to holographic ones, and the basic properties of the models correspond to the properties of brain memory. The random choice of weights providing the noncorrelatedness of separate recordings gives rise to disturbances. It was shown that in passing from a full-connected model to non-full-connected and semi-full-connected models, the disturbances increase and limitations arise. The degree of the increase in disturbances depends upon the type of the model, the methods of recording and methods of the use of the models, and the mechanisms of the origination of disturbances.     

Optimal mechanisms for finding and selecting mates: how threespine stickleback (<Emphasis Type="Italic">Gasterosteus aculeatus</Emphasis>) should encode male throat colors

Male threespine stickleback (Gasterosteus aculeatus) use nuptial colors to attract mates and intimidate rivals. We quantified stickleback color and environmental lighting using methods independent of human perception to evaluate the information transmitted by male signals in a habitat where these signals are displayed. We also developed models of chromatic processing based on four cone photopigments (peak absorptions at 360, 445, 530, and 605 nm) characterized microspectrophotometrically in G. aculeatus and three other stickleback species. We show that a simple opponent mechanism receiving equally weighted inputs from cones with peak absorptions at 445 nm and 605 nm efficiently encodes variation in male throat colors. An orthogonal opponent mechanism—the difference between outputs of 530-nm cones and mean of outputs of 445- and 605-nm cones—produces a neural signal that could be used for species recognition and would be largely insensitive to variation in male throat color. We also show that threespine stickleback throats/photopigments are optimized for this coding scheme. These and other findings lead to testable hypotheses about the spectral processing mechanisms present in the threespine stickleback visual systems and the evolutionary interactions that have shaped this signal/receiver system.Abbreviations LWS long-wave sensitive - MWS middle-wave sensitive - SWS short-wave sensitive - UVS ultra-violet sensitive     

Functional Brain Networks Develop from a “Local to Distributed” Organization

The mature human brain is organized into a collection of specialized functional networks that flexibly interact to support various cognitive functions. Studies of development often attempt to identify the organizing principles that guide the maturation of these functional networks. In this report, we combine resting state functional connectivity MRI (rs-fcMRI), graph analysis, community detection, and spring-embedding visualization techniques to analyze four separate networks defined in earlier studies. As we have previously reported, we find, across development, a trend toward ‘segregation’ (a general decrease in correlation strength) between regions close in anatomical space and ‘integration’ (an increased correlation strength) between selected regions distant in space. The generalization of these earlier trends across multiple networks suggests that this is a general developmental principle for changes in functional connectivity that would extend to large-scale graph theoretic analyses of large-scale brain networks. Communities in children are predominantly arranged by anatomical proximity, while communities in adults predominantly reflect functional relationships, as defined from adult fMRI studies. In sum, over development, the organization of multiple functional networks shifts from a local anatomical emphasis in children to a more “distributed” architecture in young adults. We argue that this “local to distributed” developmental characterization has important implications for understanding the development of neural systems underlying cognition. Further, graph metrics (e.g., clustering coefficients and average path lengths) are similar in child and adult graphs, with both showing “small-world”-like properties, while community detection by modularity optimization reveals stable communities within the graphs that are clearly different between young children and young adults. These observations suggest that early school age children and adults both have relatively efficient systems that may solve similar information processing problems in divergent ways.     

Functional Brain Networks Develop from a “Local to Distributed” Organization

The mature human brain is organized into a collection of specialized functional networks that flexibly interact to support various cognitive functions. Studies of development often attempt to identify the organizing principles that guide the maturation of these functional networks. In this report, we combine resting state functional connectivity MRI (rs-fcMRI), graph analysis, community detection, and spring-embedding visualization techniques to analyze four separate networks defined in earlier studies. As we have previously reported, we find, across development, a trend toward ‘segregation’ (a general decrease in correlation strength) between regions close in anatomical space and ‘integration’ (an increased correlation strength) between selected regions distant in space. The generalization of these earlier trends across multiple networks suggests that this is a general developmental principle for changes in functional connectivity that would extend to large-scale graph theoretic analyses of large-scale brain networks. Communities in children are predominantly arranged by anatomical proximity, while communities in adults predominantly reflect functional relationships, as defined from adult fMRI studies. In sum, over development, the organization of multiple functional networks shifts from a local anatomical emphasis in children to a more “distributed” architecture in young adults. We argue that this “local to distributed” developmental characterization has important implications for understanding the development of neural systems underlying cognition. Further, graph metrics (e.g., clustering coefficients and average path lengths) are similar in child and adult graphs, with both showing “small-world”-like properties, while community detection by modularity optimization reveals stable communities within the graphs that are clearly different between young children and young adults. These observations suggest that early school age children and adults both have relatively efficient systems that may solve similar information processing problems in divergent ways.     

Associative Processing Is Inherent in Scene Perception

How are complex visual entities such as scenes represented in the human brain? More concretely, along what visual and semantic dimensions are scenes encoded in memory? One hypothesis is that global spatial properties provide a basis for categorizing the neural response patterns arising from scenes. In contrast, non-spatial properties, such as single objects, also account for variance in neural responses. The list of critical scene dimensions has continued to grow—sometimes in a contradictory manner—coming to encompass properties such as geometric layout, big/small, crowded/sparse, and three-dimensionality. We demonstrate that these dimensions may be better understood within the more general framework of associative properties. That is, across both the perceptual and semantic domains, features of scene representations are related to one another through learned associations. Critically, the components of such associations are consistent with the dimensions that are typically invoked to account for scene understanding and its neural bases. Using fMRI, we show that non-scene stimuli displaying novel associations across identities or locations recruit putatively scene-selective regions of the human brain (the parahippocampal/lingual region, the retrosplenial complex, and the transverse occipital sulcus/occipital place area). Moreover, we find that the voxel-wise neural patterns arising from these associations are significantly correlated with the neural patterns arising from everyday scenes providing critical evidence whether the same encoding principals underlie both types of processing. These neuroimaging results provide evidence for the hypothesis that the neural representation of scenes is better understood within the broader theoretical framework of associative processing. In addition, the results demonstrate a division of labor that arises across scene-selective regions when processing associations and scenes providing better understanding of the functional roles of each region within the cortical network that mediates scene processing.