Efficient Parallel Levenberg-Marquardt Model Fitting towards Real-Time Automated Parametric Imaging Microscopy

We present a fast, accurate and robust parallel Levenberg-Marquardt minimization optimizer, GPU-LMFit, which is implemented on graphics processing unit for high performance scalable parallel model fitting processing. GPU-LMFit can provide a dramatic speed-up in massive model fitting analyses to enable real-time automated pixel-wise parametric imaging microscopy. We demonstrate the performance of GPU-LMFit for the applications in superresolution localization microscopy and fluorescence lifetime imaging microscopy.     

On the behavior of some associative neural networks

Since Hopfield published his work on an associative memory model, a large number of works have studied the model from several angles and showed in particular its weaknesses, and presented ways to overcome them. Most of the proposed solutions seem to us however not biologically plausible. In this paper we present a simple statistical analysis of two networks similar to the Hopfield net, and show that the usage of positive feedback enhances the net recognizing capability without jeopardizing the stability. We also describe a layered parallel network composed of modules, each module being a modified Hopfield net. We finally present computer simulation results to support our analytical findings. The most important principles of this network are supported by data from the world of neurobiology.     

Geographical information system parallelization for spatial big data processing: a review

With the increasing interest in large-scale, high-resolution and real-time geographic information system (GIS) applications and spatial big data processing, traditional GIS is not efficient enough to handle the required loads due to limited computational capabilities.Various attempts have been made to adopt high performance computation techniques from different applications, such as designs of advanced architectures, strategies of data partition and direct parallelization method of spatial analysis algorithm, to address such challenges. This paper surveys the current state of parallel GIS with respect to parallel GIS architectures, parallel processing strategies, and relevant topics. We present the general evolution of the GIS architecture which includes main two parallel GIS architectures based on high performance computing cluster and Hadoop cluster. Then we summarize the current spatial data partition strategies, key methods to realize parallel GIS in the view of data decomposition and progress of the special parallel GIS algorithms. We use the parallel processing of GRASS as a case study. We also identify key problems and future potential research directions of parallel GIS.     

Associative database of protein sequences.

MOTIVATION: We present a new concept that combines data storage and data analysis in genome research, based on an associative network memory. As an illustration, 115 000 conserved regions from over 73 000 published sequences (i.e. from the entire annotated part of the SWISSPROT sequence database) were identified and clustered by a self-organizing network. Similarity and kinship, as well as degree of distance between the conserved protein segments, are visualized as neighborhood relationship on a two-dimensional topographical map. RESULTS: Such a display overcomes the restrictions of linear list processing and allows local and global sequence relationships to be studied visually. Families are memorized as prototype vectors of conserved regions. On a massive parallel machine, clustering and updating of the database take only a few seconds; a rapid analysis of incoming data such as protein sequences or ESTs is carried out on present-day workstations. AVAILABILITY: Access to the database is available at http://www.bioinf.mdc-berlin.de/unter2.html++ + CONTACT: (hanke,lehmann,reich)@mdc-berlin.de; bork@embl-heidelberg.de     

Configural and elemental processing in associative learning: commentary on Melchers, Shanks and Lachnit

The paper by Melchers, Shanks, and Lachnit (2007) reviews the available evidence suggesting that there is flexible processing, such that some associative learning tasks can be solved either configurally or elementally. We find the evidence provocative but limited in its demonstrated generality and silent with respect to the theoretical mechanisms that might regulate the alleged flexibility of processing. Further research is invited to determine the scope of the variation involved and how best to account for it. At present, theories, either elemental or configural, that include mechanisms to shift the weight assigned to component stimuli appear to be potential candidates for embracing the data.     

Observations on associative grouping (in honor of Jacob Beck)

Beck (1972) pointed out that grouping tasks, where one set of elements is distinguished from another, do not imply associative links within either set of elements but could rely on processes of segregation. Although many physiological theories of grouping are purely associative there are not many methods of isolating associative processes psychophysically. One method has been to measure common resolutions of elements when they are presented under conditions of depth and motion ambiguity. In the present paper a number of the issues surrounding grouping are considered, particularly in relation to associative processes. These include whether grouping is post-constancy and post-completion, grouping with and without emergent features, the role of top-down processing and attention and the explanatory role of general principles such as simplicity and likelihood.     

Modeling complex tone perception: grouping harmonics with combination-sensitive neurons

 Perception of complex communication sounds is a major function of the auditory system. To create a coherent percept of these sounds the auditory system may instantaneously group or bind multiple harmonics within complex sounds. This perception strategy simplifies further processing of complex sounds and facilitates their meaningful integration with other sensory inputs. Based on experimental data and a realistic model, we propose that associative learning of combinations of harmonic frequencies and nonlinear facilitation of responses to those combinations, also referred to as “combination-sensitivity,” are important for spectral grouping. For our model, we simulated combination sensitivity using Hebbian and associative types of synaptic plasticity in auditory neurons. We also provided a parallel tonotopic input that converges and diverges within the network. Neurons in higher-order layers of the network exhibited an emergent property of multifrequency tuning that is consistent with experimental findings. Furthermore, this network had the capacity to “recognize” the pitch or fundamental frequency of a harmonic tone complex even when the fundamental frequency itself was missing. Received: 6 October 2001 / Accepted in revised form: 21 January 2002     

A holographic model for associative memory chains

Holographic brain models are well suited to describe specific brain functions. Central nervous systems and holographic systems both show parallel information processing and non-localized storage in common. To process information both systems use correlation functions suggesting to develop cybernetical brain models in terms of holography. Associative holographic storage is done with two simultaneously existing patterns. They may reconstruct each other mutually. Time-sequentially existing patterns are connected to associative chains, if every two succeeding patterns do exist within a common period of time in order to be stored in pairs. Read out (recall) of associative chains—reconstructing coupled patterns which didn't exist simultaneously—requires advanced holographic techniques. Three different methods are described and tested experimentally. The underlying principles are feedback mechanisms, nonlinearities of the storage material and tridimensional architecture of the voluminous recording medium. Those principles evidently occur in neural storage systems supporting analogous information processing in neural- and holographic systems.     

Associative short-term synaptic plasticity mediated by endocannabinoids

Associative learning is important on rapid timescales, but no suitable form of short-term plasticity has been identified that is both associative and synapse specific. Here, we assess whether endocannabinoids can mediate such plasticity. In the cerebellum, bursts of parallel fiber (PF) activity evoke endocannabinoid release from Purkinje cell dendrites that results in retrograde synaptic inhibition lasting seconds. We find that the powerful climbing fiber (CF) to Purkinje cell synapse regulates this inhibition. Compared to PF stimulation alone, coactivation of PF and CF synapses greatly enhanced endocannabinoid-mediated inhibition of PF synapses. Retrograde inhibition was restricted to PFs activated within several hundred milliseconds of CF activation. This associative plasticity reflects two aspects of calcium-dependent endocannabinoid release. First, PF-mediated activation of metabotropic glutamate receptors locally reduced the dendritic calcium levels required for endocannabinoid release. Second, CF and PF coactivation evoked localized supralinear dendritic calcium signals. Thus, endocannabinoids mediate transient associative synaptic plasticity.     

Reversible associative depression and nonassociative potentiation at a parallel fiber synapse

The electrosensory lobe (ELL) of mormyrid electric fish is one of several cerebellum-like sensory structures in fish that remove predictable features of the sensory inflow. This adaptive process obeys anti-Hebbian rules and appears to be mediated by associative depression at the synapses between parallel fibers and Purkinje-like cells of ELL. We show here that there is also a nonassociative potentiation at this synapse that depends only on the repeated occurrence of the EPSP. The depression can be reversed by the potentiation and vice versa. Finally, we show that the associative depression requires NMDA receptor activation, changes in postsynaptic calcium, and the occurrence of a postsynaptic dendritic spike within a few milliseconds following EPSP onset.     

Response selection in operant learning

We show that simple, contiguity-based, nonassociative response-selection process provides a qualitative account for both anomalous and nonanomalous properties of operant conditioning. The process can easily be extended to permit associative effects; it may therefore represent the initial processing stage for all conditioning in higher vertebrates.     

The evolution of associative learning: A factor in the Cambrian explosion

The Cambrian explosion is probably the most spectacular diversification in evolutionary history, and understanding it has been a challenge for biologists since the time of Darwin. We propose that one of the key factors that drove this great diversification was associative learning. Although the evolutionary emergence of associative learning required only small modifications in already existing memory mechanisms and may have occurred in parallel in several groups, once this type of learning appeared on the evolutionary scene, it led to extreme diversifying selection at the ecological level: it enabled animals to exploit new niches, promoted new types of relations and arms races, and led to adaptive responses that became fixed through genetic accommodation processes. This learning-based diversification was accompanied by neurohormonal stress, which led to an ongoing destabilization and re-patterning of the epigenome, which, in turn, enabled further morphological, physiological, and behavioral diversification. Our hypothesis combines several previous ideas about the dynamics of the Cambrian explosion and provides a unifying framework that includes both ecological and genomic factors. We conclude by suggesting research directions that would clarify the timing and manner in which associative learning evolved, and the effects it had on the evolution of nervous systems, genomes, and animal morphology.     

Associative fear learning enhances sparse network coding in primary sensory cortex

Several models of associative learning predict that stimulus processing changes during association formation. How associative learning reconfigures neural circuits in primary sensory cortex to "learn" associative attributes of a stimulus remains unknown. Using 2-photon in vivo calcium imaging to measure responses of networks of neurons in primary somatosensory cortex, we discovered that associative fear learning, in which whisker stimulation is paired with foot shock, enhances sparse population coding and robustness of the conditional stimulus, yet decreases total network activity. Fewer cortical neurons responded to stimulation of the trained whisker than in controls, yet their response strength was enhanced. These responses were not observed in mice exposed to a nonassociative learning procedure. Our results define how the cortical representation of a sensory stimulus is shaped by associative fear learning. These changes are proposed to enhance efficient sensory processing after associative learning.     

The cortical motor system

The cortical motor system of primates is formed by a mosaic of anatomically and functionally distinct areas. These areas are not only involved in motor functions, but also play a role in functions formerly attributed to higher order associative cortical areas. In the present review, we discuss three types of higher functions carried out by the motor cortical areas: sensory-motor transformations, action understanding, and decision processing regarding action execution. We submit that generating internal representations of actions is central to cortical motor function. External contingencies and motivational factors determine then whether these action representations are transformed into actual actions.     

Bio-computational model of object-recognition: quantum Hebbian processing with neurally shaped Gabor wavelets

Theoretical and simulational evidence, as well as experimental indications, are accumulating that quantum associative memory and imaging are possible. We compare these data with biological evidence, since we find them to a significant extent compatible. This paper presents a computationally implementable integrative model of appearance-based viewpoint-invariant recognition of objects. The neuro-quantum hybrid model incorporates neural processing up to V1 and quantum associative processing in V1, achieving together an object-recognition result in V2 and ITC. Results of our simulation of the central quantum-like parts of the bio-model, receiving neurally pre-processed inputs, are presented. This part contains our original simulated storage by multiple quantum interference of image-encoding Gabor wavelets done in a Hebbian way, especially using the Griniasty et al. pose-sequence learning rule.     

Conserved fMRI and LFP signals during new associative learning in the human and macaque monkey medial temporal lobe

We measured local field potential (LFP) and blood-oxygen-level-dependent (BOLD) functional magnetic resonance imaging (fMRI) in the medial temporal lobes of monkeys and humans, respectively, as they performed the same conditional motor associative learning task. Parallel analyses were used to examine both data sets. Despite significantly faster learning in humans relative to monkeys, we found equivalent neural signals differentiating new versus highly familiar stimuli, first stimulus presentation, trial outcome, and learning strength in the entorhinal cortex and hippocampus of both species. Thus, the use of parallel behavioral tasks and analyses in monkeys and humans revealed conserved patterns of neural activity across the medial temporal lobe during an associative learning task.     

Individual vocal recognition and the effect of partial lesions to HVc on discrimination, learning, and categorization of conspecific song in adult songbirds

Among songbirds, the capacity to associate particular songs with particular singers (i.e., vocal recognition) forms the cognitive basis for more complex communication behaviors such as female choice and territoriality. In the present study, we combine operant conditioning techniques and excitotoxic lesions to the forebrain nucleus HVc to examine the role of this region in the discrimination, associative learning, and categorization of conspecific song. We trained adult male and female European starlings, Sturnus vulgaris, to recognize simultaneously the songs of several conspecific males. Then, using a series of transfer procedures, we demonstrate that correct recognition does not generalize to song bouts containing novel motifs from familiar singers. This suggests that starlings do not make use of individually invariant source or filter characteristics for vocal recognition. We then lesioned a portion of HVc bilaterally with ibotenic acid, and exposed the birds to a series of manipulations testing the discrimination, associative learning, and categorization of conspecific song. The lesions attenuated song production among males, but retention of the basic recognition task (i.e., maintenance of the discrimination) was unaffected. However, when the response contingencies were reversed-as a test of associative learning independent of discrimination-the initial performance and subsequent learning rate were negatively correlated with the size of the HVc lesions. This suggests that HVc plays a role in the formation of associations between a song and some referent. The results of this study are discussed in light of earlier claims regarding the role of HVc in the perceptual processing of conspecific song.     

Massively parallel acceleration methods for image handling operations

In the image handling and image processing areas, most operations can be executed in a pixel-by-pixel or cluster-by-cluster manner. These parallel and simultaneous executions have many benefits, and many researchers showed remarkable improvements. In this paper, we started from a specific and practical image handling and feature extraction sequences. We focused on the detailed design and robust implementation on the modern massively parallel architecture of CUDA. We present the enhanced features of our implementation and their design details. Our final result shows 13 times faster execution speed, in comparison with its previous CPU-based implementation. These methods can be applied to the variety of image manipulation processes.     

Error tolerant associative memory

We present a new approach to enlarging the basin of attraction of associative memory, including auto-associative memory and temporal associative memory. The memory trained by means of this method can tolerate and recover from seriously noisy patterns. Simulations show that this approach will greatly reduce the number of limit cycles.