Estimating the dimensionality of the manifold underlying multi-electrode neural recordings

It is generally accepted that the number of neurons in a given brain area far exceeds the number of neurons needed to carry any specific function controlled by that area. For example, motor areas of the human brain contain tens of millions of neurons that control the activation of tens or at most hundreds of muscles. This massive redundancy implies the covariation of many neurons, which constrains the population activity to a low-dimensional manifold within the space of all possible patterns of neural activity. To gain a conceptual understanding of the complexity of the neural activity within a manifold, it is useful to estimate its dimensionality, which quantifies the number of degrees of freedom required to describe the observed population activity without significant information loss. While there are many algorithms for dimensionality estimation, we do not know which are well suited for analyzing neural activity. The objective of this study was to evaluate the efficacy of several representative algorithms for estimating the dimensionality of linearly and nonlinearly embedded data. We generated synthetic neural recordings with known intrinsic dimensionality and used them to test the algorithms’ accuracy and robustness. We emulated some of the important challenges associated with experimental data by adding noise, altering the nature of the embedding of the low-dimensional manifold within the high-dimensional recordings, varying the dimensionality of the manifold, and limiting the amount of available data. We demonstrated that linear algorithms overestimate the dimensionality of nonlinear, noise-free data. In cases of high noise, most algorithms overestimated the dimensionality. We thus developed a denoising algorithm based on deep learning, the “Joint Autoencoder”, which significantly improved subsequent dimensionality estimation. Critically, we found that all algorithms failed when the intrinsic dimensionality was high (above 20) or when the amount of data used for estimation was low. Based on the challenges we observed, we formulated a pipeline for estimating the dimensionality of experimental neural data.     

Novel method for the display of multivariate data using neural networks

A neural network has been used to reduce the dimensionality of multivariate data sets to produce two-dimensional (2D) displays of these sets. The data consisted of physicochemical properties for sets of biologically active molecules calculated by computational chemistry methods. Previous work has demonstrated that these data contain sufficient relevant information to classify the compounds according to their biological activity. The plots produced by the neural network are compared with results from two other techniques for linear and nonlinear dimension reduction, and are shown to give comparable and, in one case, superior results. Advantages of this technique are discussed.     

多通道神经信号分析方法及其应用

在神经科学研究中,多通道记录方法被普遍应用在对神经元群体活动特性的研究中.通过分析多个神经元的活动,可以了解神经系统协同编码外界信息的规则以及大脑实现各项功能的机制.为了挖掘出多通道神经信号中携带的信息及其潜在的相关性,需要合适的计算方法辅助对神经元放电活动进行解码.本文回顾了多通道神经信号分析中的一些常见方法,以及它...     

Stepping out of the box: information processing in the neural networks of the basal ganglia

The Albin-DeLong 'box and arrow' model has long been the accepted standard model for the basal ganglia network. However, advances in physiological and anatomical research have enabled a more detailed neural network approach. Recent computational models hold that the basal ganglia use reinforcement signals and local competitive learning rules to reduce the dimensionality of sparse cortical information. These models predict a steady-state situation with diminished efficacy of lateral inhibition and low synchronization. In this framework, Parkinson's disease can be characterized as a persistent state of negative reinforcement, inefficient dimensionality reduction, and abnormally synchronized basal ganglia activity.     

Nonlinear dimensionality reduction: Alternative ordination approaches for extracting and visualizing biodiversity patterns in tropical montane forest vegetation data

Ecological patterns are difficult to extract directly from vegetation data. The respective surveys provide a high number of interrelated species occurrence variables. Since often only a limited number of ecological gradients determine species distributions, the data might be represented by much fewer but effectively independent variables. This can be achieved by reducing the dimensionality of the data. Conventional methods are either limited to linear feature extraction (e.g., principal component analysis, and Classical Multidimensional Scaling, CMDS) or require a priori assumptions on the intrinsic data dimensionality (e.g., Nonmetric Multidimensional Scaling, NMDS, and self organized maps, SOM).In this study we explored the potential of Isometric Feature Mapping (Isomap). This new method of dimensionality reduction is a nonlinear generalization of CMDS. Isomap is based on a nonlinear geodesic inter-point distance matrix. Estimating geodesic distances requires one free threshold parameter, which defines linear geometry to be preserved in the global nonlinear distance structure. We compared Isomap to its linear (CMDS) and nonmetric (NMDS) equivalents. Furthermore, the use of geodesic distances allowed also extending NMDS to a version that we called NMDS-G. In addition we investigated a supervised Isomap variant (S-Isomap) and showed that all these techniques are interpretable within a single methodical framework.As an example we investigated seven plots (subdivided in 456 subplots) in different secondary tropical montane forests with 773 species of vascular plants. A key problem for the study of tropical vegetation data is the heterogeneous small scale variability implying large ranges of β-diversity. The CMDS and NMDS methods did not reduce the data dimensionality reasonably. On the contrary, Isomap explained 95% of the data variance in the first five dimensions and provided ecologically interpretable visualizations; NMDS-G yielded similar results. The main shortcoming of the latter was the high computational cost and the requirement to predefine the dimension of the embedding space. The S-Isomap learning scheme did not improve the Isomap variant for an optimal threshold parameter but substantially improved the nonoptimal solutions.We conclude that Isomap as a new ordination method allows effective representations of high dimensional vegetation data sets. The method is promising since it does not require a priori assumptions, and is computationally highly effective.     

High-resolution multiple-unit EEG in cat auditory cortex reveals large spatio-temporal stochastic interactions

It has been argued that information processing in the cortex is optimised with regard to certain information theoretic principles. We have, for instance, recently shown that spike-timing dependent plasticity can improve an information-theoretic measure called spatio-temporal stochastic interaction which captures how strongly a set of neurons cooperates in space and time. Systems with high stochastic interaction reveal Poisson spike trains but nonetheless occupy only a strongly reduced area in their global phase space, they reveal repetiting but complex global activation patterns, and they can be interpreted as computational systems operating on selected sets of collective patterns or "global states" in a rule-like manner. In the present work we investigate stochastic interaction in high-resolution EEG-data from cat auditory cortex. Using Kohonen maps to reduce the high-dimensional dynamics of the system, we are able to detect repetiting system states and estimate the stochastic interaction in the data, which turns out to be fairly high. This suggests an organised cooperation in the underlying neural networks which cause the data and may reflect generic intrinsic computational capabilities of the cortex.     

Stimulus-induced bifurcations in discrete-time neural oscillators

Based on theoretical issues and neurobiological evidence, considerable interest has recently focused on dynamic computational elements in neural systems. Such elements respond to stimuli by altering their dynamical behavior rather than by changing a scalar output. In particular, neural oscillators capable of chaotic dynamics represent a potentially very rich substrate for complex spatiotemporal information processing. However, the response properties of such systems must be studied in detail before they can be used as computational elements in neural models. In this paper, we focus on the response of a very simple discrete-time neural oscillator model to a fixed input. We show that the oscillator responds to the stimulus through a fairly complex set of bifurcations, and shows critical switching between attractors. This information can be used to construct very sophisticated dynamic computational elements with well-understood response properties. Examples of such elements are presented in the paper. We end with a brief discussion of simple architectures for networks of dynamical elements, and the relevance of our results to neurobiological models. Received: 7 August 1997 / Accepted in revised form: 22 April 1998     

Degenerate coding in neural systems

When the dimensionality of a neural circuit is substantially larger than the dimensionality of the variable it encodes, many different degenerate network states can produce the same output. In this review I will discuss three different neural systems that are linked by this theme. The pyloric network of the lobster, the song control system of the zebra finch, and the odor encoding system of the locust, while different in design, all contain degeneracies between their internal parameters and the outputs they encode. Indeed, although the dynamics of song generation and odor identification are quite different, computationally, odor recognition can be thought of as running the song generation circuitry backwards. In both of these systems, degeneracy plays a vital role in mapping a sparse neural representation devoid of correlations onto external stimuli (odors or song structure) that are strongly correlated. I argue that degeneracy between input and output states is an inherent feature of many neural systems, which can be exploited as a fault-tolerant method of reliably learning, generating, and discriminating closely related patterns.     

Computing with neural synchrony

Neurons communicate primarily with spikes, but most theories of neural computation are based on firing rates. Yet, many experimental observations suggest that the temporal coordination of spikes plays a role in sensory processing. Among potential spike-based codes, synchrony appears as a good candidate because neural firing and plasticity are sensitive to fine input correlations. However, it is unclear what role synchrony may play in neural computation, and what functional advantage it may provide. With a theoretical approach, I show that the computational interest of neural synchrony appears when neurons have heterogeneous properties. In this context, the relationship between stimuli and neural synchrony is captured by the concept of synchrony receptive field, the set of stimuli which induce synchronous responses in a group of neurons. In a heterogeneous neural population, it appears that synchrony patterns represent structure or sensory invariants in stimuli, which can then be detected by postsynaptic neurons. The required neural circuitry can spontaneously emerge with spike-timing-dependent plasticity. Using examples in different sensory modalities, I show that this allows simple neural circuits to extract relevant information from realistic sensory stimuli, for example to identify a fluctuating odor in the presence of distractors. This theory of synchrony-based computation shows that relative spike timing may indeed have computational relevance, and suggests new types of neural network models for sensory processing with appealing computational properties.     

The mystery of structure and function of sensory processing areas of the neocortex: a resolution

Many different neural models have been proposed to account for major characteristics of the memory phenomenon family in primates. However, in spite of the large body of neurophysiological, anatomical and behavioral data, there is no direct evidence for supporting one model while falsifying the others. And yet, we can discriminate models based on their complexity and/or their predictive power. In this paper we present a computational framework with our basic assumption that neural information processing is performed by generative networks. A complex architecture is 'derived' by using information-theoretic principles. We find that our approach seems to uncover possible relations among the functional memory units (declarative and implicit memory) and the process of information encoding in primates. The architecture can also be related to the entorhinal-hippocampal loop. An effort is made to form a prototype of this computational architecture and to map it onto the functional units of the neocortex. This mapping leads us to claim that one may gain a better understanding by considering that anatomical and functional layers of the cortex differ. Philosophical consequences regarding the homunculus fallacy are also considered.     

Embedding optimization reveals long-lasting history dependence in neural spiking activity

Information processing can leave distinct footprints on the statistics of neural spiking. For example, efficient coding minimizes the statistical dependencies on the spiking history, while temporal integration of information may require the maintenance of information over different timescales. To investigate these footprints, we developed a novel approach to quantify history dependence within the spiking of a single neuron, using the mutual information between the entire past and current spiking. This measure captures how much past information is necessary to predict current spiking. In contrast, classical time-lagged measures of temporal dependence like the autocorrelation capture how long—potentially redundant—past information can still be read out. Strikingly, we find for model neurons that our method disentangles the strength and timescale of history dependence, whereas the two are mixed in classical approaches. When applying the method to experimental data, which are necessarily of limited size, a reliable estimation of mutual information is only possible for a coarse temporal binning of past spiking, a so-called past embedding. To still account for the vastly different spiking statistics and potentially long history dependence of living neurons, we developed an embedding-optimization approach that does not only vary the number and size, but also an exponential stretching of past bins. For extra-cellular spike recordings, we found that the strength and timescale of history dependence indeed can vary independently across experimental preparations. While hippocampus indicated strong and long history dependence, in visual cortex it was weak and short, while in vitro the history dependence was strong but short. This work enables an information-theoretic characterization of history dependence in recorded spike trains, which captures a footprint of information processing that is beyond time-lagged measures of temporal dependence. To facilitate the application of the method, we provide practical guidelines and a toolbox.     

A chemotactic model of trunk neural crest cell migration

Trunk neural crest cells follow a common ventral migratory pathway but are distributed into two distinct locations to form discrete sympathetic and dorsal root ganglia along the vertebrate axis. Although fluorescent cell labeling and time‐lapse studies have recorded complex trunk neural crest cell migratory behaviors, the signals that underlie this dynamic patterning remain unclear. The absence of molecular information has led to a number of mechanistic hypotheses for trunk neural crest cell migration. Here, we review recent data in support of three distinct mechanisms of trunk neural crest cell migration and develop and simulate a computational model based on chemotactic signaling. We show that by integrating the timing and spatial location of multiple chemotactic signals, trunk neural crest cells may be accurately positioned into two distinct targets that correspond to the sympathetic and dorsal root ganglia. In doing so, we honor the contributions of Wilhelm His to his identification of the neural crest and extend the observations of His and others to better understand a complex question in neural crest cell biology.     

Bursting dynamics remarkably improve the performance of neural networks on liquid computing

Burst firings are functionally important behaviors displayed by neural circuits, which plays a primary role in reliable transmission of electrical signals for neuronal communication. However, with respect to the computational capability of neural networks, most of relevant studies are based on the spiking dynamics of individual neurons, while burst firing is seldom considered. In this paper, we carry out a comprehensive study to compare the performance of spiking and bursting dynamics on the capability of liquid computing, which is an effective approach for intelligent computation of neural networks. The results show that neural networks with bursting dynamic have much better computational performance than those with spiking dynamics, especially for complex computational tasks. Further analysis demonstrate that the fast firing pattern of bursting dynamics can obviously enhance the efficiency of synaptic integration from pre-neurons both temporally and spatially. This indicates that bursting dynamic can significantly enhance the complexity of network activity, implying its high efficiency in information processing.     

Efficient computation via sparse coding in electrosensory neural networks

The electric sense combines spatial aspects of vision and touch with temporal features of audition. Its accessible neural architecture shares similarities with mammalian sensory systems and allows for recordings from successive brain areas to test hypotheses about neural coding. Further, electrosensory stimuli encountered during prey capture, navigation, and communication, can be readily synthesized in the laboratory. These features enable analyses of the neural circuitry that reveal general principles of encoding and decoding, such as segregation of information into separate streams and neural response sparsification. A systems level understanding arises via linkage between cellular differentiation and network architecture, revealed by in vitro and in vivo analyses, while computational modeling reveals how single cell dynamics and connectivity shape the sparsification process.     

Sentence alignment using feed forward neural network

Parallel corpora have become an essential resource for work in multi lingual natural language processing. However, sentence aligned parallel corpora are more efficient than non-aligned parallel corpora for cross language information retrieval and machine translation applications. In this paper, we present a new approach to align sentences in bilingual parallel corpora based on feed forward neural network classifier. A feature parameter vector is extracted from the text pair under consideration. This vector contains text features such as length, punctuate score, and cognate score values. A set of manually prepared training data has been assigned to train the feed forward neural network. Another set of data was used for testing. Using this new approach, we could achieve an error reduction of 60% over length based approach when applied on English-Arabic parallel documents. Moreover this new approach is valid for any language pair and it is quite flexible approach since the feature parameter vector may contain more/less or different features than that we used in our system such as lexical match feature.     

Inevitable evolutionary temporal elements in neural processing: a study based on evolutionary simulations

Recent studies have suggested that some neural computational mechanisms are based on the fine temporal structure of spiking activity. However, less effort has been devoted to investigating the evolutionary aspects of such mechanisms. In this paper we explore the issue of temporal neural computation from an evolutionary point of view, using a genetic simulation of the evolutionary development of neural systems. We evolve neural systems in an environment with selective pressure based on mate finding, and examine the temporal aspects of the evolved systems. In repeating evolutionary sessions, there was a significant increase during evolution in the mutual information between the evolved agent's temporal neural representation and the external environment. In ten different simulated evolutionary sessions, there was an increased effect of time-related neural ablations on the agents' fitness. These results suggest that in some fitness landscapes the emergence of temporal elements in neural computation is almost inevitable. Future research using similar evolutionary simulations may shed new light on various biological mechanisms.     

Modelling memory functions with recurrent neural networks consisting of input compensation units: I. Static situations

Humans are able to form internal representations of the information they process—a capability which enables them to perform many different memory tasks. Therefore, the neural system has to learn somehow to represent aspects of the environmental situation; this process is assumed to be based on synaptic changes. The situations to be represented are various as for example different types of static patterns but also dynamic scenes. How are neural networks consisting of mutually connected neurons capable of performing such tasks? Here we propose a new neuronal structure for artificial neurons. This structure allows one to disentangle the dynamics of the recurrent connectivity from the dynamics induced by synaptic changes due to the learning processes. The error signal is computed locally within the individual neuron. Thus, online learning is possible without any additional structures. Recurrent neural networks equipped with these computational units cope with different memory tasks. Examples illustrate how information is extracted from environmental situations comprising fixed patterns to produce sustained activity and to deal with simple algebraic relations.