The Effect of Slow Electrical Stimuli to Achieve Learning in Cultured Networks of Rat Cortical Neurons

Learning, or more generally, plasticity may be studied using cultured networks of rat cortical neurons on multi electrode arrays. Several protocols have been proposed to affect connectivity in such networks. One of these protocols, proposed by Shahaf and Marom, aimed to train the input-output relationship of a selected connection in a network using slow electrical stimuli. Although the results were quite promising, the experiments appeared difficult to repeat and the training protocol did not serve as a basis for wider investigation yet. Here, we repeated their protocol, and compared our ‘learning curves’ to the original results. Although in some experiments the protocol did not seem to work, we found that on average, the protocol showed a significantly improved stimulus response indeed. Furthermore, the protocol always induced functional connectivity changes that were much larger than changes that occurred after a comparable period of random or no stimulation. Finally, our data shows that stimulation at a fixed electrode induces functional connectivity changes of similar magnitude as stimulation through randomly varied sites; both larger than spontaneous connectivity fluctuations. We concluded that slow electrical stimulation always induced functional connectivity changes, although uncontrolled. The magnitude of change increased when we applied the adaptive (closed-loop) training protocol. We hypothesize that networks develop an equilibrium between connectivity and activity. Induced connectivity changes depend on the combination of applied stimulus and initial connectivity. Plain stimuli may drive networks to the nearest equilibrium that accommodates this input, whereas adaptive stimulation may direct the space for exploration and force networks to a new balance, at a larger distance from the initial state.     

A Three-Threshold Learning Rule Approaches the Maximal Capacity of Recurrent Neural Networks

Understanding the theoretical foundations of how memories are encoded and retrieved in neural populations is a central challenge in neuroscience. A popular theoretical scenario for modeling memory function is the attractor neural network scenario, whose prototype is the Hopfield model. The model simplicity and the locality of the synaptic update rules come at the cost of a poor storage capacity, compared with the capacity achieved with perceptron learning algorithms. Here, by transforming the perceptron learning rule, we present an online learning rule for a recurrent neural network that achieves near-maximal storage capacity without an explicit supervisory error signal, relying only upon locally accessible information. The fully-connected network consists of excitatory binary neurons with plastic recurrent connections and non-plastic inhibitory feedback stabilizing the network dynamics; the memory patterns to be memorized are presented online as strong afferent currents, producing a bimodal distribution for the neuron synaptic inputs. Synapses corresponding to active inputs are modified as a function of the value of the local fields with respect to three thresholds. Above the highest threshold, and below the lowest threshold, no plasticity occurs. In between these two thresholds, potentiation/depression occurs when the local field is above/below an intermediate threshold. We simulated and analyzed a network of binary neurons implementing this rule and measured its storage capacity for different sizes of the basins of attraction. The storage capacity obtained through numerical simulations is shown to be close to the value predicted by analytical calculations. We also measured the dependence of capacity on the strength of external inputs. Finally, we quantified the statistics of the resulting synaptic connectivity matrix, and found that both the fraction of zero weight synapses and the degree of symmetry of the weight matrix increase with the number of stored patterns.     

Functional Calcium Imaging in Developing Cortical Networks

A hallmark pattern of activity in developing nervous systems is spontaneous, synchronized network activity. Synchronized activity has been observed in intact spinal cord, brainstem, retina, cortex and dissociated neuronal culture preparations. During periods of spontaneous activity, neurons depolarize to fire single or bursts of action potentials, activating many ion channels. Depolarization activates voltage-gated calcium channels on dendrites and spines that mediate calcium influx. Highly synchronized electrical activity has been measured from local neuronal networks using field electrodes. This technique enables high temporal sampling rates but lower spatial resolution due to integrated read-out of multiple neurons at one electrode. Single cell resolution of neuronal activity is possible using patch-clamp electrophysiology on single neurons to measure firing activity. However, the ability to measure from a network is limited to the number of neurons patched simultaneously, and typically is only one or two neurons. The use of calcium-dependent fluorescent indicator dyes has enabled the measurement of synchronized activity across a network of cells. This technique gives both high spatial resolution and sufficient temporal sampling to record spontaneous activity of the developing network.A key feature of newly-forming cortical and hippocampal networks during pre- and early postnatal development is spontaneous, synchronized neuronal activity (Katz & Shatz, 1996; Khaziphov & Luhmann, 2006). This correlated network activity is believed to be essential for the generation of functional circuits in the developing nervous system (Spitzer, 2006). In both primate and rodent brain, early electrical and calcium network waves are observed pre- and postnatally in vivo and in vitro (Adelsberger et al., 2005; Garaschuk et al., 2000; Lamblin et al., 1999). These early activity patterns, which are known to control several developmental processes including neuronal differentiation, synaptogenesis and plasticity (Rakic & Komuro, 1995; Spitzer et al., 2004) are of critical importance for the correct development and maturation of the cortical circuitry.In this JoVE video, we demonstrate the methods used to image spontaneous activity in developing cortical networks. Calcium-sensitive indicators, such as Fura 2-AM ester diffuse across the cell membrane where intracellular esterase activity cleaves the AM esters to leave the cell-impermeant form of indicator dye. The impermeant form of indicator has carboxylic acid groups which are able to then detect and bind calcium ions intracellularly.. The fluorescence of the calcium-sensitive dye is transiently altered upon binding to calcium. Single or multi-photon imaging techniques are used to measure the change in photons being emitted from the dye, and thus indicate an alteration in intracellular calcium. Furthermore, these calcium-dependent indicators can be combined with other fluorescent markers to investigate cell types within the active network.     

Unsupervised fuzzy clustering using Weighted Incremental Neural Networks

A new more efficient variant of a recently developed algorithm for unsupervised fuzzy clustering is introduced. A Weighted Incremental Neural Network (WINN) is introduced and used for this purpose. The new approach is called FC-WINN (Fuzzy Clustering using WINN). The WINN algorithm produces a net of nodes connected by edges, which reflects and preserves the topology of the input data set. Additional weights, which are proportional to the local densities in input space, are associated with the resulting nodes and edges to store useful information about the topological relations in the given input data set. A fuzziness factor, proportional to the connectedness of the net, is introduced in the system. A watershed-like procedure is used to cluster the resulting net. The number of the resulting clusters is determined by this procedure. Only two parameters must be chosen by the user for the FC-WINN algorithm to determine the resolution and the connectedness of the net. Other parameters that must be specified are those which are necessary for the used incremental neural network, which is a modified version of the Growing Neural Gas algorithm (GNG). The FC-WINN algorithm is computationally efficient when compared to other approaches for clustering large high-dimensional data sets.     

The Causal Inference of Cortical Neural Networks during Music Improvisations

We present an EEG study of two music improvisation experiments. Professional musicians with high level of improvisation skills were asked to perform music either according to notes (composed music) or in improvisation. Each piece of music was performed in two different modes: strict mode and “let-go” mode. Synchronized EEG data was measured from both musicians and listeners. We used one of the most reliable causality measures: conditional Mutual Information from Mixed Embedding (MIME), to analyze directed correlations between different EEG channels, which was combined with network theory to construct both intra-brain and cross-brain networks. Differences were identified in intra-brain neural networks between composed music and improvisation and between strict mode and “let-go” mode. Particular brain regions such as frontal, parietal and temporal regions were found to play a key role in differentiating the brain activities between different playing conditions. By comparing the level of degree centralities in intra-brain neural networks, we found a difference between the response of musicians and the listeners when comparing the different playing conditions.     

Oscillations in Large-Scale Cortical Networks: Map-Based Model

We develop a new computationally efficient approach for the analysis of complex large-scale neurobiological networks. Its key element is the use of a new phenomenological model of a neuron capable of replicating important spike pattern characteristics and designed in the form of a system of difference equations (a map). We developed a set of map-based models that replicate spiking activity of cortical fast spiking, regular spiking and intrinsically bursting neurons. Interconnected with synaptic currents these model neurons demonstrated responses very similar to those found with Hodgkin-Huxley models and in experiments. We illustrate the efficacy of this approach in simulations of one- and two-dimensional cortical network models consisting of regular spiking neurons and fast spiking interneurons to model sleep and activated states of the thalamocortical system. Our study suggests that map-based models can be widely used for large-scale simulations and that such models are especially useful for tasks where the modeling of specific firing patterns of different cell classes is important.     

Order-Based Representation in Random Networks of Cortical Neurons

The wide range of time scales involved in neural excitability and synaptic transmission might lead to ongoing change in the temporal structure of responses to recurring stimulus presentations on a trial-to-trial basis. This is probably the most severe biophysical constraint on putative time-based primitives of stimulus representation in neuronal networks. Here we show that in spontaneously developing large-scale random networks of cortical neurons in vitro the order in which neurons are recruited following each stimulus is a naturally emerging representation primitive that is invariant to significant temporal changes in spike times. With a relatively small number of randomly sampled neurons, the information about stimulus position is fully retrievable from the recruitment order. The effective connectivity that makes order-based representation invariant to time warping is characterized by the existence of stations through which activity is required to pass in order to propagate further into the network. This study uncovers a simple invariant in a noisy biological network in vitro; its applicability under in vivo constraints remains to be seen.     

培养海马神经元网络学习模型的构建

对于培养的神经元网络而言,学习是外界刺激与网络响应之间联系建立和调控的过程.为构建合适的神经元网络学习模型,采用闭环低频(1 Hz)成对电极的电刺激模拟认知任务,在多通道微电极阵列系统中对培养的海马神经元网络进行训练,使其发生网络层次上的学习行为.经过训练后,神经元网络在刺激后20～80ms内的早期突触后响应明显增加,响应/刺激比(在闭环训练中,电极上任一阶段连续10次刺激的早期突触后响应的个数/10)增大,响应时延减小,并且响应具有选择性,即表明,神经元网络与外界刺激之间已建立可调控的联系,该可调控联系是通过网络的响应来表现的,建立神经元网络与外界刺激之间的可调控联系即网络层次的学习.     

Unsupervised Learning and Adaptation in a Model of Adult Neurogenesis

Adult neurogenesis has long been documented in the vertebrate brain and recently even in humans. Although it has been conjectured for many years that its functional role is related to the renewing of memories, no clear mechanism as to how this can be achieved has been proposed. Using the mammalian olfactory bulb as a paradigm, we present a scheme in which incorporation of new neurons proceeds at a constant rate, while their survival is activity-dependent and thus contingent on new neurons establishing suitable connections. We show that a simple mathematical model following these rules organizes its activity so as to maximize the difference between its responses and can adapt to changing environmental conditions in unsupervised fashion, in agreement with current neurophysiological data.     

Latency-Related Development of Functional Connections in Cultured Cortical Networks

To study plasticity, we cultured cortical networks on multielectrode arrays, enabling simultaneous recording from multiple neurons. We used conditional firing probabilities to describe functional network connections by their strength and latency. These are abstract representations of neuronal pathways and may arise from direct pathways between two neurons or from a common input. Functional connections based on direct pathways should reflect synaptic properties. Therefore, we searched for long-term potentiation (this mechanism occurs in vivo when presynaptic action potentials precede postsynaptic ones with interspike intervals up to ∼20 ms) in vitro. To investigate if the strength of functional connections showed a similar latency-related development, we selected periods of monotonously increasing or decreasing strength. We observed increased incidence of short latencies (5-30 ms) during strengthening, whereas these rarely occurred during weakening. Furthermore, we saw an increased incidence of 40-65 ms latencies during weakening. Conversely, functional connections tended to strengthen in periods with short latency, whereas strengthening was significantly less than average during long latency. Our data suggest that functional connections contain information about synaptic connections, that conditional firing probability analysis is sensitive enough to detect it and that a substantial fraction of all functional connections is based on direct pathways.     

Deep Supervised,but Not Unsupervised,Models May Explain IT Cortical Representation

Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.     

Unsupervised Learning of Cone Spectral Classes from Natural Images

The first step in the evolution of primate trichromatic color vision was the expression of a third cone class not present in ancestral mammals. This observation motivates a fundamental question about the evolution of any sensory system: how is it possible to detect and exploit the presence of a novel sensory class? We explore this question in the context of primate color vision. We present an unsupervised learning algorithm capable of both detecting the number of spectral cone classes in a retinal mosaic and learning the class of each cone using the inter-cone correlations obtained in response to natural image input. The algorithm''s ability to classify cones is in broad agreement with experimental evidence about functional color vision for a wide range of mosaic parameters, including those characterizing dichromacy, typical trichromacy, anomalous trichromacy, and possible tetrachromacy.     

Using an Hebbian Learning Rule for Multi-Class SVM Classifiers

Regarding biological visual classification, recent series of experiments have enlighten the fact that data classification can be realized in the human visual cortex with latencies of about 100-150 ms, which, considering the visual pathways latencies, is only compatible with a very specific processing architecture, described by models from Thorpe et al. Surprisingly enough, this experimental evidence is in coherence with algorithms derived from the statistical learning theory. More precisely, there is a double link: on one hand, the so-called Vapnik theory offers tools to evaluate and analyze the biological model performances and on the other hand, this model is an interesting front-end for algorithms derived from the Vapnik theory. The present contribution develops this idea, introducing a model derived from the statistical learning theory and using the biological model of Thorpe et al. We experiment its performances using a restrained sign language recognition experiment. This paper intends to be read by biologist as well as statistician, as a consequence basic material in both fields have been reviewed.     

The Role of Axonal Delay in the Synchronization of Networks of Coupled Cortical Oscillators

Coupled oscillator models use a single phase variable to approximate the voltage oscillation of each neuron during repetitivefiring where the behavior of the model depends on the connectivityand the interaction function chosen to describe the coupling. Weintroduce a network model consisting of a continuum of theseoscillators that includes the effects of spatially decaying coupling and axonal delay. We derive equations for determining the stability of solutions and analyze the network behavior for two different interaction functions. The first is a sine function, and the second is derived from a compartmental model of a pyramidal cell.In both cases, the system of coupled neural oscillators can undergo a bifurcation from synchronous oscillations to waves.The change in qualitative behavior is due to the axonal delay,which causes distant connections to encourage a phase shift between cells. We suggest that this mechanism could contribute to the behavior observed in several neurobiological systems.     

Processing of Feature Selectivity in Cortical Networks with Specific Connectivity

Although non-specific at the onset of eye opening, networks in rodent visual cortex attain a non-random structure after eye opening, with a specific bias for connections between neurons of similar preferred orientations. As orientation selectivity is already present at eye opening, it remains unclear how this specificity in network wiring contributes to feature selectivity. Using large-scale inhibition-dominated spiking networks as a model, we show that feature-specific connectivity leads to a linear amplification of feedforward tuning, consistent with recent electrophysiological single-neuron recordings in rodent neocortex. Our results show that optimal amplification is achieved at an intermediate regime of specific connectivity. In this configuration a moderate increase of pairwise correlations is observed, consistent with recent experimental findings. Furthermore, we observed that feature-specific connectivity leads to the emergence of orientation-selective reverberating activity, and entails pattern completion in network responses. Our theoretical analysis provides a mechanistic understanding of subnetworks’ responses to visual stimuli, and casts light on the regime of operation of sensory cortices in the presence of specific connectivity.     

Supervised and Unsupervised Learning with Two Sites of Synaptic Integration

Many learning rules for neural networks derive from abstract objective functions. The weights in those networks are typically optimized utilizing gradient ascent on the objective function. In those networks each neuron needs to store two variables. One variable, called activity, contains the bottom-up sensory-fugal information involved in the core signal processing. The other variable typically describes the derivative of the objective function with respect to the cell's activity and is exclusively used for learning. This variable allows the objective function's derivative to be calculated with respect to each weight and thus the weight update. Although this approach is widely used, the mapping of such two variables onto physiology is unclear, and these learning algorithms are often considered biologically unrealistic. However, recent research on the properties of cortical pyramidal neurons shows that these cells have at least two sites of synaptic integration, the basal and the apical dendrite, and are thus appropriately described by at least two variables. Here we discuss whether these results could constitute a physiological basis for the described abstract learning rules. As examples we demonstrate an implementation of the backpropagation of error algorithm and a specific self-supervised learning algorithm using these principles. Thus, compared to standard, one-integration-site neurons, it is possible to incorporate interesting properties in neural networks that are inspired by physiology with a modest increase of complexity.     

Determination of Cup to Disc Ratio Using Unsupervised Machine Learning Techniques for Glaucoma Detection

The cup nerve head, optic cup, optic disc ratio and neural rim configuration are observed as important for detecting glaucoma at an early stage in clinical practice. The main clinical indicator of glaucoma optic cup to disc ratio is currently determined manually by limiting the mass screening was potential. This paper proposes the following methods for an automatic cup to disc ratio determination. In the first part of the work, fundus image of the optic disc region is considered. Clustering means K is used automatically to extract the optic disc whereas K-value is automatically selected by algorithm called hill climbing. The segmented contour of optic cup has been smoothened by two methods namely elliptical fitting and morphological fitting. Cup to disc ratio is calculated for 50 normal images and 50 fundus images of glaucoma patients. Throughout this paper, the same set of images has been used and for these images, the cup to disc ratio values are provided by ophthalmologist which is taken as the gold standard value. The error is calculated with reference to this gold standard value throughout the paper for cup to disc ratio comparison. The mean error of the K-means clustering method for elliptical and morphological fitting is 4.5% and 4.1%, respectively. Since the error is high, fuzzy C-mean clustering has been chosen and the mean error of the method for elliptical and morphological fitting is 3.83% and 3.52%. The error can further be minimized by considering the inter pixel relation. To achieve another algorithm is by Spatially Weighted fuzzy C-means Clustering (SWFCM) is used. The optic disc and optic cup have clustered and segmented by SWFCM Clustering. The SWFCM mean error clustering method for elliptical and morphological fitting is 3.06% and 1.67%, respectively. In this work fundus images were collected from Aravind eye Hospital, Pondicherry.