Learning in human neural networks on microelectrode arrays

This paper describes experiments involving the growth of human neural networks of stem cells on a MEA (microelectrode array) support. The microelectrode arrays (MEAs) are constituted by a glass support in which a set of tungsten electrodes are inserted. The artificial neural network (ANN) paradigm was used by stimulating the neurons in parallel with digital patterns distributed on eight channels, then by analyzing a parallel multichannel output. In particular, the microelectrodes were connected following two different architectures, one inspired by the Kohonen's SOM, the other by the Hopfield network. The output signals have been analyzed in order to evaluate the possibility of organized reactions by the natural neurons.f The results show that the network of human neurons reacts selectively to the subministered digital signals, i.e., it produces similar output signals referred to identical or similar patterns, and clearly differentiates the outputs coming from different stimulations. Analyses performed with a special artificial neural network called ITSOM show the possibility to codify the neural responses to different patterns, thus to interpret the signals coming from the network of biological neurons, assigning a code to each output. It is straightforward to verify that identical codes are generated by the neural reactions to similar patterns. Further experiments are to be designed that improve the hybrid neural networks' capabilities and to test the possibility of utilizing the organized answers of the neurons in several ways.     

Liquid state machines and cultured cortical networks: the separation property

In vitro neural networks of cortical neurons interfaced to a computer via multichannel microelectrode arrays (MEA) provide a unique paradigm to create a hybrid neural computer. Unfortunately, only rudimentary information about these in vitro network's computational properties or the extent of their abilities are known. To study those properties, a liquid state machine (LSM) approach was employed in which the liquid (typically an artificial neural network) was replaced with a living cortical network and the input and readout functions were replaced by the MEA-computer interface. A key requirement of the LSM architecture is that inputs into the liquid state must result in separable outputs based on the liquid's response (separation property). In this paper, high and low frequency multi-site stimulation patterns were applied to the living cortical networks. Two template-based classifiers, one based on Euclidean distance and a second based on a cross-correlation were then applied to measure the separation of the input-output relationship. The result was over a 95% (99.8% when nonstationarity is compensated) input reconstruction accuracy for the high and low frequency patterns, confirming the existence of the separation property in these biological networks.     

Information processing in social insect networks

Investigating local-scale interactions within a network makes it possible to test hypotheses about the mechanisms of global network connectivity and to ask whether there are general rules underlying network function across systems. Here we use motif analysis to determine whether the interactions within social insect colonies resemble the patterns exhibited by other animal associations or if they exhibit characteristics of biological regulatory systems. Colonies exhibit a predominance of feed-forward interaction motifs, in contrast to the densely interconnected clique patterns that characterize human interaction and animal social networks. The regulatory motif signature supports the hypothesis that social insect colonies are shaped by selection for network patterns that integrate colony functionality at the group rather than individual level, and demonstrates the utility of this approach for analysis of selection effects on complex systems across biological levels of organization.     

Network Connections That Evolve to Circumvent the Inverse Optics Problem

A fundamental problem in vision science is how useful perceptions and behaviors arise in the absence of information about the physical sources of retinal stimuli (the inverse optics problem). Psychophysical studies show that human observers contend with this problem by using the frequency of occurrence of stimulus patterns in cumulative experience to generate percepts. To begin to understand the neural mechanisms underlying this strategy, we examined the connectivity of simple neural networks evolved to respond according to the cumulative rank of stimulus luminance values. Evolved similarities with the connectivity of early level visual neurons suggests that biological visual circuitry uses the same mechanisms as a means of creating useful perceptions and behaviors without information about the real world.     

Kinetic models of odor transduction implemented as artificial neural networks

We present a formal model of olfactory transduction corresponding to the biochemical reaction cascade found in chemosensory neurons. It assumes that odorants bind to receptor proteins which, in turn, activate transducer mechanisms corresponding to second messenger-mediated processes. The model is reformulated as a mathematically equivalent artificial neural network (ANN). To enable comparison of the computational power of our model, previously suggested models of chemosensory transduction are also presented in ANN versions. In ANNs, certain biological parameters, such as rate constants and affinities, are transformed into weights that can be fitted by training with a given experimental data set. After training, these weights do not necessarily equal the real biological parameters, but represent a set of values that is sufficient to simulate an experimental set of data. We used ANNs to simulate data recorded from bee subplacodes and compare the capacity of our model with ANN versions of other models. Receptor neurons of the nonpheromonal, general odor-processing subsystem of the honeybee are broadly tuned, have overlapping response spectra, and show highly nonlinear concentration dependencies and mixture interactions, i.e., synergistic and inhibitory effects. Our full model alone has the necessary complexity to simulate these complex response characteristics. To account for the complex response characteristics of honeybee receptor neurons, we suggest that several different receptor protein types and at least two second messenger systems are necessary that may interact at various levels of the transduction cascade and may eventually have opposing effects on receptor neuron excitability.     

Computational neurogenetic modelling: a pathway to new discoveries in genetic neuroscience

The paper presents a methodology for using computational neurogenetic modelling (CNGM) to bring new original insights into how genes influence the dynamics of brain neural networks. CNGM is a novel computational approach to brain neural network modelling that integrates dynamic gene networks with artificial neural network model (ANN). Interaction of genes in neurons affects the dynamics of the whole ANN model through neuronal parameters, which are no longer constant but change as a function of gene expression. Through optimization of interactions within the internal gene regulatory network (GRN), initial gene/protein expression values and ANN parameters, particular target states of the neural network behaviour can be achieved, and statistics about gene interactions can be extracted. In such a way, we have obtained an abstract GRN that contains predictions about particular gene interactions in neurons for subunit genes of AMPA, GABAA and NMDA neuro-receptors. The extent of sequence conservation for 20 subunit proteins of all these receptors was analysed using standard bioinformatics multiple alignment procedures. We have observed abundance of conserved residues but the most interesting observation has been the consistent conservation of phenylalanine (F at position 269) and leucine (L at position 353) in all 20 proteins with no mutations. We hypothesise that these regions can be the basis for mutual interactions. Existing knowledge on evolutionary linkage of their protein families and analysis at molecular level indicate that the expression of these individual subunits should be coordinated, which provides the biological justification for our optimized GRN.     

Dynamic Artificial Neural Networks with Affective Systems

Artificial neural networks (ANNs) are processors that are trained to perform particular tasks. We couple a computational ANN with a simulated affective system in order to explore the interaction between the two. In particular, we design a simple affective system that adjusts the threshold values in the neurons of our ANN. The aim of this paper is to demonstrate that this simple affective system can control the firing rate of the ensemble of neurons in the ANN, as well as to explore the coupling between the affective system and the processes of long term potentiation (LTP) and long term depression (LTD), and the effect of the parameters of the affective system on its performance. We apply our networks with affective systems to a simple pole balancing example and briefly discuss the effect of affective systems on network performance.     

A model biological neural network: the cephalopod vestibular system

Artificial neural networks (ANNs) have become increasingly sophisticated and are widely used for the extraction of patterns or meaning from complicated or imprecise datasets. At the same time, our knowledge of the biological systems that inspired these ANNs has also progressed and a range of model systems are emerging where there is detailed information not only on the architecture and components of the system but also on their ontogeny, plasticity and the adaptive characteristics of their interconnections. We describe here a biological neural network contained in the cephalopod statocysts; the statocysts are analogous to the vertebrae vestibular system and provide the animal with sensory information on its orientation and movements in space. The statocyst network comprises only a small number of cells, made up of just three classes of neurons but, in combination with the large efferent innervation from the brain, forms an 'active' sense organs that uses feedback and feed-forward mechanisms to alter and dynamically modulate the activity within cells and how the various components are interconnected. The neurons are fully accessible to physiological investigation and the system provides an excellent model for describing the mechanisms underlying the operation of a sophisticated neural network.     

Driving reservoir models with oscillations: a solution to the extreme structural sensitivity of chaotic networks

A large body of experimental and theoretical work on neural coding suggests that the information stored in brain circuits is represented by time-varying patterns of neural activity. Reservoir computing, where the activity of a recurrently connected pool of neurons is read by one or more units that provide an output response, successfully exploits this type of neural activity. However, the question of system robustness to small structural perturbations, such as failing neurons and synapses, has been largely overlooked. This contrasts with well-studied dynamical perturbations that lead to divergent network activity in the presence of chaos, as is the case for many reservoir networks. Here, we distinguish between two types of structural network perturbations, namely local (e.g., individual synaptic or neuronal failure) and global (e.g., network-wide fluctuations). Surprisingly, we show that while global perturbations have a limited impact on the ability of reservoir models to perform various tasks, local perturbations can produce drastic effects. To address this limitation, we introduce a new architecture where the reservoir is driven by a layer of oscillators that generate stable and repeatable trajectories. This model outperforms previous implementations while being resistant to relatively large local and global perturbations. This finding has implications for the design of reservoir models that capture the capacity of brain circuits to perform cognitively and behaviorally relevant tasks while remaining robust to various forms of perturbations. Further, our work proposes a novel role for neuronal oscillations found in cortical circuits, where they may serve as a collection of inputs from which a network can robustly generate complex dynamics and implement rich computations.     

An extended model for a spiking neuron class

This paper proposes an extension to the model of a spiking neuron for information processing in artificial neural networks, developing a new approach for the dynamic threshold of the integrate-and-fire neuron. This new approach invokes characteristics of biological neurons such as the behavior of chemical synapses and the receptor field. We demonstrate how such a digital model of spiking neurons can solve complex nonlinear classification with a single neuron, performing experiments for the classical XOR problem. Compared with rate-coded networks and the classical integrate-and-fire model, the trained network demonstrated faster information processing, requiring fewer neurons and shorter learning periods. The extended model validates all the logic functions of biological neurons when such functions are necessary for the proper flow of binary codes through a neural network.     

Linking dynamics of the inhibitory network to the input structure

Networks of inhibitory interneurons are found in many distinct classes of biological systems. Inhibitory interneurons govern the dynamics of principal cells and are likely to be critically involved in the coding of information. In this theoretical study, we describe the dynamics of a generic inhibitory network in terms of low-dimensional, simplified rate models. We study the relationship between the structure of external input applied to the network and the patterns of activity arising in response to that stimulation. We found that even a minimal inhibitory network can generate a great diversity of spatio-temporal patterning including complex bursting regimes with non-trivial ratios of burst firing. Despite the complexity of these dynamics, the network’s response patterns can be predicted from the rankings of the magnitudes of external inputs to the inhibitory neurons. This type of invariant dynamics is robust to noise and stable in densely connected networks with strong inhibitory coupling. Our study predicts that the response dynamics generated by an inhibitory network may provide critical insights about the temporal structure of the sensory input it receives.     

Identification of Cryptosporidium parvum oocysts by an artificial neural network approach

Microscopic detection of Cryptosporidium parvum oocysts is time-consuming, requires trained analysts, and is frequently subject to significant human errors. Artificial neural networks (ANN) were developed to help identify immunofluorescently labeled C. parvum oocysts. A total of 525 digitized images of immunofluorescently labeled oocysts, fluorescent microspheres, and other miscellaneous nonoocyst images were employed in the training of the ANN. The images were cropped to a 36- by 36-pixel image, and the cropped images were placed into two categories, oocyst and nonoocyst images. The images were converted to grayscale and processed into a histogram of gray color pixel intensity. Commercially available software was used to develop and train the ANN. The networks were optimized by varying the number of training images, number of hidden neurons, and a combination of these two parameters. The network performance was then evaluated using a set of 362 unique testing images which the network had never "seen" before. Under optimized conditions, the correct identification of authentic oocyst images ranged from 81 to 97%, and the correct identification of nonoocyst images ranged from 78 to 82%, depending on the type of fluorescent antibody that was employed. The results indicate that the ANN developed were able to generalize the training images and subsequently discern previously unseen oocyst images efficiently and reproducibly. Thus, ANN can be used to reduce human errors associated with the microscopic detection of Cryptosporidium oocysts.     

Optimization of culture conditions for the production of Pleuromutilin from Pleurotus Mutilus using a hybrid method based on central composite design, neural network, and particle swarm optimization

This study aims at optimizing the culture conditions (agitation speed, temperature and pH) of the Pleuromutilin production by Pleurotus mutilus. A hybrid methodology including a central composite design (CCD), an artificial neural network (ANN), and a particle swarm optimization algorithm (PSO) was used. Specifically, the CCD and ANN were used for conducting experiments and modeling the non-linear process, respectively. The PSO was used for two purposes: Replacing the standard back propagation in training the ANN (PSONN) and optimizing the process. In comparison to the response surface methodology (RSM) and to the Bayesian regularization neural network (BRNN), PSONN model has shown the highest modeling ability. Under this hybrid approach (PSONN-PSO), the optimum levels of culture conditions were: 242 rpm agitation speed; temperature 26.88 and pH 6.06. A production of 10,074 ± 500 ??g/g, which was in very good agreement with the prediction (10,149 ??g/g), was observed in verification experiment. The hybrid PSONN-PSO gave a yield of 27.5% greater than that obtained by the hybrid BRNN-PSO. This work shows that the combination of PSONN with the generic PSO algorithm has a good predictability and a good accuracy for bio-process optimization. This hybrid approach is sufficiently general and thus can be helpful for modeling and optimization of other industrial bio-processes.     

Classification des stades de sommeil par des réseaux de neurones artificiels hiérarchiques

The goal of our work is to provide an automatic analysis and decision tool for sleep stages classification based on an artificial neural networks (ANN). The first difficulty lies in choosing the physiological signals representation and in particular the electroencephalogram (EEG). Once the representation adopted, the next step is to design the optimal neural network determined by a learning and validation process of data from a set of sleep records. We studied several configurations of conventional ANN giving results varying from 62 to 71 %, then we proposed a new hierarchical configuration, which gives a rate of 74 % correct classification for six stages. These results lead us to further explore this issue at the representation and design of ANNs to improve the performance of our tool.     

The information theory of developmental pruning: Optimizing global network architectures using local synaptic rules

During development, biological neural networks produce more synapses and neurons than needed. Many of these synapses and neurons are later removed in a process known as neural pruning. Why networks should initially be over-populated, and the processes that determine which synapses and neurons are ultimately pruned, remains unclear. We study the mechanisms and significance of neural pruning in model neural networks. In a deep Boltzmann machine model of sensory encoding, we find that (1) synaptic pruning is necessary to learn efficient network architectures that retain computationally-relevant connections, (2) pruning by synaptic weight alone does not optimize network size and (3) pruning based on a locally-available measure of importance based on Fisher information allows the network to identify structurally important vs. unimportant connections and neurons. This locally-available measure of importance has a biological interpretation in terms of the correlations between presynaptic and postsynaptic neurons, and implies an efficient activity-driven pruning rule. Overall, we show how local activity-dependent synaptic pruning can solve the global problem of optimizing a network architecture. We relate these findings to biology as follows: (I) Synaptic over-production is necessary for activity-dependent connectivity optimization. (II) In networks that have more neurons than needed, cells compete for activity, and only the most important and selective neurons are retained. (III) Cells may also be pruned due to a loss of synapses on their axons. This occurs when the information they convey is not relevant to the target population.     

Robust emergence of small-world structure in networks of spiking neurons

Spontaneous activity in biological neural networks shows patterns of dynamic synchronization. We propose that these patterns support the formation␣of a small-world structure—network connectivity␣optimal for distributed information processing. We␣present numerical simulations with connected Hindmarsh–Rose neurons in which, starting from random connection distributions, small-world networks evolve as a result of applying an adaptive rewiring rule. The rule connects pairs of neurons that tend fire in synchrony, and disconnects ones that fail to synchronize. Repeated application of the rule leads to small-world structures. This mechanism is robustly observed for bursting and irregular firing regimes.     

Modulation of neural network activity by patterning.

Using neuronal cultures on microelectrode arrays, researchers have shown that recordable electrical activity can be influenced by chemicals in the culture environment, thus demonstrating potential applicability to biosensors or drug screening. Since practical success requires the design of robust networks with repeatable, reliable responses understanding the sources of variation is important. In this report, we used lithographic technologies to confine neurons to highly defined patterns (40 microm wide stripes); in turn these patterns gave us a measure of control over the local density of neurons (100-500 cells/mm(2)). We found that the apparent electrical activity of the network, as measured by the fraction of electrodes from which signals were recordable, increases 8-10-fold with greater local density. Also, average-firing rates of the active neurons increased 3-5-fold. We conclude that patterned networks offer one means of controlling and enhancing the responsiveness of cultured neural networks.     

Functional identification of biological neural networks using reservoir adaptation for point processes

The complexity of biological neural networks does not allow to directly relate their biophysical properties to the dynamics of their electrical activity. We present a reservoir computing approach for functionally identifying a biological neural network, i.e. for building an artificial system that is functionally equivalent to the reference biological network. Employing feed-forward and recurrent networks with fading memory, i.e. reservoirs, we propose a point process based learning algorithm to train the internal parameters of the reservoir and the connectivity between the reservoir and the memoryless readout neurons. Specifically, the model is an Echo State Network (ESN) with leaky integrator neurons, whose individual leakage time constants are also adapted. The proposed ESN algorithm learns a predictive model of stimulus-response relations in in vitro and simulated networks, i.e. it models their response dynamics. Receiver Operating Characteristic (ROC) curve analysis indicates that these ESNs can imitate the response signal of a reference biological network. Reservoir adaptation improved the performance of an ESN over readout-only training methods in many cases. This also held for adaptive feed-forward reservoirs, which had no recurrent dynamics. We demonstrate the predictive power of these ESNs on various tasks with cultured and simulated biological neural networks.     

Recurrent fractal neural networks: a strategy for the exchange of local and global information processing in the brain

The regulation of biological networks relies significantly on convergent feedback signaling loops that render a global output locally accessible. Ideally, the recurrent connectivity within these systems is self-organized by a time-dependent phase-locking mechanism. This study analyzes recurrent fractal neural networks (RFNNs), which utilize a self-similar or fractal branching structure of dendrites and downstream networks for phase-locking of reciprocal feedback loops: output from outer branch nodes of the network tree enters inner branch nodes of the dendritic tree in single neurons. This structural organization enables RFNNs to amplify re-entrant input by over-the-threshold signal summation from feedback loops with equivalent signal traveling times. The columnar organization of pyramidal neurons in the neocortical layers V and III is discussed as the structural substrate for this network architecture. RFNNs self-organize spike trains and render the entire neural network output accessible to the dendritic tree of each neuron within this network. As the result of a contraction mapping operation, the local dendritic input pattern contains a downscaled version of the network output coding structure. RFNNs perform robust, fractal data compression, thus coping with a limited number of feedback loops for signal transport in convergent neural networks. This property is discussed as a significant step toward the solution of a fundamental problem in neuroscience: how is neuronal computation in separate neurons and remote brain areas unified as an instance of experience in consciousness? RFNNs are promising candidates for engaging neural networks into a coherent activity and provide a strategy for the exchange of global and local information processing in the human brain, thereby ensuring the completeness of a transformation from neuronal computation into conscious experience.