Neural networks for protein classification

This paper describes a biomolecular classification methodology based on multilayer perceptron neural networks. The system developed is used to classify enzymes found in the Protein Data Bank. The primary goal of classification, here, is to infer the function of an (unknown) enzyme by analysing its structural similarity to a given family of enzymes. A new codification scheme was devised to convert the primary structure of enzymes into a real-valued vector. The system was tested with a different number of neural networks, training set sizes and training epochs. For all experiments, the proposed system achieved a higher accuracy rate when compared with profile hidden Markov models. Results demonstrated the robustness of this approach and the possibility of implementing fast and efficient biomolecular classification using neural networks.     

Fast time delay neural networks

This paper presents a new approach to speed up the operation of time delay neural networks. The entire data are collected together in a long vector and then tested as a one input pattern. The proposed fast time delay neural networks (FTDNNs) use cross correlation in the frequency domain between the tested data and the input weights of neural networks. It is proved mathematically and practically that the number of computation steps required for the presented time delay neural networks is less than that needed by conventional time delay neural networks (CTDNNs). Simulation results using MATLAB confirm the theoretical computations.     

An extended model for a spiking neuron class

This paper proposes an extension to the model of a spiking neuron for information processing in artificial neural networks, developing a new approach for the dynamic threshold of the integrate-and-fire neuron. This new approach invokes characteristics of biological neurons such as the behavior of chemical synapses and the receptor field. We demonstrate how such a digital model of spiking neurons can solve complex nonlinear classification with a single neuron, performing experiments for the classical XOR problem. Compared with rate-coded networks and the classical integrate-and-fire model, the trained network demonstrated faster information processing, requiring fewer neurons and shorter learning periods. The extended model validates all the logic functions of biological neurons when such functions are necessary for the proper flow of binary codes through a neural network.     

Nonlinear system identification and control based on modular neural networks

A new approach for nonlinear system identification and control based on modular neural networks (MNN) is proposed in this paper. The computational complexity of neural identification can be greatly reduced if the whole system is decomposed into several subsystems. This is obtained using a partitioning algorithm. Each local nonlinear model is associated with a nonlinear controller. These are also implemented by neural networks. The switching between the neural controllers is done by a dynamical switcher, also implemented by neural networks, that tracks the different operating points. The proposed multiple modelling and control strategy has been successfully tested on simulated laboratory scale liquid-level system.     

Application of neural networks using the volume learning algorithm for the study of structure-activity relationship in chemical compounds

A volume learning algorithm for artificial neural networks was developed to quantitatively describe the three-dimensional structure-activity relationships using as an example N-benzylpiperidine derivatives. The new algorithm combines two types of neural networks, the Kohonen and the feed-forward artificial neural networks, which are used to analyze the input grid data generated by the comparative molecular field approach. Selection of the most informative parameters using the algorithm helped to reveal the most important spatial properties of the molecules, which affect their biological activities. Cluster regions determined using the new algorithm adequately predicted the activity of molecules from a control data set.     

Neural modeling with dynamically adjustable threshold and refractory period.

A variant of the FitzHugh-Nagumo model is proposed in order to fully make use of the computational properties of intraneuronal dynamics. The mechanisms of threshold and refractory periods resulting from the double dynamical processes are qualitatively studied through computer simulation. The results show that the variant neuron model has the property that its threshold, refractory period and response amplitude are dynamically adjustable. This paper has also discussed some problems relating to collective property, learning and implementation of the neural network based on the neuron model proposed. It is noted that the implicit way to describe threshold and refractory period is advantageous to adaptive learning in neural networks and that molecular electronics probably provides an effective approach to implementing the above neuron model.     

Physiological and pathological oscillatory networks in the human motor system.

Human brain functions are heavily contingent on neural interactions both at the single neuron and the neural population or system level. Accumulating evidence from neurophysiological studies strongly suggests that coupling of oscillatory neural activity provides an important mechanism to establish neural interactions. With the availability of whole-head magnetoencephalography (MEG) macroscopic oscillatory activity can be measured non-invasively from the human brain with high temporal and spatial resolution. To localise, quantify and map oscillatory activity and interactions onto individual brain anatomy we have developed the 'dynamic imaging of coherent sources' (DICS) method which allows to identify and analyse cerebral oscillatory networks from MEG recordings. Using this approach we have characterized physiological and pathological oscillatory networks in the human sensorimotor system. Coherent 8 Hz oscillations emerge from a cerebello-thalamo-premotor-motor cortical network and exert an 8 Hz oscillatory drive on the spinal motor neurons which can be observed as a physiological tremulousness of the movement termed movement discontinuities. This network represents the neurophysiological substrate of a discrete mode of motor control. In parkinsonian resting tremor we have identified an extensive cerebral network consisting of primary motor and lateral premotor cortex, supplementary motor cortex, thalamus/basal ganglia, posterior parietal cortex and secondary somatosensory cortex, which are entrained in the tremor or twice the tremor rhythm. This low frequency entrapment of motor areas likely plays an important role in the pathophysiology of parkinsonian motor symptoms. Finally, studies on patients with postural tremor in hepatic encephalopathy revealed that this type of tremor results from a pathologically slow thalamocortical and cortico-muscular coupling during isometric hold tasks. In conclusion, the analysis of oscillatory cerebral networks provides new insights into physiological mechanisms of motor control and pathophysiological mechanisms of tremor disorders.     

Functional model of biological neural networks

A functional model of biological neural networks, called temporal hierarchical probabilistic associative memory (THPAM), is proposed in this paper. THPAM comprises functional models of dendritic trees for encoding inputs to neurons, a first type of neuron for generating spike trains, a second type of neuron for generating graded signals to modulate neurons of the first type, supervised and unsupervised Hebbian learning mechanisms for easy learning and retrieving, an arrangement of dendritic trees for maximizing generalization, hardwiring for rotation-translation-scaling invariance, and feedback connections with different delay durations for neurons to make full use of present and past informations generated by neurons in the same and higher layers. These functional models and their processing operations have many functions of biological neural networks that have not been achieved by other models in the open literature and provide logically coherent answers to many long-standing neuroscientific questions. However, biological justifications of these functional models and their processing operations are required for THPAM to qualify as a macroscopic model (or low-order approximate) of biological neural networks.     

How structure determines correlations in neuronal networks

Networks are becoming a ubiquitous metaphor for the understanding of complex biological systems, spanning the range between molecular signalling pathways, neural networks in the brain, and interacting species in a food web. In many models, we face an intricate interplay between the topology of the network and the dynamics of the system, which is generally very hard to disentangle. A dynamical feature that has been subject of intense research in various fields are correlations between the noisy activity of nodes in a network. We consider a class of systems, where discrete signals are sent along the links of the network. Such systems are of particular relevance in neuroscience, because they provide models for networks of neurons that use action potentials for communication. We study correlations in dynamic networks with arbitrary topology, assuming linear pulse coupling. With our novel approach, we are able to understand in detail how specific structural motifs affect pairwise correlations. Based on a power series decomposition of the covariance matrix, we describe the conditions under which very indirect interactions will have a pronounced effect on correlations and population dynamics. In random networks, we find that indirect interactions may lead to a broad distribution of activation levels with low average but highly variable correlations. This phenomenon is even more pronounced in networks with distance dependent connectivity. In contrast, networks with highly connected hubs or patchy connections often exhibit strong average correlations. Our results are particularly relevant in view of new experimental techniques that enable the parallel recording of spiking activity from a large number of neurons, an appropriate interpretation of which is hampered by the currently limited understanding of structure-dynamics relations in complex networks.     

Signal flow graphs and neural networks

The application of signal flow graphs to the learning process of neural networks is presented. By introducing the so-called adjoint graph, new insight into the mechanism of learning phenomena of the weights in neural networks has been obtained. The derived updating formulas are valid for both feedforward and recurrent neural networks and are especially useful from the hardware implementation point of view of the self-learning networks. The presented numerical experiments confirmed the usefulness of the presented approach. Received: 24 February 1993/Accepted in revised form: 28 July 1993     

Probabilistic computation by neuromine networks

In this paper, we address the question, can biologically feasible neural nets compute more than can be computed by deterministic polynomial time algorithms? Since we want to maintain a claim of plausibility and reasonableness we restrict ourselves to algorithmically easy to construct nets and we rule out infinite precision in parameters and in any analog parts of the computation. Our approach is to consider the recent advances in randomized algorithms and see if such randomized computations can be described by neural nets. We start with a pair of neurons and show that by connecting them with reciprocal inhibition and some tonic input, then the steady-state will be one neuron ON and one neuron OFF, but which neuron will be ON and which neuron will be OFF will be chosen at random (perhaps, it would be better to say that microscopic noise in the analog computation will be turned into a megascale random bit). We then show that we can build a small network that uses this random bit process to generate repeatedly random bits. This random bit generator can then be connected with a neural net representing the deterministic part of randomized algorithm. We, therefore, demonstrate that these neural nets can carry out probabilistic computation and thus be less limited than classical neural nets.     

Manifestation of function-follow-form in cultured neuronal networks

We expose hidden function-follow-form schemata in the recorded activity of cultured neuronal networks by comparing the activity with simulation results of a new modeling approach. Cultured networks grown from an arbitrary mixture of neuron and glia cells in the absence of external stimulations and chemical cues spontaneously form networks of different sizes (from 50 to several millions of neurons) that exhibit non-arbitrary complex spatio-temporal patterns of activity. The latter is marked by formation of a sequence of synchronized bursting events (SBEs)--short time windows (approximately 200 ms) of rapid neuron firing, separated by longer time intervals (seconds) of sporadic neuron firing. The new dynamical synapse and soma (DSS) model, used here, has been successful in generating sequences of SBEs with the same statistical scaling properties (over six time decades) as those of the small networks. Large networks generate statistically distinct sub-groups of SBEs, each with its own characteristic pattern of neuronal firing ('fingerprint'). This special function (activity) motif has been proposed to emanate from a structural (form) motif--self-organization of the large networks into a fabric of overlapping sub-networks of about 1 mm in size. Here we test this function-follow-form idea by investigating the influence of the connectivity architecture of a model network (form) on the structure of its spontaneous activity (function). We show that a repertoire of possible activity states similar to the observed ones can be generated by networks with proper underlying architecture. For example, networks composed of two overlapping sub-networks exhibit distinct types of SBEs, each with its own characteristic pattern of neuron activity that starts at a specific sub-network. We further show that it is possible to regulate the temporal appearance of the different sub-groups of SBEs by an additional non-synaptic current fed into the soma of the modeled neurons. The ability to regulate the relative temporal ordering of different SBEs might endow the networks with higher plasticity and complexity. These findings call for additional mechanisms yet to be discovered. Recent experimental observations indicate that glia cells coupled to neuronal soma might generate such non-synaptic regulating currents.     

HuC:Kaede, a useful tool to label neural morphologies in networks in vivo

A simple and efficient procedure for labeling neurons is a prerequisite for investigating the development of neural networks in zebrafish. To label neurons we used Kaede, a fluorescent protein with a photoconversion property allowing conversion from green to red fluorescence following irradiation with UV or violet light. We established a zebrafish stable transgenic line, Tg(HuC:Kaede), expressing Kaede in neurons under the control of the HuC promoter. This transgenic line was used to label a small number of neurons in the trigeminal ganglion. Also, using embryos injected with the transgene, we labeled peripheral axon arbors of a Rohon-Beard neuron at 4 days postfertilization and observed the dendrite development of a tectal neuron for 3 days. These data indicate that Kaede is a useful tool to selectively label neural networks in zebrafish.     

Attractability and location of equilibrium point of cellular neural networks with time-varying delays

This paper presents new theoretical results on global exponential stability of cellular neural networks with time-varying delays. The stability conditions depend on external inputs, connection weights and delays of cellular neural networks. Using these results, global exponential stability of cellular neural networks can be derived, and the estimate for location of equilibrium point can also be obtained. Finally, the simulating results demonstrate the validity and feasibility of our proposed approach.     

Protein interaction networks in bacteria

The complement of expressed cellular proteins - the proteome - is organized into functional, structured networks of protein interactions that mediate assembly of molecular machines and dynamic cellular pathways. Recent studies reveal the biological roles of protein interactions in bacteriophage T7 and Helicobacter pylori, and new methods allow to compare and to predict interaction networks in other species. Smaller scale networks provide biological insights into DNA replication and chromosome dynamics in Bacillus subtilis and Archeoglobus fulgidus, and into the assembly of multiprotein complexes such as the type IV secretion system of Agrobacterium tumefaciens, and the cell division machinery of Escherichia coli. Genome-wide interaction networks in several species are needed to obtain a biologically meaningful view of the higher order organization of the proteome in bacteria.     

Synchrony based learning rule of Hopfield like chaotic neural networks with desirable structure

In this paper a new learning rule for the coupling weights tuning of Hopfield like chaotic neural networks is developed in such a way that all neurons behave in a synchronous manner, while the desirable structure of the network is preserved during the learning process. The proposed learning rule is based on sufficient synchronization criteria, on the eigenvalues of the weight matrix belonging to the neural network and the idea of Structured Inverse Eigenvalue Problem. Our developed learning rule not only synchronizes all neuron’s outputs with each other in a desirable topology, but also enables us to enhance the synchronizability of the networks by choosing the appropriate set of weight matrix eigenvalues. Specifically, this method is evaluated by performing simulations on the scale-free topology.     

Pruning artificial neural networks using neural complexity measures

This paper describes a new method for pruning artificial neural networks, using a measure of the neural complexity of the neural network. This measure is used to determine the connections that should be pruned. The measure computes the information-theoretic complexity of a neural network, which is similar to, yet different from previous research on pruning. The method proposed here shows how overly large and complex networks can be reduced in size, whilst retaining learnt behaviour and fitness. The technique proposed here helps to discover a network topology that matches the complexity of the problem it is meant to solve. This novel pruning technique is tested in a robot control domain, simulating a racecar. It is shown, that the proposed pruning method is a significant improvement over the most commonly used pruning method Magnitude Based Pruning. Furthermore, some of the pruned networks prove to be faster learners than the benchmark network that they originate from. This means that this pruning method can also help to unleash hidden potential in a network, because the learning time decreases substantially for a pruned a network, due to the reduction of dimensionality of the network.