The complexity of linkage analysis with neural networks

As the focus of genome-wide scans for disease loci have shifted from simple Mendelian traits to genetically complex traits, researchers have begun to consider new alternative ways to detect linkage that will consider more than the marginal effects of a single disease locus at a time. One interesting new method is to train a neural network on a genome-wide data set in order to search for the best non-linear relationship between identity-by-descent sharing among affected siblings at markers and their disease status. We investigate here the repeatability of the neural network results from run to run, and show that the results obtained by multiple runs of the neural network method may differ quite a bit. This is most likely due to the fact that training a neural network involves minimizing an error function with a multitude of local minima.     

Stability analysis of higher-order neural networks for combinatorial optimization

Recurrent neural networks with higher order connections, from here on referred to as higher-order neural networks (HONNs), may be used for the solution of combinatorial optimization problems. In Ref. 5 a mapping of the traveling salesman problem (TSP) onto a HONN of arbitrary order was developed, thereby creating a family of related networks that can be used to solve the TSP. In this paper, we explore the trade-off between network complexity and quality of solution that is made available by the HONN mapping of the TSP. The trade-off is investigated by undertaking an analysis of the stability of valid solutions to the TSP in a HONN of arbitrary order. The techniques used to perform the stability analysis are not new, but have been widely used elsewhere in the literature. The original contribution in this paper is the application of these techniques to a HONN of arbitrary order used to solve the TSP. The results of the stability analysis show that the quality of solution is improved by increasing the network complexity, as measured by the order of the network. Furthermore, it is shown that the Hopfield network, as the simplest network in the family of higher-order networks, is expected to produce the poorest quality of solution.     

LMI-based asymptotic stability analysis of neural networks with time-varying delays

The problem of the global asymptotic stability for a class of neural networks with time-varying delays is investigated in this paper, where the activation functions are assumed to be neither monotonic, nor differentiable, nor bounded. By constructing suitable Lyapunov functionals and combining with linear matrix inequality (LMI) technique, new global asymptotic stability criteria about different types of time-varying delays are obtained. It is shown that the criteria can provide less conservative result than some existing ones. Numerical examples are given to demonstrate the applicability of the proposed approach.     

Bayesian neural networks

A neural network that uses the basic Hebbian learning rule and the Bayesian combination function is defined. Analogously to Hopfield's neural network, the convergence for the Bayesian neural network that asynchronously updates its neurons' states is proved. The performance of the Bayesian neural network in four medical domains is compared with various classification methods. The Bayesian neural network uses more sophisticated combination function than Hopfield's neural network and uses more economically the available information. The naive Bayesian classifier typically outperforms the basic Bayesian neural network since iterations in network make too many mistakes. By restricting the number of iterations and increasing the number of fixed points the network performs better than the naive Bayesian classifier. The Bayesian neural network is designed to learn very quickly and incrementally.     

Cephalopod neural networks

Cephalopods have arguably the largest and most complex nervous systems amongst the invertebrates; but despite the squid giant axon being one of the best studied nerve cells in neuroscience, and the availability of superb information on the morphology of some cephalopod brains, there is surprisingly little known about the operation of the neural networks that underlie the sophisticated range of behaviour these animals display. This review focuses on a few of the best studied neural networks: the giant fiber system, the chromatophore system, the statocyst system, the visual system and the learning and memory system, with a view to summarizing our current knowledge and stimulating new studies, particularly on the activities of identified central neurons, to provide a more complete understanding of networks within the cephalopod nervous system.     

Entropic analysis and incremental synthesis of multilayered feedforward neural networks

Neural network architecture optimization is often a critical issue, particularly when VLSI implementation is considered. This paper proposes a new minimization method for multilayered feedforward ANNs and an original approach to their synthesis, both based on the analysis of the information quantity (entropy) flowing through the network. A layer is described as an information filter which selects the relevant characteristics until the complete classification is performed. The basic incremental synthesis method, including the supervised training procedure, is derived to design application-tailored neural paradigms with good generalization capability.     

Numerical bifurcation analysis of distance-dependent on-center off-surround shunting neural networks

 On-center off-surround shunting neural networks are often applied as models for content-addressable memory (CAM), the equilibria being the stored memories. One important demand of biological plausible CAMs is that they function under a broad range of parameters, since several parameters vary due to postnatal maturation or learning. Ellias, Cohen and Grossberg have put much effort into showing the stability properties of several configurations of on-center off-surround shunting neural networks. In this article we present numerical bifurcation analysis of distance-dependent on-center off-surround shunting neural networks with fixed external input. We varied four parameters that may be subject to postnatal maturation: the range of both excitatory and inhibitory connections and the strength of both inhibitory and excitatory connections. These analyses show that fold bifurcations occur in the equilibrium behavior of the network by variation of all four parameters. The most important result is that the number of activation peaks in the equilibrium behavior varies from one to many if the range of inhibitory connections is decreased. Moreover, under a broad range of the parameters the stability of the network is maintained. The examined network is implemented in an ART network, Exact ART, where it functions as the classification layer F2. The stability of the ART network with the F2-field in different dynamic regimes is maintained and the behavior is functional in Exact ART. Through a bifurcation the learning behavior of Exact ART may even change from forming local representations to forming distributed representations. Received: 23 January 1996 / Accepted in revised form: 1 July 1996     

Detection of heart murmurs using wavelet analysis and artificial neural networks

This paper presents the algorithm and technical aspects of an intelligent diagnostic system for the detection of heart murmurs. The purpose of this research is to address the lack of effectively accurate cardiac auscultation present at the primary care physician office by development of an algorithm capable of operating within the hectic environment of the primary care office. The proposed algorithm consists of three main stages. First; denoising of input data (digital recordings of heart sounds), via Wavelet Packet Analysis. Second; input vector preparation through the use of Principal Component Analysis and block processing. Third; classification of the heart sound using an Artificial Neural Network. Initial testing revealed the intelligent diagnostic system can differentiate between normal healthy heart sounds and abnormal heart sounds (e.g., murmurs), with a specificity of 70.5% and a sensitivity of 64.7%.     

Stability analysis of memristor-based fractional-order neural networks with different memductance functions

Cognitive Neurodynamics - In this paper, the problem of the existence, uniqueness and uniform stability of memristor-based fractional-order neural networks (MFNNs) with two different types of...     

Clustering with neural networks

Partitioning a set ofN patterns in ad-dimensional metric space intoK clusters — in a way that those in a given cluster are more similar to each other than the rest — is a problem of interest in many fields, such as, image analysis, taxonomy, astrophysics, etc. As there are approximatelyK ^N/K! possible ways of partitioning the patterns amongK clusters, finding the best solution is beyond exhaustive search whenN is large. We show that this problem, in spite of its exponential complexity, can be formulated as an optimization problem for which very good, but not necessarily optimal, solutions can be found by using a Hopfield model of neural networks. To obtain a very good solution, the network must start from many randomly selected initial states. The network is simulated on the MPP, a 128 × 128 SIMD array machine, where we use the massive parallelism not only in solving the differential equations that govern the evolution of the network, but also in starting the network from many initial states at once thus obtaining many solutions in one run. We achieve speedups of two to three orders of magnitude over serial implementations and the promise through Analog VLSI implementations of further speedups of three to six orders of magnitude.Supported by a National Research Council-NASA Research Associatship     

Properties of small neural networks

Networks containing neuronal models of the type considered in the previous paper can be described by a set of first order differential equations. Steady-state solutions and the stability of these solutions to small perturbations can be obtained. Networks of physiological interest which give rise to second, third and fourth order linear equations are analysed in detail. Conditions are derived under which such networks can be condensed into a single neuron of similar order. Simple mechanisms for memory storage, for the generation of oscillatory activity and for decision making in neural systems are suggested.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

Synthesis of reverberating neural networks

In the first Part explicit methods are given, following the work of Refs. [1–3], for the design of networks whose reverberations cannot exceed prefixed periods no matter how coefficients are changed, as well as of networks obeying pre-assigned constants of motion. In the second Part the role of coupling strengths in determining cyclic behaviors is investigated and shown to lead to new methods for the design of reverberating networks.     

Coherency and connectivity in oscillating neural networks: linear partialization analysis

 This paper studies the relation between the functional synaptic connections between two artificial neural networks and the correlation of their spiking activities. The model neurons had realistic non-oscillatory dynamic properties and the networks showed oscillatory behavior as a result of their internal synaptic connectivity. We found that both excitation and inhibition cause phase locking of the oscillating activities. When the two networks excite each other the oscillations synchronize with zero phase lag, whereas mutual inhibition between the networks resulted in an anti-phase (half period phase difference) synchronization. Correlations between the activities of the two networks can also be caused by correlated external inputs driving the systems (common input). Our analysis shows that when the networks exhibit oscillatory behavior and the rate of the common input is smaller than a characteristic network oscillator frequency, the cross-correlation functions between the activities of two systems still carry information about the mutual synaptic connectivity. This information can be retrieved with linear partialization, removing the influence of the common input. We further explored the network responses to periodic external input. We found that when the input is of a frequency smaller than a certain threshold, the network responds with bursts at the same frequency as the input. Above the threshold, the network responds with a fraction of the input frequency. This frequency threshold, characterizing the oscillatory properties of the network, is also found to determine the limit to which linear partialization works. Received: 20 October 1995 / Accepted in revised form: 20 May 1996     

Fermentation seed quality analysis with self‐organising neural networks

Industrial fermentation processes operate under well defined operating conditions to attempt to minimise production variability. Variability occurs for many reasons but a long held belief is that variation in the state of the seed is highly influential. In this paper a seed stage (a batch process) of an industrial antibiotic fermentation is considered and the performance of the main production fermentations is correlated with the quality of the seed using an unsupervised Kohonen self‐organising feature map (SOM). It is shown that using only seed information poor performance in the final stage fermentations can be predicted. Data from industrial penicillin G fermenters is used to demonstrate the procedure. © 1999 John Wiley & Sons, Inc. Biotechnol Bioeng 64: 82–91, 1999.     

Artificial neural networks and decision tree model analysis of liver cancer proteomes

Hepatocellular carcinoma (HCC) is a heterogeneous cancer and usually diagnosed at late advanced tumor stages of high lethality. The present study attempted to obtain a proteome-wide analysis of HCC in comparison with adjacent non-tumor liver tissues, in order to facilitate biomarkers' discovery and to investigate the mechanisms of HCC development. A cohort of 66 Chinese patients with HCC was included for proteomic profiling study by two-dimensional gel electrophoresis (2-DE) analysis. Artificial neural network (ANN) and decision tree (CART) data-mining methods were employed to analyze the profiling data and to delineate significant patterns and trends for discriminating HCC from non-malignant liver tissues. Protein markers were identified by tandem MS/MS. A total of 132 proteome datasets were generated by 2-DE expression profiling analysis, and each with 230 consolidated protein expression intensities. Both the data-mining algorithms successfully distinguished the HCC phenotype from other non-malignant liver samples. The detection sensitivity and specificity of ANN were 96.97% and 87.88%, while those of CART were 81.82% and 78.79%, respectively. The three biological classifiers in the CART model were identified as cytochrome b5, heat shock 70 kDa protein 8 isoform 2, and cathepsin B. The 2-DE-based proteomic profiling approach combined with the ANN or CART algorithm yielded satisfactory performance on identifying HCC and revealed potential candidate cancer biomarkers.     

Stability and synchronization analysis of inertial memristive neural networks with time delays

This paper is concerned with the problem of stability and pinning synchronization of a class of inertial memristive neural networks with time delay. In contrast to general inertial neural networks, inertial memristive neural networks is applied to exhibit the synchronization and stability behaviors due to the physical properties of memristors and the differential inclusion theory. By choosing an appropriate variable transmission, the original system can be transformed into first order differential equations. Then, several sufficient conditions for the stability of inertial memristive neural networks by using matrix measure and Halanay inequality are derived. These obtained criteria are capable of reducing computational burden in the theoretical part. In addition, the evaluation is done on pinning synchronization for an array of linearly coupled inertial memristive neural networks, to derive the condition using matrix measure strategy. Finally, the two numerical simulations are presented to show the effectiveness of acquired theoretical results.     

Pruning artificial neural networks using neural complexity measures

This paper describes a new method for pruning artificial neural networks, using a measure of the neural complexity of the neural network. This measure is used to determine the connections that should be pruned. The measure computes the information-theoretic complexity of a neural network, which is similar to, yet different from previous research on pruning. The method proposed here shows how overly large and complex networks can be reduced in size, whilst retaining learnt behaviour and fitness. The technique proposed here helps to discover a network topology that matches the complexity of the problem it is meant to solve. This novel pruning technique is tested in a robot control domain, simulating a racecar. It is shown, that the proposed pruning method is a significant improvement over the most commonly used pruning method Magnitude Based Pruning. Furthermore, some of the pruned networks prove to be faster learners than the benchmark network that they originate from. This means that this pruning method can also help to unleash hidden potential in a network, because the learning time decreases substantially for a pruned a network, due to the reduction of dimensionality of the network.     

Molecular networks as a sub-neural factor of neural networks

We describe a new approach in the research of neural networks. This research is based on molecular networks in the neuron. If we use molecular networks as a sub-neuron factor of neural networks, it is a more realistic approach than today's concepts in this new computer technology field, because the artificial neural activity profile is similar to the profile of the action potential in the natural neuron. The molecular networks approach can be used in three technologies: neurocomputer, neurochip and molecular chip. This means that molecular networks open new fields of science and engineering called molecular-like machines and molecular machines.