The complexity of linkage analysis with neural networks

As the focus of genome-wide scans for disease loci have shifted from simple Mendelian traits to genetically complex traits, researchers have begun to consider new alternative ways to detect linkage that will consider more than the marginal effects of a single disease locus at a time. One interesting new method is to train a neural network on a genome-wide data set in order to search for the best non-linear relationship between identity-by-descent sharing among affected siblings at markers and their disease status. We investigate here the repeatability of the neural network results from run to run, and show that the results obtained by multiple runs of the neural network method may differ quite a bit. This is most likely due to the fact that training a neural network involves minimizing an error function with a multitude of local minima.     

Artificial neural networks for solution scattering data analysis

1. Download : Download high-res image (142KB)
2. Download : Download full-size image

     

Stability analysis of higher-order neural networks for combinatorial optimization

Recurrent neural networks with higher order connections, from here on referred to as higher-order neural networks (HONNs), may be used for the solution of combinatorial optimization problems. In Ref. 5 a mapping of the traveling salesman problem (TSP) onto a HONN of arbitrary order was developed, thereby creating a family of related networks that can be used to solve the TSP. In this paper, we explore the trade-off between network complexity and quality of solution that is made available by the HONN mapping of the TSP. The trade-off is investigated by undertaking an analysis of the stability of valid solutions to the TSP in a HONN of arbitrary order. The techniques used to perform the stability analysis are not new, but have been widely used elsewhere in the literature. The original contribution in this paper is the application of these techniques to a HONN of arbitrary order used to solve the TSP. The results of the stability analysis show that the quality of solution is improved by increasing the network complexity, as measured by the order of the network. Furthermore, it is shown that the Hopfield network, as the simplest network in the family of higher-order networks, is expected to produce the poorest quality of solution.     

The applicability of recurrent neural networks for biological sequence analysis

Selection of machine learning techniques requires a certain sensitivity to the requirements of the problem. In particular, the problem can be made more tractable by deliberately using algorithms that are biased toward solutions of the requisite kind. In this paper, we argue that recurrent neural networks have a natural bias toward a problem domain of which biological sequence analysis tasks are a subset. We use experiments with synthetic data to illustrate this bias. We then demonstrate that this bias can be exploitable using a data set of protein sequences containing several classes of subcellular localization targeting peptides. The results show that, compared with feed forward, recurrent neural networks will generally perform better on sequence analysis tasks. Furthermore, as the patterns within the sequence become more ambiguous, the choice of specific recurrent architecture becomes more critical.     

Entropic analysis and incremental synthesis of multilayered feedforward neural networks

Neural network architecture optimization is often a critical issue, particularly when VLSI implementation is considered. This paper proposes a new minimization method for multilayered feedforward ANNs and an original approach to their synthesis, both based on the analysis of the information quantity (entropy) flowing through the network. A layer is described as an information filter which selects the relevant characteristics until the complete classification is performed. The basic incremental synthesis method, including the supervised training procedure, is derived to design application-tailored neural paradigms with good generalization capability.     

Numerical bifurcation analysis of distance-dependent on-center off-surround shunting neural networks

 On-center off-surround shunting neural networks are often applied as models for content-addressable memory (CAM), the equilibria being the stored memories. One important demand of biological plausible CAMs is that they function under a broad range of parameters, since several parameters vary due to postnatal maturation or learning. Ellias, Cohen and Grossberg have put much effort into showing the stability properties of several configurations of on-center off-surround shunting neural networks. In this article we present numerical bifurcation analysis of distance-dependent on-center off-surround shunting neural networks with fixed external input. We varied four parameters that may be subject to postnatal maturation: the range of both excitatory and inhibitory connections and the strength of both inhibitory and excitatory connections. These analyses show that fold bifurcations occur in the equilibrium behavior of the network by variation of all four parameters. The most important result is that the number of activation peaks in the equilibrium behavior varies from one to many if the range of inhibitory connections is decreased. Moreover, under a broad range of the parameters the stability of the network is maintained. The examined network is implemented in an ART network, Exact ART, where it functions as the classification layer F2. The stability of the ART network with the F2-field in different dynamic regimes is maintained and the behavior is functional in Exact ART. Through a bifurcation the learning behavior of Exact ART may even change from forming local representations to forming distributed representations. Received: 23 January 1996 / Accepted in revised form: 1 July 1996     

Markovian neural networks

The neural network that efficiently and nearly optimally solves difficult optimization problems is defined. The convergence proof for the Markovian neural network that asynchronously updates its neurons' states is also presented. The comparison of the performance of the Markovian neural network with various combinatorial optimization methods in two domains is described. The Markovian neural network is shown to be an efficient tool for solving optimization problems.     

Cephalopod neural networks

Cephalopods have arguably the largest and most complex nervous systems amongst the invertebrates; but despite the squid giant axon being one of the best studied nerve cells in neuroscience, and the availability of superb information on the morphology of some cephalopod brains, there is surprisingly little known about the operation of the neural networks that underlie the sophisticated range of behaviour these animals display. This review focuses on a few of the best studied neural networks: the giant fiber system, the chromatophore system, the statocyst system, the visual system and the learning and memory system, with a view to summarizing our current knowledge and stimulating new studies, particularly on the activities of identified central neurons, to provide a more complete understanding of networks within the cephalopod nervous system.     

LMI-based asymptotic stability analysis of neural networks with time-varying delays

The problem of the global asymptotic stability for a class of neural networks with time-varying delays is investigated in this paper, where the activation functions are assumed to be neither monotonic, nor differentiable, nor bounded. By constructing suitable Lyapunov functionals and combining with linear matrix inequality (LMI) technique, new global asymptotic stability criteria about different types of time-varying delays are obtained. It is shown that the criteria can provide less conservative result than some existing ones. Numerical examples are given to demonstrate the applicability of the proposed approach.     

Bayesian neural networks

A neural network that uses the basic Hebbian learning rule and the Bayesian combination function is defined. Analogously to Hopfield's neural network, the convergence for the Bayesian neural network that asynchronously updates its neurons' states is proved. The performance of the Bayesian neural network in four medical domains is compared with various classification methods. The Bayesian neural network uses more sophisticated combination function than Hopfield's neural network and uses more economically the available information. The naive Bayesian classifier typically outperforms the basic Bayesian neural network since iterations in network make too many mistakes. By restricting the number of iterations and increasing the number of fixed points the network performs better than the naive Bayesian classifier. The Bayesian neural network is designed to learn very quickly and incrementally.     

Detection of heart murmurs using wavelet analysis and artificial neural networks

This paper presents the algorithm and technical aspects of an intelligent diagnostic system for the detection of heart murmurs. The purpose of this research is to address the lack of effectively accurate cardiac auscultation present at the primary care physician office by development of an algorithm capable of operating within the hectic environment of the primary care office. The proposed algorithm consists of three main stages. First; denoising of input data (digital recordings of heart sounds), via Wavelet Packet Analysis. Second; input vector preparation through the use of Principal Component Analysis and block processing. Third; classification of the heart sound using an Artificial Neural Network. Initial testing revealed the intelligent diagnostic system can differentiate between normal healthy heart sounds and abnormal heart sounds (e.g., murmurs), with a specificity of 70.5% and a sensitivity of 64.7%.     

Stability analysis of memristor-based fractional-order neural networks with different memductance functions

Cognitive Neurodynamics - In this paper, the problem of the existence, uniqueness and uniform stability of memristor-based fractional-order neural networks (MFNNs) with two different types of...     

Clustering with neural networks

Partitioning a set ofN patterns in ad-dimensional metric space intoK clusters — in a way that those in a given cluster are more similar to each other than the rest — is a problem of interest in many fields, such as, image analysis, taxonomy, astrophysics, etc. As there are approximatelyK ^N/K! possible ways of partitioning the patterns amongK clusters, finding the best solution is beyond exhaustive search whenN is large. We show that this problem, in spite of its exponential complexity, can be formulated as an optimization problem for which very good, but not necessarily optimal, solutions can be found by using a Hopfield model of neural networks. To obtain a very good solution, the network must start from many randomly selected initial states. The network is simulated on the MPP, a 128 × 128 SIMD array machine, where we use the massive parallelism not only in solving the differential equations that govern the evolution of the network, but also in starting the network from many initial states at once thus obtaining many solutions in one run. We achieve speedups of two to three orders of magnitude over serial implementations and the promise through Analog VLSI implementations of further speedups of three to six orders of magnitude.Supported by a National Research Council-NASA Research Associatship     

Synthesis of reverberating neural networks

In the first Part explicit methods are given, following the work of Refs. [1–3], for the design of networks whose reverberations cannot exceed prefixed periods no matter how coefficients are changed, as well as of networks obeying pre-assigned constants of motion. In the second Part the role of coupling strengths in determining cyclic behaviors is investigated and shown to lead to new methods for the design of reverberating networks.     

Properties of small neural networks

Networks containing neuronal models of the type considered in the previous paper can be described by a set of first order differential equations. Steady-state solutions and the stability of these solutions to small perturbations can be obtained. Networks of physiological interest which give rise to second, third and fourth order linear equations are analysed in detail. Conditions are derived under which such networks can be condensed into a single neuron of similar order. Simple mechanisms for memory storage, for the generation of oscillatory activity and for decision making in neural systems are suggested.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

VLSI implementation of neural networks

Currently, fuzzy controllers are the most popular choice for hardware implementation of complex control surfaces because they are easy to design. Neural controllers are more complex and hard to train, but provide an outstanding control surface with much less error than that of a fuzzy controller. There are also some problems that have to be solved before the networks can be implemented on VLSI chips. First, an approximation function needs to be developed because CMOS neural networks have an activation function different than any function used in neural network software. Next, this function has to be used to train the network. Finally, the last problem for VLSI designers is the quantization effect caused by discrete values of the channel length (L) and width (W) of MOS transistor geometries. Two neural networks were designed in 1.5 microm technology. Using adequate approximation functions solved the problem of activation function. With this approach, trained networks were characterized by very small errors. Unfortunately, when the weights were quantized, errors were increased by an order of magnitude. However, even though the errors were enlarged, the results obtained from neural network hardware implementations were superior to the results obtained with fuzzy system approach.     

Fermentation seed quality analysis with self‐organising neural networks

Industrial fermentation processes operate under well defined operating conditions to attempt to minimise production variability. Variability occurs for many reasons but a long held belief is that variation in the state of the seed is highly influential. In this paper a seed stage (a batch process) of an industrial antibiotic fermentation is considered and the performance of the main production fermentations is correlated with the quality of the seed using an unsupervised Kohonen self‐organising feature map (SOM). It is shown that using only seed information poor performance in the final stage fermentations can be predicted. Data from industrial penicillin G fermenters is used to demonstrate the procedure. © 1999 John Wiley & Sons, Inc. Biotechnol Bioeng 64: 82–91, 1999.     

Coherency and connectivity in oscillating neural networks: linear partialization analysis

 This paper studies the relation between the functional synaptic connections between two artificial neural networks and the correlation of their spiking activities. The model neurons had realistic non-oscillatory dynamic properties and the networks showed oscillatory behavior as a result of their internal synaptic connectivity. We found that both excitation and inhibition cause phase locking of the oscillating activities. When the two networks excite each other the oscillations synchronize with zero phase lag, whereas mutual inhibition between the networks resulted in an anti-phase (half period phase difference) synchronization. Correlations between the activities of the two networks can also be caused by correlated external inputs driving the systems (common input). Our analysis shows that when the networks exhibit oscillatory behavior and the rate of the common input is smaller than a characteristic network oscillator frequency, the cross-correlation functions between the activities of two systems still carry information about the mutual synaptic connectivity. This information can be retrieved with linear partialization, removing the influence of the common input. We further explored the network responses to periodic external input. We found that when the input is of a frequency smaller than a certain threshold, the network responds with bursts at the same frequency as the input. Above the threshold, the network responds with a fraction of the input frequency. This frequency threshold, characterizing the oscillatory properties of the network, is also found to determine the limit to which linear partialization works. Received: 20 October 1995 / Accepted in revised form: 20 May 1996