Abstractive morphological learning with a recurrent neural network

In traditional word-and-paradigm models of morphology, an inflectional system is represented via a set of exemplary paradigms. Novel wordforms are produced by analogy with previously encountered forms. This paper describes a recurrent neural network which can use this strategy to learn the paradigms of a morphologically complex language based on incomplete and randomized input. Results are given which show good performance for a range of typologically diverse languages.     

On a recurrent neural network producing oscillations

A recurrent two-node neural network producing oscillations is analyzed. The network has no true inputs and the outputs from the network exhibit a circular phase portrait. The weight configuration of the network is investigated, resulting in analytical weight expressions, which are compared with numerical weight estimates obtained by training the network on the desired trajectories. The values predicted by the analytical expressions agree well with the findings from the numerical study, and can also explain the asymptotic properties of the networks studied.     

Pattern storage in a sparsely coded neural network with cyclic activation

We investigate an artificial neural network model with a modified Hebb rule. It is an auto-associative neural network similar to the Hopfield model and to the Willshaw model. It has properties of both of these models. Another property is that the patterns are sparsely coded and are stored in cycles of synchronous neural activities. The cycles of activity for some ranges of parameter increase the capacity of the model. We discuss basic properties of the model and some of the implementation issues, namely optimizing of the algorithms. We describe the modification of the Hebb learning rule, the learning algorithm, the generation of patterns, decomposition of patterns into cycles and pattern recall.     

Software reliability prediction using recurrent neural network with Bayesian regularization

A recurrent neural network modeling approach for software reliability prediction with respect to cumulative failure time is proposed. Our proposed network structure has the capability of learning and recognizing the inherent internal temporal property of cumulative failure time sequence. Further, by adding a penalty term of sum of network connection weights, Bayesian regularization is applied to our network training scheme to improve the generalization capability and lower the susceptibility of overfitting. The performance of our proposed approach has been tested using four real-time control and flight dynamic application data sets. Numerical results show that our proposed approach is robust across different software projects, and has a better performance with respect to both goodness-of-fit and next-step-predictability compared to existing neural network models for failure time prediction.     

Training simultaneous recurrent neural network with resilient propagation for static optimization

This paper proposes a non-recurrent training algorithm, resilient propagation, for the Simultaneous Recurrent Neural network operating in relaxation-mode for computing high quality solutions of static optimization problems. Implementation details related to adaptation of the recurrent neural network weights through the non-recurrent training algorithm, resilient backpropagation, are formulated through an algebraic approach. Performance of the proposed neuro-optimizer on a well-known static combinatorial optimization problem, the Traveling Salesman Problem, is evaluated on the basis of computational complexity measures and, subsequently, compared to performance of the Simultaneous Recurrent Neural network trained with the standard backpropagation, and recurrent backpropagation for the same static optimization problem. Simulation results indicate that the Simultaneous Recurrent Neural network trained with the resilient backpropagation algorithm is able to locate superior quality solutions through comparable amount of computational effort for the Traveling Salesman Problem.     

Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition

Experimental evidence suggests that the maintenance of an item in working memory is achieved through persistent activity in selective neural assemblies of the cortex. To understand the mechanisms underlying this phenomenon, it is essential to investigate how persistent activity is affected by external inputs or neuromodulation. We have addressed these questions using a recurrent network model of object working memory. Recurrence is dominated by inhibition, although persistent activity is generated through recurrent excitation in small subsets of excitatory neurons.Our main findings are as follows. (1) Because of the strong feedback inhibition, persistent activity shows an inverted U shape as a function of increased external drive to the network. (2) A transient external excitation can switch off a network from a selective persistent state to its spontaneous state. (3) The maintenance of the sample stimulus in working memory is not affected by intervening stimuli (distractors) during the delay period, provided the stimulation intensity is not large. On the other hand, if stimulation intensity is large enough, distractors disrupt sample-related persistent activity, and the network is able to maintain a memory only of the last shown stimulus. (4) A concerted modulation of GABA _A and NMDA conductances leads to a decrease of spontaneous activity but an increase of persistent activity; the enhanced signal-to-noise ratio is shown to increase the resistance of the network to distractors. (5) Two mechanisms are identified that produce an inverted U shaped dependence of persistent activity on modulation. The present study therefore points to several mechanisms that enhance the signal-to-noise ratio in working memory states. These mechanisms could be implemented in the prefrontal cortex by dopaminergic projections from the midbrain.     

Emergence of clusters in the hidden layer of a dynamic recurrent neural network

The neural integrator of the oculomotor system is a privileged field for artificial neural network simulation. In this paper, we were interested in an improvement of the biologically plausible features of the Arnold-Robinson network. This improvement was done by fixing the sign of the connection weights in the network (in order to respect the biological Dale's Law). We also introduced a notion of distance in the network in the form of transmission delays between its units. These modifications necessitated the introduction of a general supervisor in order to train the network to act as a leaky integrator. When examining the lateral connection weights of the hidden layer, the distribution of the weights values was found to exhibit a conspicuous structure: the high-value weights were grouped in what we call clusters. Other zones are quite flat and characterized by low-value weights. Clusters are defined as particular groups of adjoining neurons which have strong and privileged connections with another neighborhood of neurons. The clusters of the trained network are reminiscent of the small clusters or patches that have been found experimentally in the nucleus prepositus hypoglossi, where the neural integrator is located. A study was conducted to determine the conditions of emergence of these clusters in our network: they include the fixation of the weight sign, the introduction of a distance, and a convergence of the information from the hidden layer to the motoneurons. We conclude that this spontaneous emergence of clusters in artificial neural networks, performing a temporal integration, is due to computational constraints, with a restricted space of solutions. Thus, information processing could induce the emergence of iterated patterns in biological neural networks. Received: 18 September 1996 / Accepted in revised form: 7 January 1997     

Coding the presence of visual objects in a recurrent neural network of visual cortex

Before we can recognize a visual object, our visual system has to segregate it from its background. This requires a fast mechanism for establishing the presence and location of objects independently of their identity. Recently, border-ownership neurons were recorded in monkey visual cortex which might be involved in this task [Zhou, H., Friedmann, H., von der Heydt, R., 2000. Coding of border ownership in monkey visual cortex. J. Neurosci. 20 (17), 6594-6611]. In order to explain the basic mechanisms required for fast coding of object presence, we have developed a neural network model of visual cortex consisting of three stages. Feed-forward and lateral connections support coding of Gestalt properties, including similarity, good continuation, and convexity. Neurons of the highest area respond to the presence of an object and encode its position, invariant of its form. Feedback connections to the lowest area facilitate orientation detectors activated by contours belonging to potential objects, and thus generate the experimentally observed border-ownership property. This feedback control acts fast and significantly improves the figure-ground segregation required for the consecutive task of object recognition.     

Aminergic and peptidergic amplification of intracellular cyclic AMP levels in a molluscan neural network

1. Serotonin (5-hydroxytryptamine; 5-HT), dopamine (DA), and small cardioactive peptide B (SCPB) can activate adenylate cyclase and increase the intracellular cyclic AMP (cAMP) levels in the Limax procerebrum (PC), with differing time courses and to differing extents. 5-HT and SCPB are potent stimulators of adenylate cyclase, and when both were applied simultaneously, an additive effect was observed. 2. In contrast, DA shows a great variability in the time course of cAMP synthesis and is a weak stimulator. Ergonovine, a DA antagonist, failed to inhibit cyclase activation, indicating that ergonovine-sensitive receptors are absent or ergonovine-sensitive DA receptors are not coupled to adenylate cyclase. 3. 5-HT and SCPB cause a rapid synthesis of cAMP, reaching the maximum 20- to 30-fold increase within a minute. DA's effect is slow in onset and very prolonged, reaching a maximum of only a two- to three-fold increase in the cAMP level. Reasons for variability in DA action are discussed.     

Inference of genetic regulatory networks with recurrent neural network models using particle swarm optimization

Genetic regulatory network inference is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. The availability of time series gene expression data makes it possible to investigate the gene activities of whole genomes, rather than those of only a pair of genes or among several genes. However, current computational methods do not sufficiently consider the temporal behavior of this type of data and lack the capability to capture the complex nonlinear system dynamics. We propose a recurrent neural network (RNN) and particle swarm optimization (PSO) approach to infer genetic regulatory networks from time series gene expression data. Under this framework, gene interaction is explained through a connection weight matrix. Based on the fact that the measured time points are limited and the assumption that the genetic networks are usually sparsely connected, we present a PSO-based search algorithm to unveil potential genetic network constructions that fit well with the time series data and explore possible gene interactions. Furthermore, PSO is used to train the RNN and determine the network parameters. Our approach has been applied to both synthetic and real data sets. The results demonstrate that the RNN/PSO can provide meaningful insights in understanding the nonlinear dynamics of the gene expression time series and revealing potential regulatory interactions between genes.     

Circadian locomotory rhythm and the influence of moulting in Australian field cricket nymphs

ABSTRACT. Evidence is presented for a circadian control of locomotory activity in the larval stadia of the cricket, Teleogryllus commodus Walker. Under light—dark cycles (LD), maximal activity occurs around the L/D transition and/or in the hours preceding it. Free-running rhythm patterns longer than 24 h are observed in constant light. Re-entrainment to phase advances in the LD cycle is also accompanied by several transient cycles. However, free-running rhythms under constant darkness or transients when exposed to LD cycle delays were not found. LD cycles during the eighth stadium set the phase of a free-running rhythm in the adult, even if the nymph does not show a rhythm. Nymphal activity is often erratic and is disrupted periodically by the moulting cycle, but moulting does not interrupt the operation of the circadian system. The daily timing of the moult itself is not under circadian control.     

Fitting of dynamic recurrent neural network models to sensory stimulus-response data

We present a theoretical study aiming at model fitting for sensory neurons. Conventional neural network training approaches are not applicable to this problem due to lack of continuous data. Although the stimulus can be considered as a smooth time-dependent variable, the associated response will be a set of neural spike timings (roughly the instants of successive action potential peaks) that have no amplitude information. A recurrent neural network model can be fitted to such a stimulus-response data pair by using the maximum likelihood estimation method where the likelihood function is derived from Poisson statistics of neural spiking. The universal approximation feature of the recurrent dynamical neuron network models allows us to describe excitatory-inhibitory characteristics of an actual sensory neural network with any desired number of neurons. The stimulus data are generated by a phased cosine Fourier series having a fixed amplitude and frequency but a randomly shot phase. Various values of amplitude, stimulus component size, and sample size are applied in order to examine the effect of the stimulus to the identification process. Results are presented in tabular and graphical forms at the end of this text. In addition, to demonstrate the success of this research, a study involving the same model, nominal parameters and stimulus structure, and another study that works on different models are compared to that of this research.     

Novel tracking function of moving target using chaotic dynamics in a recurrent neural network model

Chaotic dynamics introduced in a recurrent neural network model is applied to controlling an object to track a moving target in two-dimensional space, which is set as an ill-posed problem. The motion increments of the object are determined by a group of motion functions calculated in real time with firing states of the neurons in the network. Several cyclic memory attractors that correspond to several simple motions of the object in two-dimensional space are embedded. Chaotic dynamics introduced in the network causes corresponding complex motions of the object in two-dimensional space. Adaptively real-time switching of control parameter results in constrained chaos (chaotic itinerancy) in the state space of the network and enables the object to track a moving target along a certain trajectory successfully. The performance of tracking is evaluated by calculating the success rate over 100 trials with respect to nine kinds of trajectories along which the target moves respectively. Computer experiments show that chaotic dynamics is useful to track a moving target. To understand the relations between these cases and chaotic dynamics, dynamical structure of chaotic dynamics is investigated from dynamical viewpoint.     

Glycosidase inhibition by cyclic sulfonium compounds

Inhibitory activities of various cyclic sulfonium compounds including salacinol against several glycosidases were studied and some compounds showed significant inhibition. The sulfonium ion structure was found to be essential for the inhibitory activity. Specific inhibition of salacinol toward rice alpha-glucosidase was ascribed to the tether arm.     

Validation of protein models by a neural network approach

Background  

The development and improvement of reliable computational methods designed to evaluate the quality of protein models is relevant in the context of protein structure refinement, which has been recently identified as one of the bottlenecks limiting the quality and usefulness of protein structure prediction.     

Prediction of continuous B-cell epitopes in an antigen using recurrent neural network

B-cell epitopes play a vital role in the development of peptide vaccines, in diagnosis of diseases, and also for allergy research. Experimental methods used for characterizing epitopes are time consuming and demand large resources. The availability of epitope prediction method(s) can rapidly aid experimenters in simplifying this problem. The standard feed-forward (FNN) and recurrent neural network (RNN) have been used in this study for predicting B-cell epitopes in an antigenic sequence. The networks have been trained and tested on a clean data set, which consists of 700 non-redundant B-cell epitopes obtained from Bcipep database and equal number of non-epitopes obtained randomly from Swiss-Prot database. The networks have been trained and tested at different input window length and hidden units. Maximum accuracy has been obtained using recurrent neural network (Jordan network) with a single hidden layer of 35 hidden units for window length of 16. The final network yields an overall prediction accuracy of 65.93% when tested by fivefold cross-validation. The corresponding sensitivity, specificity, and positive prediction values are 67.14, 64.71, and 65.61%, respectively. It has been observed that RNN (JE) was more successful than FNN in the prediction of B-cell epitopes. The length of the peptide is also important in the prediction of B-cell epitopes from antigenic sequences. The webserver ABCpred is freely available at www.imtech.res.in/raghava/abcpred/.