Application of chaotic dynamics in a recurrent neural network to control: hardware implementation into a novel autonomous roving robot

Originating from a viewpoint that complex/chaotic dynamics would play an important role in biological system including brains, chaotic dynamics introduced in a recurrent neural network was applied to control. The results of computer experiment was successfully implemented into a novel autonomous roving robot, which can only catch rough target information with uncertainty by a few sensors. It was employed to solve practical two-dimensional mazes using adaptive neural dynamics generated by the recurrent neural network in which four prototype simple motions are embedded. Adaptive switching of a system parameter in the neural network results in stationary motion or chaotic motion depending on dynamical situations. The results of hardware implementation and practical experiment using it show that, in given two-dimensional mazes, the robot can successfully avoid obstacles and reach the target. Therefore, we believe that chaotic dynamics has novel potential capability in controlling, and could be utilized to practical engineering application.     

Recognition of culture state using two-dimensional gel electrophoresis with an artificial neural network

Proteomic technologies were applied to the examination of nutrient components in culture broth. In bioprocesses, many types of media have been proposed and used on the commercial scale. Natural nutrients, the chemical components of which cannot be identified completely, are often used in fermentation processes such as in the production of baker's yeast, alcoholic beverages, amino acids, and pharmaceuticals. The catabolic activities of the microorganisms in these processes vary with the species used. We used an artificial neural network (ANN) to recognize the sufficiency of chemical elements based on the protein spots resolved in 2-DE, and we evaluated this technique using the leave-one-out method. We also attempted to reduce the number of input data for spot selection based on sensitivity analysis of the ANN, and the selected data were used to improve accuracy.     

A prototype neural network supervised control system for Bacillus thuringiensis fermentations

This article discusses the development of a prototype neural network-based supervisory control system for Bacillus thuringiensis fermentations. The input pattern to the neural network included the type of inoculum, operation temperature, pH value, accumulated process time, optical density in fermentation medium, and change in optical density. The output from the neural network was the predicted optical density for the next sampling time. The control system has been implemented in both a computer simulation and a laboratory fermentation experiment with promising results. (c) 1994 John Wiley & Sons, Inc.     

ProDCoNN: Protein design using a convolutional neural network

Designing protein sequences that fold to a given three-dimensional (3D) structure has long been a challenging problem in computational structural biology with significant theoretical and practical implications. In this study, we first formulated this problem as predicting the residue type given the 3D structural environment around the C _α atom of a residue, which is repeated for each residue of a protein. We designed a nine-layer 3D deep convolutional neural network (CNN) that takes as input a gridded box with the atomic coordinates and types around a residue. Several CNN layers were designed to capture structure information at different scales, such as bond lengths, bond angles, torsion angles, and secondary structures. Trained on a very large number of protein structures, the method, called ProDCoNN (protein design with CNN), achieved state-of-the-art performance when tested on large numbers of test proteins and benchmark datasets.     

Rules extraction from neural networks applied to the prediction and recognition of prokaryotic promoters

Promoters are DNA sequences located upstream of the gene region and play a central role in gene expression. Computational techniques show good accuracy in gene prediction but are less successful in predicting promoters, primarily because of the high number of false positives that reflect characteristics of the promoter sequences. Many machine learning methods have been used to address this issue. Neural Networks (NN) have been successfully used in this field because of their ability to recognize imprecise and incomplete patterns characteristic of promoter sequences. In this paper, NN was used to predict and recognize promoter sequences in two data sets: (i) one based on nucleotide sequence information and (ii) another based on stability sequence information. The accuracy was approximately 80% for simulation (i) and 68% for simulation (ii). In the rules extracted, biological consensus motifs were important parts of the NN learning process in both simulations.     

Extrapolating a hierarchy of building block systems towards future neural network organisms

Is it possible to predict future life forms? In this paper it is argued that the answer to this question may well be positive. As a basis for predictions a rationale is used that is derived from historical data, e.g. from a hierarchical classification that ranks all building block systems, that have evolved so far. This classification is based on specific emergent properties that allow stepwise transitions, from low level building blocks to higher level ones. This paper shows how this hierarchy can be used for predicting future life forms. The extrapolations suggest several future neural network organisms. Major aspects of the structures of these organisms are predicted. The results can be considered of fundamental importance for several reasons. Firstly, assuming that the operator hierarchy is a proper basis for predictions, the result yields insight into the structure of future organisms. Secondly, the predictions are not extrapolations of presently observed trends, but are fully integrated with all historical system transitions in evolution. Thirdly, the extrapolations suggest the structures of intelligences that, one day, will possess more powerful brains than human beings. This study ends with a discussion of possibilities for falsification of the present theory, the implications of the present predictions in relation to recent developments in artificial intelligence and the philosophical implications of the role of humanity in evolution with regard to the creation of future neural network organisms.     

Numerically evaluated functional equivalence between chaotic dynamics in neural networks and cellular automata under totalistic rules

Chaotic dynamics in a recurrent neural network model and in two-dimensional cellular automata, where both have finite but large degrees of freedom, are investigated from the viewpoint of harnessing chaos and are applied to motion control to indicate that both have potential capabilities for complex function control by simple rule(s). An important point is that chaotic dynamics generated in these two systems give us autonomous complex pattern dynamics itinerating through intermediate state points between embedded patterns (attractors) in high-dimensional state space. An application of these chaotic dynamics to complex controlling is proposed based on an idea that with the use of simple adaptive switching between a weakly chaotic regime and a strongly chaotic regime, complex problems can be solved. As an actual example, a two-dimensional maze, where it should be noted that the spatial structure of the maze is one of typical ill-posed problems, is solved with the use of chaos in both systems. Our computer simulations show that the success rate over 300 trials is much better, at least, than that of a random number generator. Our functional simulations indicate that both systems are almost equivalent from the viewpoint of functional aspects based on our idea, harnessing of chaos.     

Analysis of reflex modulation with a biologically realistic neural network

In this study, a neuromusculoskeletal model was built to give insight into the mechanisms behind the modulation of reflexive feedback strength as experimentally identified in the human shoulder joint. The model is an integration of a biologically realistic neural network consisting of motoneurons and interneurons, modeling 12 populations of spinal neurons, and a one degree-of-freedom musculoskeletal model, including proprioceptors. The model could mimic the findings of human postural experiments, using presynaptic inhibition of the Ia afferents to modulate the feedback gains. In a pathological case, disabling one specific neural connection between the inhibitory interneurons and the motoneurons could mimic the experimental findings in complex regional pain syndrome patients. It is concluded that the model is a valuable tool to gain insight into the spinal contributions to human motor control. Applications lay in the fields of human motor control and neurological disorders, where hypotheses on motor dysfunction can be tested, like spasticity, clonus, and tremor. Action Editor: Karen Sigvardt     

An artificial neural network that utilizes hip joint actuations to control bifurcations and chaos in a passive dynamic bipedal walking model

Chaos is a central feature of human locomotion and has been suggested to be a window to the control mechanisms of locomotion. In this investigation, we explored how the principles of chaos can be used to control locomotion with a passive dynamic bipedal walking model that has a chaotic gait pattern. Our control scheme was based on the scientific evidence that slight perturbations to the unstable manifolds of points in a chaotic system will promote the transition to new stable behaviors embedded in the rich chaotic attractor. Here we demonstrate that hip joint actuations during the swing phase can provide such perturbations for the control of bifurcations and chaos in a locomotive pattern. Our simulations indicated that systematic alterations of the hip joint actuations resulted in rapid transitions to any stable locomotive pattern available in the chaotic locomotive attractor. Based on these insights, we further explored the benefits of having a chaotic gait with a biologically inspired artificial neural network (ANN) that employed this chaotic control scheme. Remarkably, the ANN was quite robust and capable of selecting a hip joint actuation that rapidly transitioned the passive dynamic bipedal model to a stable gait embedded in the chaotic attractor. Additionally, the ANN was capable of using hip joint actuations to accommodate unstable environments and to overcome unforeseen perturbations. Our simulations provide insight on the advantage of having a chaotic locomotive system and provide evidence as to how chaos can be used as an advantageous control scheme for the nervous system.     

Software sensing for glucose concentration in industrial antibiotic fedbatch culture using fuzzy neural network

In order to control glucose concentration during fed-batch culture for antibiotic production, we applied so called “software sensor” which estimates unmeasured variable of interest from measured process variables using software. All data for analysis were collected from industrial scale cultures in a pharmaceutical company. First, we constructed an estimation model for glucose feed rate to keep glucose concentration at target value. In actual fed-batch culture, glucose concentration, was kept at relatively high and measured once a day, and the glucose feed rate until the next measurement time was determined by an expert worker based on the actual consumption rate. Fuzzy neural network (FNN) was applied to construct the estimation model. From the simulation results using this model, the average error for glucose concentration was 0.88 g/L. The FNN model was also applied for a special culture to keep glucose concentration at low level. Selecting the optimal input variables, it was possible to simulate the culture with a low glucose concentration from the data sets of relatively high glucose concentration. Next, a simulation model to estimate time course of glucose concentration during one day was constructed using the on-line measurable process variables, since glucose concentration was only measured off-line once a day. Here, the recursive fuzzy neural network (RFNN) was applied for the simulation model. As the result of the simulation, average error of RFNN model was 0.91 g/L and this model was found to be useful to supervise the fed-batch culture.     

Inertial motion capture in conjunction with an artificial neural network can differentiate the gait patterns of hemiparetic stroke patients compared with able-bodied counterparts

Clinical gait analysis has proven to reduce uncertainties in selecting the appropriate quantity and type of treatment for patients with neuromuscular disorders. However, gait analysis as a clinical tool is under-utilised due to the limitations and cost of acquiring and managing data. To overcome these obstacles, inertial motion capture (IMC) recently emerged to counter the limitations attributed to other methods. This paper investigates the use of IMC for training and testing a back-propagation artificial neural network (ANN) for the purpose of distinguishing between hemiparetic stroke and able-bodied ambulation. Routine gait analysis was performed on 30 able-bodied control subjects and 28 hemiparetic stroke patients using an IMC system. An ANN was optimised to classify the two groups, achieving a repeatable network accuracy of 99.4%. It is concluded that an IMC system and appropriate computer methods may be useful for the planning and monitoring of gait rehabilitation therapy of stroke victims.     

CODEX,a neural network approach to explore signaling dynamics landscapes

Current studies of cell signaling dynamics that use live cell fluorescent biosensors routinely yield thousands of single‐cell, heterogeneous, multi‐dimensional trajectories. Typically, the extraction of relevant information from time series data relies on predefined, human‐interpretable features. Without a priori knowledge of the system, the predefined features may fail to cover the entire spectrum of dynamics. Here we present CODEX, a data‐driven approach based on convolutional neural networks (CNNs) that identifies patterns in time series. It does not require a priori information about the biological system and the insights into the data are built through explanations of the CNNs'' predictions. CODEX provides several views of the data: visualization of all the single‐cell trajectories in a low‐dimensional space, identification of prototypic trajectories, and extraction of distinctive motifs. We demonstrate how CODEX can provide new insights into ERK and Akt signaling in response to various growth factors, and we recapitulate findings in p53 and TGFβ‐SMAD2 signaling.     

Novel recurrent neural network for modelling biological networks: Oscillatory p53 interaction dynamics

Understanding the control of cellular networks consisting of gene and protein interactions and their emergent properties is a central activity of Systems Biology research. For this, continuous, discrete, hybrid, and stochastic methods have been proposed. Currently, the most common approach to modelling accurate temporal dynamics of networks is ordinary differential equations (ODE). However, critical limitations of ODE models are difficulty in kinetic parameter estimation and numerical solution of a large number of equations, making them more suited to smaller systems. In this article, we introduce a novel recurrent artificial neural network (RNN) that addresses above limitations and produces a continuous model that easily estimates parameters from data, can handle a large number of molecular interactions and quantifies temporal dynamics and emergent systems properties. This RNN is based on a system of ODEs representing molecular interactions in a signalling network. Each neuron represents concentration change of one molecule represented by an ODE. Weights of the RNN correspond to kinetic parameters in the system and can be adjusted incrementally during network training. The method is applied to the p53-Mdm2 oscillation system – a crucial component of the DNA damage response pathways activated by a damage signal. Simulation results indicate that the proposed RNN can successfully represent the behaviour of the p53-Mdm2 oscillation system and solve the parameter estimation problem with high accuracy. Furthermore, we presented a modified form of the RNN that estimates parameters and captures systems dynamics from sparse data collected over relatively large time steps. We also investigate the robustness of the p53-Mdm2 system using the trained RNN under various levels of parameter perturbation to gain a greater understanding of the control of the p53-Mdm2 system. Its outcomes on robustness are consistent with the current biological knowledge of this system. As more quantitative data become available on individual proteins, the RNN would be able to refine parameter estimation and mapping of temporal dynamics of individual signalling molecules as well as signalling networks as a system. Moreover, RNN can be used to modularise large signalling networks.     

Using a neural network and spatial clustering to predict the location of active sites in enzymes

Structural genomics projects aim to provide a sharp increase in the number of structures of functionally unannotated, and largely unstudied, proteins. Algorithms and tools capable of deriving information about the nature, and location, of functional sites within a structure are increasingly useful therefore. Here, a neural network is trained to identify the catalytic residues found in enzymes, based on an analysis of the structure and sequence. The neural network output, and spatial clustering of the highly scoring residues are then used to predict the location of the active site.A comparison of the performance of differently trained neural networks is presented that shows how information from sequence and structure come together to improve the prediction accuracy of the network. Spatial clustering of the network results provides a reliable way of finding likely active sites. In over 69% of the test cases the active site is correctly predicted, and a further 25% are partially correctly predicted. The failures are generally due to the poor quality of the automatically generated sequence alignments.We also present predictions identifying the active site, and potential functional residues in five recently solved enzyme structures, not used in developing the method. The method correctly identifies the putative active site in each case. In most cases the likely functional residues are identified correctly, as well as some potentially novel functional groups.     

Subnetwork identification and chemical modulation for neural regeneration: A study combining network guided forest and heat diffusion model

Background: The induction of neural regeneration is vital to the repair of spinal cord injury (SCI). While compared with peripheral nervous system (PNS), the regenerative capacity of the central nervous system (CNS) is extremely limited. This indicates that modulating the molecular pathways underlying PNS repair may lead to the discovery of potential treatment for CNS injury.Methods: Based on the gene expression profiles of dorsal root ganglion (DRG) after a sciatic nerve injury, we utilized network guided forest (NGF) to rank genes in terms of their capacity of distinguishing injured DRG from sham-operated controls. Gene importance scores deriving from NGF were used as initial heat in a heat diffusion model (HotNet2) to infer the subnetworks underlying neural regeneration in the DRG. After potential regulators of the subnetworks were found through Connectivity Map (cMap), candidate compounds were experimentally evaluated for their capacity to regenerate the damaged neurons.Results: Gene ontology analysis of the subnetworks revealed ubiquinone biosynthetic process is crucial for neural regeneration. Moreover, almost half of the genes in these subnetworks are found to be related to neural regeneration via text mining. After screening compounds that are likely to modulate gene expressions of the subnetworks, three compounds were selected for the experiment. Of them, trichostatin A, a histone deacetylase inhibitor, was validated to enhance neurite outgrowth in vivo via an optic nerve crush mouse model.Conclusions: Our study identified subnetworks underlying neural regeneration, and validated a compound can promote neurite outgrowth by modulating these subnetworks. This work also suggests an alternative approach for drug repositioning that can be easily extended to other disease phenotypes.     

A robust soft sensor to monitor 1,3-propanediol fermentation process by Clostridium butyricum based on artificial neural network

With the aggravation of environmental pollution and energy crisis, the sustainable microbial fermentation process of converting glycerol to 1,3-propanediol (1,3-PDO) has become an attractive alternative. However, the difficulty in the online measurement of glycerol and 1,3-PDO creates a barrier to the fermentation process and then leads to the residual glycerol and therefore, its wastage. Thus, in the present study, the four-input artificial neural network (ANN) model was developed successfully to predict the concentration of glycerol, 1,3-PDO, and biomass with high accuracy. Moreover, an ANN model combined with a kinetic model was also successfully developed to simulate the fed-batch fermentation process accurately. Hence, a soft sensor from the ANN model based on NaOH-related parameters has been successfully developed which cannot only be applied in software to solve the difficulty of glycerol and 1,3-PDO online measurement during the industrialization process, but also offer insight and reference for similar fermentation processes.     

A neural network prediction of environmental determinants of Anopheles sinensis knockdown resistance mutation to pyrethroids in China

Selection pressure caused by long‐term intensive use of insecticides is the key driving force in resistance development. Additional parameters such as environmental conditions may affect both the mosquito response to insecticides and the selection of resistance mechanisms. In this context, we analyzed the environmental determinants of kdr prevalence in Anopheles sinensis across China. We collected kdr frequency from 48 sites across central and southern China, together with key environmental factors including long‐term climatic data, topographic features, main crops, and land cover types. Trend surface analysis found that the distribution of kdr frequency can be partitioned into three regions, namely central China (kdr frequency >80%), western China (kdr frequency varies from 0% to 60%), and southern China (kdr frequency <10%). Seven predictor variables were selected based on a radial basis function neural network model. A multilayer perceptron (MLP) network model revealed that the number of crops in a year was the most important predictor for the kdr mutation rate. Topography, long‐term mean climate and land cover all contributed to the kdr mutation rate. The observed mean kdr frequency was 53.0% and the MLP network model‐predicted mean was 52.6%, a 0.1% relative error. Predicted kdr frequencies closely matched the observed values. The model explained 92% of the total variance in kdr frequency. The results indicated that kdr was associated with the intensity of pesticide usage. Crop cultivation information, together with environmental factors, may well predict the spatial heterogeneity of kdr mutations in An. sinensis in China.