Developing combinatorial multi-component therapies (CMCT) of drugs that are more specific and have fewer side effects than traditional one drug therapies

Drugs designed for a specific target are always found to have multiple effects. Rather than hope that one bullet can be designed to hit only one target, nonlinear interactions across genomic and proteomic networks could be used to design Combinatorial Multi-Component Therapies (CMCT) that are more targeted with fewer side effects. We show here how computational approaches can be used to predict which combinations of drugs would produce the best effects. Using a nonlinear model of how the output effect depends on multiple input drugs, we show that an artificial neural network can accurately predict the effect of all 2¹⁵ = 32,768 combinations of drug inputs using only the limited data of the output effect of the drugs presented one-at-a-time and pairs-at-a-time.     

Adaptation supports short-term memory in a visual change detection task

The maintenance of short-term memories is critical for survival in a dynamically changing world. Previous studies suggest that this memory can be stored in the form of persistent neural activity or using a synaptic mechanism, such as with short-term plasticity. Here, we compare the predictions of these two mechanisms to neural and behavioral measurements in a visual change detection task. Mice were trained to respond to changes in a repeated sequence of natural images while neural activity was recorded using two-photon calcium imaging. We also trained two types of artificial neural networks on the same change detection task as the mice. Following fixed pre-processing using a pretrained convolutional neural network, either a recurrent neural network (RNN) or a feedforward neural network with short-term synaptic depression (STPNet) was trained to the same level of performance as the mice. While both networks are able to learn the task, the STPNet model contains units whose activity are more similar to the in vivo data and produces errors which are more similar to the mice. When images are omitted, an unexpected perturbation which was absent during training, mice often do not respond to the omission but are more likely to respond to the subsequent image. Unlike the RNN model, STPNet produces a similar pattern of behavior. These results suggest that simple neural adaptation mechanisms may serve as an important bottom-up memory signal in this task, which can be used by downstream areas in the decision-making process.     

A Model for Improving the Learning Curves of Artificial Neural Networks

In this article, the performance of a hybrid artificial neural network (i.e. scale-free and small-world) was analyzed and its learning curve compared to three other topologies: random, scale-free and small-world, as well as to the chemotaxis neural network of the nematode Caenorhabditis Elegans. One hundred equivalent networks (same number of vertices and average degree) for each topology were generated and each was trained for one thousand epochs. After comparing the mean learning curves of each network topology with the C. elegans neural network, we found that the networks that exhibited preferential attachment exhibited the best learning curves.     

Calculation of the Volterra kernels of non-linear dynamic systems using an artificial neural network

The Volterra series is a well-known method of describing non-linear dynamic systems. A major limitation of this technique is the difficulty involved in the calculation of the kernels. More recently, artificial neural networks have been used to produce black box models of non-linear dynamic systems. In this paper we show how a certain class of artificial neural networks are equivalent to Volterra series and give the equation for the nth order Volterra kernel in terms of the internal parameters of the network. The technique is then illustrated using a specific non-linear system. The kernels obtained by the method described in the paper are compared with those obtained by a Toeplitz matrix inversion technique. Received: 4 June 1993/Accepted in revised form: 2 March 1994     

Neural network approach to reference evapotranspiration modeling from limited climatic data in arid regions

In order to better manage the limited water resources in arid regions, accurate determination of plant water requirements is necessary. For that, the evaluation of reference evapotranspiration (ET0)--a basic component of the hydrological cycle--is essential. In this context, the Penman Monteith equation, known for its accuracy, requires a high number of climatic parameters that are not always fully available from most meteorological stations. Our study examines the effectiveness of the use of artificial neural networks (ANN) for the evaluation of ET0 using incomplete meteorological parameters. These neural networks use daily climatic data (temperature, relative humidity, wind speed and the insolation duration) as inputs, and ET0 values estimated by the Penman-Monteith formula as outputs. The results show that the proper choice of neural network architecture allows not only error minimization but also maximizes the relationship between the dependent variable and the independent variables. In fact, with a network of two hidden layers and eight neurons per layer, we obtained, during the test phase, values of 1, 1 and 0.01 for the determination coefficient, the criterion of Nash and the mean square error, respectively. Comparing results between multiple linear regression and the neural method revealed the good modeling quality and high performance of the latter, due to the possibility of improving performance criteria. In this work, we considered correlations between input variables that improve the accuracy of the model and do not pose problems of multi-collinearity. Furthermore, we succeeded in avoiding overfitting and could generalize the model for other similar areas.     

Nonlinear torque and air-to-fuel ratio control of spark ignition engines using neuro-sliding mode techniques

This paper presents a new approach for the calibration and control of spark ignition engines using a combination of neural networks and sliding mode control technique. Two parallel neural networks are utilized to realize a neuro-sliding mode control (NSLMC) for self-learning control of automotive engines. The equivalent control and the corrective control terms are the outputs of the neural networks. Instead of using error backpropagation algorithm, the network weights of equivalent control are updated using the Levenberg-Marquardt algorithm. Moreover, a new approach is utilized to update the gain of corrective control. Both modifications of the NSLMC are aimed at improving the transient performance and speed of convergence. Using the data from a test vehicle with a V8 engine, we built neural network models for the engine torque (TRQ) and the air-to-fuel ratio (AFR) dynamics and developed NSLMC controllers to achieve tracking control. The goal of TRQ control and AFR control is to track the commanded values under various operating conditions. From simulation studies, the feasibility and efficiency of the approach are illustrated. For both control problems, excellent tracking performance has been achieved.     

A hybrid genetic--neural algorithm for modeling the biodegradation process of DnBP in AAO system

A hybrid artificial neural network - genetic algorithm numerical technique was successfully developed to model, and to simulate the biodegradation process of di-n-butyl phthalate in an anaerobic/anoxic/oxic (AAO) system. The fate of DnBP was investigated, and a removal kinetic model including sorption and biodegradation was formulated. To correlate the experimental data with available models or some modified empirical equations, the steady state model equations describing the biodegradation process have been solved using genetic algorithm (GA) and artificial neural network (ANN) from the water quality characteristic parameters. Compared with the kinetic model, the performance of the GA-ANN for modeling the DnBP was found to be more impressive. The results show that the predicted values well fit measured concentrations, which was also supported by the relatively low RMSE (0.2724), MAPE (3.6137) and MSE (0.0742)and very high R (0.9859) values, and which illustrates the GA-ANN model predicting effluent DnBP more accurately than the mechanism model forecasting.     

Artificial astrocytes improve neural network performance

Compelling evidence indicates the existence of bidirectional communication between astrocytes and neurons. Astrocytes, a type of glial cells classically considered to be passive supportive cells, have been recently demonstrated to be actively involved in the processing and regulation of synaptic information, suggesting that brain function arises from the activity of neuron-glia networks. However, the actual impact of astrocytes in neural network function is largely unknown and its application in artificial intelligence remains untested. We have investigated the consequences of including artificial astrocytes, which present the biologically defined properties involved in astrocyte-neuron communication, on artificial neural network performance. Using connectionist systems and evolutionary algorithms, we have compared the performance of artificial neural networks (NN) and artificial neuron-glia networks (NGN) to solve classification problems. We show that the degree of success of NGN is superior to NN. Analysis of performances of NN with different number of neurons or different architectures indicate that the effects of NGN cannot be accounted for an increased number of network elements, but rather they are specifically due to astrocytes. Furthermore, the relative efficacy of NGN vs. NN increases as the complexity of the network increases. These results indicate that artificial astrocytes improve neural network performance, and established the concept of Artificial Neuron-Glia Networks, which represents a novel concept in Artificial Intelligence with implications in computational science as well as in the understanding of brain function.     

Neural Network Cascade Optimizes MicroRNA Biomarker Selection for Nasopharyngeal Cancer Prognosis

MicroRNAs (miRNAs) have been shown to be promising biomarkers in predicting cancer prognosis. However, inappropriate or poorly optimized processing and modeling of miRNA expression data can negatively affect prediction performance. Here, we propose a holistic solution for miRNA biomarker selection and prediction model building. This work introduces the use of a neural network cascade, a cascaded constitution of small artificial neural network units, for evaluating miRNA expression and patient outcome. A miRNA microarray dataset of nasopharyngeal carcinoma was retrieved from Gene Expression Omnibus to illustrate the methodology. Results indicated a nonlinear relationship between miRNA expression and patient death risk, implying that direct comparison of expression values is inappropriate. However, this method performs transformation of miRNA expression values into a miRNA score, which linearly measures death risk. Spearman correlation was calculated between miRNA scores and survival status for each miRNA. Finally, a nine-miRNA signature was optimized to predict death risk after nasopharyngeal carcinoma by establishing a neural network cascade consisting of 13 artificial neural network units. Area under the ROC was 0.951 for the internal validation set and had a prediction accuracy of 83% for the external validation set. In particular, the established neural network cascade was found to have strong immunity against noise interference that disturbs miRNA expression values. This study provides an efficient and easy-to-use method that aims to maximize clinical application of miRNAs in prognostic risk assessment of patients with cancer.     

Decreasing-Rate Pruning Optimizes the Construction of Efficient and Robust Distributed Networks

Robust, efficient, and low-cost networks are advantageous in both biological and engineered systems. During neural network development in the brain, synapses are massively over-produced and then pruned-back over time. This strategy is not commonly used when designing engineered networks, since adding connections that will soon be removed is considered wasteful. Here, we show that for large distributed routing networks, network function is markedly enhanced by hyper-connectivity followed by aggressive pruning and that the global rate of pruning, a developmental parameter not previously studied by experimentalists, plays a critical role in optimizing network structure. We first used high-throughput image analysis techniques to quantify the rate of pruning in the mammalian neocortex across a broad developmental time window and found that the rate is decreasing over time. Based on these results, we analyzed a model of computational routing networks and show using both theoretical analysis and simulations that decreasing rates lead to more robust and efficient networks compared to other rates. We also present an application of this strategy to improve the distributed design of airline networks. Thus, inspiration from neural network formation suggests effective ways to design distributed networks across several domains.     

Optimization of auto-induction medium for G-CSF production by Escherichia coli using artificial neural networks coupled with genetic algorithm

Granulocyte colony-stimulating factor (G-CSF) is a cytokine widely used in cancer patients receiving high doses of chemotherapeutic drugs to prevent the chemotherapy-induced suppression of white blood cells. The production of recombinant G-CSF should be increased to meet the increasing market demand. This study aims to model and optimize the carbon source of auto-induction medium to enhance G-CSF production using artificial neural networks coupled with genetic algorithm. In this approach, artificial neural networks served as bioprocess modeling tools, and genetic algorithm (GA) was applied to optimize the established artificial neural network models. Two artificial neural network models were constructed: the back-propagation (BP) network and the radial basis function (RBF) network. The root mean square error, coefficient of determination, and standard error of prediction of the BP model were 0.0375, 0.959, and 8.49 %, respectively, whereas those of the RBF model were 0.0257, 0.980, and 5.82 %, respectively. These values indicated that the RBF model possessed higher fitness and prediction accuracy than the BP model. Under the optimized auto-induction medium, the predicted maximum G-CSF yield by the BP-GA approach was 71.66 %, whereas that by the RBF-GA approach was 75.17 %. These predicted values are in agreement with the experimental results, with 72.4 and 76.014 % for the BP-GA and RBF-GA models, respectively. These results suggest that RBF-GA is superior to BP-GA. The developed approach in this study may be helpful in modeling and optimizing other multivariable, non-linear, and time-variant bioprocesses.     

Functional identification of biological neural networks using reservoir adaptation for point processes

The complexity of biological neural networks does not allow to directly relate their biophysical properties to the dynamics of their electrical activity. We present a reservoir computing approach for functionally identifying a biological neural network, i.e. for building an artificial system that is functionally equivalent to the reference biological network. Employing feed-forward and recurrent networks with fading memory, i.e. reservoirs, we propose a point process based learning algorithm to train the internal parameters of the reservoir and the connectivity between the reservoir and the memoryless readout neurons. Specifically, the model is an Echo State Network (ESN) with leaky integrator neurons, whose individual leakage time constants are also adapted. The proposed ESN algorithm learns a predictive model of stimulus-response relations in in vitro and simulated networks, i.e. it models their response dynamics. Receiver Operating Characteristic (ROC) curve analysis indicates that these ESNs can imitate the response signal of a reference biological network. Reservoir adaptation improved the performance of an ESN over readout-only training methods in many cases. This also held for adaptive feed-forward reservoirs, which had no recurrent dynamics. We demonstrate the predictive power of these ESNs on various tasks with cultured and simulated biological neural networks.     

Computational neurogenetic modelling: a pathway to new discoveries in genetic neuroscience

The paper presents a methodology for using computational neurogenetic modelling (CNGM) to bring new original insights into how genes influence the dynamics of brain neural networks. CNGM is a novel computational approach to brain neural network modelling that integrates dynamic gene networks with artificial neural network model (ANN). Interaction of genes in neurons affects the dynamics of the whole ANN model through neuronal parameters, which are no longer constant but change as a function of gene expression. Through optimization of interactions within the internal gene regulatory network (GRN), initial gene/protein expression values and ANN parameters, particular target states of the neural network behaviour can be achieved, and statistics about gene interactions can be extracted. In such a way, we have obtained an abstract GRN that contains predictions about particular gene interactions in neurons for subunit genes of AMPA, GABAA and NMDA neuro-receptors. The extent of sequence conservation for 20 subunit proteins of all these receptors was analysed using standard bioinformatics multiple alignment procedures. We have observed abundance of conserved residues but the most interesting observation has been the consistent conservation of phenylalanine (F at position 269) and leucine (L at position 353) in all 20 proteins with no mutations. We hypothesise that these regions can be the basis for mutual interactions. Existing knowledge on evolutionary linkage of their protein families and analysis at molecular level indicate that the expression of these individual subunits should be coordinated, which provides the biological justification for our optimized GRN.     

Shaping embodied neural networks for adaptive goal-directed behavior

The acts of learning and memory are thought to emerge from the modifications of synaptic connections between neurons, as guided by sensory feedback during behavior. However, much is unknown about how such synaptic processes can sculpt and are sculpted by neuronal population dynamics and an interaction with the environment. Here, we embodied a simulated network, inspired by dissociated cortical neuronal cultures, with an artificial animal (an animat) through a sensory-motor loop consisting of structured stimuli, detailed activity metrics incorporating spatial information, and an adaptive training algorithm that takes advantage of spike timing dependent plasticity. By using our design, we demonstrated that the network was capable of learning associations between multiple sensory inputs and motor outputs, and the animat was able to adapt to a new sensory mapping to restore its goal behavior: move toward and stay within a user-defined area. We further showed that successful learning required proper selections of stimuli to encode sensory inputs and a variety of training stimuli with adaptive selection contingent on the animat's behavior. We also found that an individual network had the flexibility to achieve different multi-task goals, and the same goal behavior could be exhibited with different sets of network synaptic strengths. While lacking the characteristic layered structure of in vivo cortical tissue, the biologically inspired simulated networks could tune their activity in behaviorally relevant manners, demonstrating that leaky integrate-and-fire neural networks have an innate ability to process information. This closed-loop hybrid system is a useful tool to study the network properties intermediating synaptic plasticity and behavioral adaptation. The training algorithm provides a stepping stone towards designing future control systems, whether with artificial neural networks or biological animats themselves.     

Estimation of oxygen mass transfer coefficient in stirred tank reactors using artificial neural networks

The estimation of volumetric mass transfer coefficient, k(L)a, in stirred tank reactors using artificial neural networks has been studied. Several operational conditions (N and V(s)), properties of fluid (μ(a)) and geometrical parameters (D and T) have been taken into account. Learning sets of input-output patterns were obtained by k(L)a experimental data in stirred tank reactors of different volumes. The inclusion of prior knowledge as an approach which improves the neural network prediction has been considered. The hybrid model combining a neural network together with an empirical equation provides a better representation of the estimated parameter values. The outputs predicted by the hybrid neural network are compared with experimental data and some correlations previously proposed in the literature for tanks of different sizes.     

Changes of mind in an attractor network of decision-making

Attractor networks successfully account for psychophysical and neurophysiological data in various decision-making tasks. Especially their ability to model persistent activity, a property of many neurons involved in decision-making, distinguishes them from other approaches. Stable decision attractors are, however, counterintuitive to changes of mind. Here we demonstrate that a biophysically-realistic attractor network with spiking neurons, in its itinerant transients towards the choice attractors, can replicate changes of mind observed recently during a two-alternative random-dot motion (RDM) task. Based on the assumption that the brain continues to evaluate available evidence after the initiation of a decision, the network predicts neural activity during changes of mind and accurately simulates reaction times, performance and percentage of changes dependent on difficulty. Moreover, the model suggests a low decision threshold and high incoming activity that drives the brain region involved in the decision-making process into a dynamical regime close to a bifurcation, which up to now lacked evidence for physiological relevance. Thereby, we further affirmed the general conformance of attractor networks with higher level neural processes and offer experimental predictions to distinguish nonlinear attractor from linear diffusion models.     

Total nitrogen and ammonia removal prediction in horizontal subsurface flow constructed wetlands: use of artificial neural networks and development of a design equation

The aim of this paper is to examine if artificial neural networks (ANNs) can predict nitrogen removal in horizontal subsurface flow (HSF) constructed wetlands (CWs). ANN development was based on experimental data from five pilot-scale CW units. The proper selection of the components entering the ANN was achieved using principal component analysis (PCA), which identified the main factors affecting TN removal, i.e., porous media porosity, wastewater temperature and hydraulic residence time. Two neural networks were examined: the first included only the three factors selected from the PCA, and the second included in addition meteorological parameters (i.e., barometric pressure, rainfall, wind speed, solar radiation and humidity). The first model could predict TN removal rather satisfactorily (R(2)=0.53), and the second resulted in even better predictions (R(2)=0.69). From the application of the ANNs, a design equation was derived for TN removal prediction, resulting in predictions comparable to those of the ANNs (R(2)=0.47). For the validation of the results of the ANNs and of the design equation, available data from the literature were used and showed a rather satisfactory performance.