Single cortical neurons as deep artificial neural networks

1. Download : Download high-res image (228KB)
2. Download : Download full-size image

     

Development of predictive models of laboratory animal growth using artificial neural networks

Traditional regression analysis of body weight growth curvesencounters problems .when the data are extremely variable. Whiletransformations are often employed to meet the criteria of theanalysis, some transformations are inadequate for normalizingthe data. Regression analysis also requires presuppositionsregarding the model to be fit and the techniques to be usedin the analysis. An alternative approach using artificial neuralnetworks is presented which may be suitable for developing predictivemodels of growth. Neural networks are simulators of the processesthat occur in the biological brain during the learning process.They are trained on the data, developing the necessary algorithmswithin their internal architecture, and produce a predictivemodel based on the learned facts. A dataset of Sprague–Dawleyrat (Rattus norvegicus) weights is analyzed by both traditionalregression analysis and neural network training. Predictionsof body weight are made from both models. While both methodsproduce models that adequately predict the body weights, theneural network model is superior in that it combines accuracyand precision, being less influenced by longitudinal variabilityin the data. Thus, the neural network provides another toolfor researchers to analyze growth curve data.     

Optimization of biopharmaceutical downstream processes supported by mechanistic models and artificial neural networks

Downstream process development is a major area of importance within the field of bioengineering. During the design of such a downstream process, important decisions have to be made regarding the type of unit operations as well as their sequence and their operating conditions. Current computational approaches addressing these issues either show a high level of simplification or struggle with computational speed. Therefore, this article presents a new approach that combines detailed mechanistic models and speed‐enhancing artificial neural networks. This approach was able to simultaneously optimize a process with three different chromatographic columns toward yield with a minimum purity of 99.9%. The addition of artificial neural networks greatly accelerated this optimization. Due to high computational speed, the approach is easily extendable to include more unit operations. Therefore, it can be of great help in the acceleration of downstream process development. © 2017 American Institute of Chemical Engineers Biotechnol. Prog., 33:696–707, 2017     

Pruning artificial neural networks using neural complexity measures

This paper describes a new method for pruning artificial neural networks, using a measure of the neural complexity of the neural network. This measure is used to determine the connections that should be pruned. The measure computes the information-theoretic complexity of a neural network, which is similar to, yet different from previous research on pruning. The method proposed here shows how overly large and complex networks can be reduced in size, whilst retaining learnt behaviour and fitness. The technique proposed here helps to discover a network topology that matches the complexity of the problem it is meant to solve. This novel pruning technique is tested in a robot control domain, simulating a racecar. It is shown, that the proposed pruning method is a significant improvement over the most commonly used pruning method Magnitude Based Pruning. Furthermore, some of the pruned networks prove to be faster learners than the benchmark network that they originate from. This means that this pruning method can also help to unleash hidden potential in a network, because the learning time decreases substantially for a pruned a network, due to the reduction of dimensionality of the network.     

Physical models of neural networks

The state of art in computer modelling of neural networks with associative memory is reviewed. The available experimental data are considered on learning and memory of small neural systems, on isolated synapses and on molecular level. Computer simulations demonstrate that realistic models of neural ensembles exhibit properties which can be interpreted as image recognition, categorization, learning, prototype forming, etc. A bilayer model of associative neural network is proposed. One layer corresponds to the short-term memory, the other one to the long-term memory. Patterns are stored in terms of the synaptic strength matrix. We have studied the relaxational dynamics of neurons firing and suppression within the short-term memory layer under the influence of the long-term memory layer. The interaction among the layers has found to create a number of novel stable states which are not the learning patterns. These synthetic patterns may consist of elements belonging to different non-intersecting learning patterns. Within the framework of a hypothesis of selective and definite coding of images in brain one can interpret the observed effect as the "idea? generating" process.     

String matching artificial neural networks.

Three artificial neural networks (ANNs) are proposed for solving a variety of on- and off-line string matching problems. The ANN structure employed as the building block of these ANNs is derived from the harmony theory (HT) ANN, whereby the resulting string matching ANNs are characterized by fast match-mismatch decisions, low computational complexity, and activation values of the ANN output nodes that can be used as indicators of substitution, insertion (addition) and deletion spelling errors.     

Cardiovascular control with artificial neural networks

This paper describes the use of artificial neural networks to model cardiovascular autonomic control in a study of the hemodynamic changes associated with space flight. Cardiovascular system models were created including four parameters: heart rate, contractility, peripheral resistance, and venous tone. Artificial neural networks were then designed and trained. A technique known as backpropagation networking was used and the results of the application of this technique to heart rate control are presented and discussed.     

Modeling of pain using artificial neural networks

In dealing with human nervous system, the sensation of pain is as sophisticated as other physiological phenomena. To obtain an acceptable model of the pain, physiology of the pain has been analysed in the present paper. Pain mechanisms are explained in block diagram representation form. Because of the nonlinear interactions existing among different sections in the diagram, artificial neural networks (ANNs) have been exploited. The basic patterns associated with chronic and acute pain have been collected and then used to obtain proper features for training the neural networks. Both static and dynamic representations of the ANNs were used in this regard. The trained networks then were employed to predict response of the body when it is exposed to special excitations. These excitations have not been used in the training phase and their behavior is interesting from the physiological view. Some of these predictions can be inferred from clinical experimentations. However, more clinical tests have to be accomplished for some of the predictions.     

Bioprocess model building using artificial neural networks

Artificial neural networks are made upon of highly interconnected layers of simple neuron-like nodes. The neurons act as non-linear processing elements within the network. An attractive property of artificial neural networks is that given the appropriate network topology, they are capable of learning and characterising non-linear functional relationships. Furthermore, the structure of the resulting neural network based process model may be considered generic, in the sense that little prior process knowledge is required in its determination. The methodology therefore provides a cost efficient and reliable process modelling technique. One area where such a technique could be useful is biotechnological systems. Here, for example, the use of a process model within an estimation scheme has long been considered an effective means of overcoming inherent on-line measurement problems. However, the development of an accurate process model is extremely time consuming and often results in a model of limited applicability. Artificial neural networks could therefore prove to be a useful model building tool when striving to improve bioprocess operability. Two large scale industrial fermentation systems have been considered as test cases; a fed-batch penicillin fermentation and a continuous mycelial fermentation. Both systems serve to demonstrate the utility, flexibility and potential of the artificial neural network approach to process modelling.     

Microarray-based cancer diagnosis with artificial neural networks

In recent years, the advent of experimental methods to probe gene expression profiles of cancer on a genome-wide scale has led to widespread use of supervised machine learning algorithms to characterize these profiles. The main applications of these analysis methods range from assigning functional classes of previously uncharacterized genes to classification and prediction of different cancer tissues. This article surveys the application of machine learning algorithms to classification and diagnosis of cancer based on expression profiles. To exemplify the important issues of the classification procedure, the emphasis of this article is on one such method, namely artificial neural networks. In addition, methods to extract genes that are important for the performance of a classifier, as well as the influence of sample selection on prediction results are discussed.     

Prediction of fingers posture using artificial neural networks

Predicting the hand and fingers posture during grasping tasks is an important issue in the frame of biomechanics. In this paper, a technique based on neural networks is proposed to learn the inverse kinematics mapping between the fingertip 3D position and the corresponding joint angles. Finger movements are obtained by an instrumented glove and are mapped to a multichain model of the hand. From the fingertip desired position, the neural networks allow predicting the corresponding finger joint angles keeping the specific subject coordination patterns. Two sets of movements are considered in this study. The first one, the training set, consisting of free fingers movements is used to construct the mapping between fingertip position and joint angles. The second one, constructed for testing purposes, is composed of a sequence of grasping tasks of everyday-life objects. The maximal mean error between fingertip measured position and fingertip position obtained from simulated joint angles and forward kinematics is 0.99+/-0.76mm for the training set and 1.49+/-1.62mm for the test set. Also, the maximal RMS error of joint angles prediction is 2.85 degrees and 5.10 degrees for the training and test sets respectively, while the maximal mean joint angles prediction error is -0.11+/-4.34 degrees and -2.52+/-6.71 degrees for the training and test sets, respectively. Results relative to the learning and generalization capabilities of this architecture are also presented and discussed.     

Clustering proteins into families using artificial neural networks

An incorrect version of Figure 3 was published in the abovearticle, the corrected version is reproduced below.     

Creating artificial human genomes using generative neural networks

Generative models have shown breakthroughs in a wide spectrum of domains due to recent advancements in machine learning algorithms and increased computational power. Despite these impressive achievements, the ability of generative models to create realistic synthetic data is still under-exploited in genetics and absent from population genetics. Yet a known limitation in the field is the reduced access to many genetic databases due to concerns about violations of individual privacy, although they would provide a rich resource for data mining and integration towards advancing genetic studies. In this study, we demonstrated that deep generative adversarial networks (GANs) and restricted Boltzmann machines (RBMs) can be trained to learn the complex distributions of real genomic datasets and generate novel high-quality artificial genomes (AGs) with none to little privacy loss. We show that our generated AGs replicate characteristics of the source dataset such as allele frequencies, linkage disequilibrium, pairwise haplotype distances and population structure. Moreover, they can also inherit complex features such as signals of selection. To illustrate the promising outcomes of our method, we showed that imputation quality for low frequency alleles can be improved by data augmentation to reference panels with AGs and that the RBM latent space provides a relevant encoding of the data, hence allowing further exploration of the reference dataset and features for solving supervised tasks. Generative models and AGs have the potential to become valuable assets in genetic studies by providing a rich yet compact representation of existing genomes and high-quality, easy-access and anonymous alternatives for private databases.     

Attractor analysis of asynchronous Boolean models of signal transduction networks

Prior work on the dynamics of Boolean networks, including analysis of the state space attractors and the basin of attraction of each attractor, has mainly focused on synchronous update of the nodes’ states. Although the simplicity of synchronous updating makes it very attractive, it fails to take into account the variety of time scales associated with different types of biological processes. Several different asynchronous update methods have been proposed to overcome this limitation, but there have not been any systematic comparisons of the dynamic behaviors displayed by the same system under different update methods. Here we fill this gap by combining theoretical analysis such as solution of scalar equations and Markov chain techniques, as well as numerical simulations to carry out a thorough comparative study on the dynamic behavior of a previously proposed Boolean model of a signal transduction network in plants. Prior evidence suggests that this network admits oscillations, but it is not known whether these oscillations are sustained. We perform an attractor analysis of this system using synchronous and three different asynchronous updating schemes both in the case of the unperturbed (wild-type) and perturbed (node-disrupted) systems. This analysis reveals that while the wild-type system possesses an update-independent fixed point, any oscillations eventually disappear unless strict constraints regarding the timing of certain processes and the initial state of the system are satisfied. Interestingly, in the case of disruption of a particular node all models lead to an extended attractor. Overall, our work provides a roadmap on how Boolean network modeling can be used as a predictive tool to uncover the dynamic patterns of a biological system under various internal and environmental perturbations.     

Improved neuronal models for studying neural networks

Previous neuronal models used for the study of neural networks are considered. Equations are developed for a model which includes: 1) a normalized range of firing rates with decreased sensitivity at large excitatory or large inhibitory input levels, 2) a single rate constant for the increase in firing rate following step changes in the input, 3) one or more rate constants, as required to fit experimental data for the adaptation of firing rates to maintained inputs. Computed responses compare well with the types of neuronal responses observed experimentally. Depending on the parameters, overdamped increases and decreases, damped oscillatory or maintained oscillatory changes in firing rate are observed to step changes in the input. The integrodifferential equations describing the neuronal models can be represented by a set of first-order differential equations. Steady-state solutions for these equations can be obtained for constant inputs, as well as the stability of the solutions to small perturbations. The linear frequency response function is derived for sufficiently small time-varying inputs. The linear responses are also compared with the computed solutions for larger non-linear responses.     

Application of artificial neural networks for yeast cells classification

The present paper describes a method for automatic classification of yeast cells in four groups: active with oval form, budding, weakened and dead. This method can be used in the previously developed structural mathematical model of the yeast cultivation process described in [1].     

Prediction of problematic wine fermentations using artificial neural networks

Artificial neural networks (ANNs) have been used for the recognition of non-linear patterns, a characteristic of bioprocesses like wine production. In this work, ANNs were tested to predict problems of wine fermentation. A database of about 20,000 data from industrial fermentations of Cabernet Sauvignon and 33 variables was used. Two different ways of inputting data into the model were studied, by points and by fermentation. Additionally, different sub-cases were studied by varying the predictor variables (total sugar, alcohol, glycerol, density, organic acids and nitrogen compounds) and the time of fermentation (72, 96 and 256 h). The input of data by fermentations gave better results than the input of data by points. In fact, it was possible to predict 100% of normal and problematic fermentations using three predictor variables: sugars, density and alcohol at 72 h (3 days). Overall, ANNs were capable of obtaining 80% of prediction using only one predictor variable at 72 h; however, it is recommended to add more fermentations to confirm this promising result.     

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

To facilitate decision support in freshwater ecosystem protection and restoration management, habitat suitability models can be very valuable. Data driven methods such as artificial neural networks (ANNs) are particularly useful in this context, seen their time-efficient development and relatively high reliability. However, specialized and technical literature on neural network modelling offers a variety of model development criteria to select model architecture, training procedure, etc. This may lead to confusion among ecosystem modellers and managers regarding the optimal training and validation methodology. This paper focuses on the analysis of ANN development and application for predicting macroinvertebrate communities, a species group commonly used in freshwater assessment worldwide. This review reflects on the different aspects regarding model development and application based on a selection of 26 papers reporting the use of ANN models for the prediction of macroinvertebrates. This analysis revealed that the applied model training and validation methodologies can often be improved and moreover crucial steps in the modelling process are often poorly documented. Therefore, suggestions to improve model development, assessment and application in ecological river management are presented. In particular, data pre-processing determines to a high extent the reliability of the induced models and their predictive relevance. This also counts for the validation criteria, that need to be better tuned to the practical simulation requirements. Moreover, the use of sensitivity methods can help to extract knowledge on the habitat preference of species and allow peer-review by ecological experts. The selection of relevant input variables remains a critical challenge as well. Model coupling is a missing crucial step to link human activities, hydrology, physical habitat conditions, water quality and ecosystem status. This last aspect is probably the most valuable aspect to enable decision support in water management based on ANN models.     

Modeling of phenol degradation system using artificial neural networks

Pseudomonas pictorum (NICM-2077) an effective strain used in the biodegradation of phenol was grown on various nutrient compounds which protect the microbes while confronting shock loads of concentrated toxic pollutants during waste water treatment. In the present study the effect of glucose, yeast extract, (NH₄)₂SO₄ and NaCl on phenol degradation has been investigated and a Artificial Neural Network (ANN) Model has been developed to predict degradation. Also the learning, recall and generalization characteristics of neural networks has been studied using phenol degradation system data. The network model was then compared with a Multiple Regression Analysis model (MRA) arrived from the same training data. Further, these two models were used to predict the percentage degradation of phenol for a blind test data. Though both the models perform equally well ANN is found to be better than MRA due to its slightly higher coefficient of correlation, lower RMS error value and lower average absolute error value during prediction.