The superior fault tolerance of artificial neural network training with a fault/noise injectionbased genetic algorithm

Artificial neural networks (ANNs) are powerful computational tools that are designed to replicate the human brain and adopted to solve a variety of problems in many different fields. Fault tolerance (FT), an important property of ANNs, ensures their reliability when significant portions of a network are lost. In this paper, a fault/noise injection-based (FIB) genetic algorithm (GA) is proposed to construct fault-tolerant ANNs. The FT performance of an FIB-GA was compared with that of a common genetic algorithm, the back-propagation algorithm, and the modification of weights algorithm. The FIB-GA showed a slower fitting speed when solving the exclusive OR (XOR) problem and the overlapping classification problem, but it significantly reduced the errors in cases of single or multiple faults in ANN weights or nodes. Further analysis revealed that the fit weights showed no correlation with the fitting errors in the ANNs constructed with the FIB-GA, suggesting a relatively even distribution of the various fitting parameters. In contrast, the output weights in the training of ANNs implemented with the use the other three algorithms demonstrated a positive correlation with the errors. Our findings therefore indicate that a combination of the fault/noise injection-based method and a GA is capable of introducing FT to ANNs and imply that the distributed ANNs demonstrate superior FT performance.     

Assessment of respiratory system mechanics by artificial neural networks: an exploratory study.

We evaluated 1) the performance of an artificial neural network (ANN)-based technology in assessing the respiratory system resistance (Rrs) and compliance (Crs) in a porcine model of acute lung injury and 2) the possibility of using, for ANN training, signals coming from an electrical analog (EA) of the lung. Two differently experienced ANNs were compared. One ANN (ANN(BIO)) was trained on tracings recorded at different time points after the administration of oleic acid in 10 anesthetized and paralyzed pigs during constant-flow mechanical ventilation. A second ANN (ANN(MOD)) was trained on EA simulations. Both ANNs were evaluated prospectively on data coming from four different pigs. Linear regression between ANN output and manually computed mechanics showed a regression coefficient (R) of 0.98 for both ANNs in assessing Crs. On Rrs, ANN(BIO) showed a performance expressed by R = 0.40 and ANN(MOD) by R = 0.61. These results suggest that ANNs can learn to assess the respiratory system mechanics during mechanical ventilation but that the assessment of resistance and compliance by ANNs may require different approaches.     

Modeling force-velocity relation in skeletal muscle isotonic contraction using an artificial neural network

The aim of this study is to design an artificial neural network (ANN) to model force-velocity relation in skeletal muscle isotonic contraction. We obtained the data set, including physiological and morphometric parameters, by myography and morphometric measurements on frog gastrocnemius muscle. Then, we designed a multilayer perceptron ANN, the inputs of which are muscle volume, muscle optimum length, tendon length, preload, and afterload. The output of the ANN is contraction velocity. The experimental data were divided randomly into two parts. The first part was used to train the ANN. In order to validate the model, the second part of experimental data, which was not used in training, was employed to the ANN and then, its output was compared with Hill model and the experimental data. The behavior of ANN in high forces was more similar to experimental data, but in low forces the Hill model had better results. Furthermore, extrapolation of ANN performance showed that our model is more or less able to simulate eccentric contraction. Our results indicate that ANNs represent a powerful tool to capture some essential features of muscle isotonic contraction.     

String matching artificial neural networks.

Three artificial neural networks (ANNs) are proposed for solving a variety of on- and off-line string matching problems. The ANN structure employed as the building block of these ANNs is derived from the harmony theory (HT) ANN, whereby the resulting string matching ANNs are characterized by fast match-mismatch decisions, low computational complexity, and activation values of the ANN output nodes that can be used as indicators of substitution, insertion (addition) and deletion spelling errors.     

Artificial neural networks: fundamentals, computing, design, and application

Artificial neural networks (ANNs) are relatively new computational tools that have found extensive utilization in solving many complex real-world problems. The attractiveness of ANNs comes from their remarkable information processing characteristics pertinent mainly to nonlinearity, high parallelism, fault and noise tolerance, and learning and generalization capabilities. This paper aims to familiarize the reader with ANN-based computing (neurocomputing) and to serve as a useful companion practical guide and toolkit for the ANNs modeler along the course of ANN project development. The history of the evolution of neurocomputing and its relation to the field of neurobiology is briefly discussed. ANNs are compared to both expert systems and statistical regression and their advantages and limitations are outlined. A bird's eye review of the various types of ANNs and the related learning rules is presented, with special emphasis on backpropagation (BP) ANNs theory and design. A generalized methodology for developing successful ANNs projects from conceptualization, to design, to implementation, is described. The most common problems that BPANNs developers face during training are summarized in conjunction with possible causes and remedies. Finally, as a practical application, BPANNs were used to model the microbial growth curves of S. flexneri. The developed model was reasonably accurate in simulating both training and test time-dependent growth curves as affected by temperature and pH.     

Temporal separation of Cerenkov radiation and scintillation using artificial neural networks in Clinical LINACs

The irradiation of scintillator-fiber optic dosimeters by clinical LINACs results in the measurement of scintillation and Cerenkov radiation. In scintillator-fiber optic dosimetry, the scintillation and Cerenkov radiation responses are separated to determine the dose deposited in the scintillator volume. Artificial neural networks (ANNs) were trained and applied in a novel single probe method for the temporal separation of scintillation and Cerenkov radiation. Six dose profiles were measured using the ANN, with the dose profiles compared to those measured using background subtraction and an ionisation chamber. The average dose discrepancy of the ANN measured dose was 2.2% with respect to the ionisation chamber dose and 1.2% with respect to the background subtraction measured dose, while the average dose discrepancy of the background subtraction dose was 1.6% with respect to the ionisation chamber dose. The ANNs performance was degraded when compared with background subtraction, arising from an inaccurate model used to synthesise ANN training data.     

Calculating confidence intervals for prediction error in microarray classification using resampling

Cross-validation based point estimates of prediction accuracy are frequently reported in microarray class prediction problems. However these point estimates can be highly variable, particularly for small sample numbers, and it would be useful to provide confidence intervals of prediction accuracy. We performed an extensive study of existing confidence interval methods and compared their performance in terms of empirical coverage and width. We developed a bootstrap case cross-validation (BCCV) resampling scheme and defined several confidence interval methods using BCCV with and without bias-correction. The widely used approach of basing confidence intervals on an independent binomial assumption of the leave-one-out cross-validation errors results in serious under-coverage of the true prediction error. Two split-sample based methods previously proposed in the literature tend to give overly conservative confidence intervals. Using BCCV resampling, the percentile confidence interval method was also found to be overly conservative without bias-correction, while the bias corrected accelerated (BCa) interval method of Efron returns substantially anti-conservative confidence intervals. We propose a simple bias reduction on the BCCV percentile interval. The method provides mildly conservative inference under all circumstances studied and outperforms the other methods in microarray applications with small to moderate sample sizes.     

Estimating genetic correlations in natural populations in the absence of pedigree information: accuracy and precision of the Lynch method

Usually, genetic correlations are estimated from breeding designs in the laboratory or greenhouse. However, estimates of the genetic correlation for natural populations are lacking, mostly because pedigrees of wild individuals are rarely known. Recently Lynch (1999) proposed a formula to estimate the genetic correlation in the absence of data on pedigree. This method has been shown to be particularly accurate provided a large sample size and a minimum (20%) proportion of relatives. Lynch (1999) proposed the use of the bootstrap to estimate standard errors associated with genetic correlations, but did not test the reliability of such a method. We tested the bootstrap and showed the jackknife can provide valid estimates of the genetic correlation calculated with the Lynch formula. The occurrence of undefined estimates, combined with the high number of replicates involved in the bootstrap, means there is a high probability of obtaining a biased upward, incomplete bootstrap, even when there is a high fraction of related pairs in a sample. It is easier to obtain complete jackknife estimates for which all the pseudovalues have been defined. We therefore recommend the use of the jackknife to estimate the genetic correlation with the Lynch formula. Provided data can be collected for more than two individuals at each location, we propose a group sampling method that produces low standard errors associated with the jackknife, even when there is a low fraction of relatives in a sample.     

An Approach to Improve the Performance of PM Forecasters

The particulate matter (PM) concentration has been one of the most relevant environmental concerns in recent decades due to its prejudicial effects on living beings and the earth’s atmosphere. High PM concentration affects the human health in several ways leading to short and long term diseases. Thus, forecasting systems have been developed to support decisions of the organizations and governments to alert the population. Forecasting systems based on Artificial Neural Networks (ANNs) have been highlighted in the literature due to their performances. In general, three ANN-based approaches have been found for this task: ANN trained via learning algorithms, hybrid systems that combine search algorithms with ANNs, and hybrid systems that combine ANN with other forecasters. Independent of the approach, it is common to suppose that the residuals (error series), obtained from the difference between actual series and forecasting, have a white noise behavior. However, it is possible that this assumption is infringed due to: misspecification of the forecasting model, complexity of the time series or temporal patterns of the phenomenon not captured by the forecaster. This paper proposes an approach to improve the performance of PM forecasters from residuals modeling. The approach analyzes the remaining residuals recursively in search of temporal patterns. At each iteration, if there are temporal patterns in the residuals, the approach generates the forecasting of the residuals in order to improve the forecasting of the PM time series. The proposed approach can be used with either only one forecaster or by combining two or more forecasting models. In this study, the approach is used to improve the performance of a hybrid system (HS) composed by genetic algorithm (GA) and ANN from residuals modeling performed by two methods, namely, ANN and own hybrid system. Experiments were performed for PM_2.5 and PM₁₀ concentration series in Kallio and Vallila stations in Helsinki and evaluated from six metrics. Experimental results show that the proposed approach improves the accuracy of the forecasting method in terms of fitness function for all cases, when compared with the method without correction. The correction via HS obtained a superior performance, reaching the best results in terms of fitness function and in five out of six metrics. These results also were found when a sensitivity analysis was performed varying the proportions of the sets of training, validation and test. The proposed approach reached consistent results when compared with the forecasting method without correction, showing that it can be an interesting tool for correction of PM forecasters.     

Kohonen network study of the results of RAPD and ISSR analyses of genomic polymorphism in the genus Capsicum L

The results of studies based on multilocus molecular analyses, including random amplified polymorphic DNA (RAPD), inter-simple sequence repeat (ISSR), and amplified fragment length polymorphism (AFLP) analyses, are usually presented in the form of images (electrophoregrams, photographs, etc.). The interpretation of this information is complicated, labor-consuming, and subjective. Artificial neural networks (ANNs), which are ideal "image processors," may be useful when solving such tasks. The possibility of using ANNs for the treatment of the results of RAPD and ISSR analyses has been studied. The RAPD and ISSR spectra have been studied in fragments of DNA of plants from the genus Capsicum L. (peppers). The results of clustering the accessions studied by means of the unweighted pair-group method with arithmetic averages (UPGMA), which is often used for phylogenetic constructions based on RAPD and ISSR data, serve as expert estimates. Fundamentally new methods of genetic polymorphism estimation using ANN technologies, namely, self-organizing feature maps (SOFMs) have been developed. The results show that the clusters obtained with the use of UPGMA and SOFM coincide by more than 90%; taking into account that ANNs can deal with high noise levels and incomplete or contradictory data, the approach proposed may prove to be efficient.     

No Longer Confidential: Estimating the Confidence of Individual Regression Predictions

Quantitative predictions in computational life sciences are often based on regression models. The advent of machine learning has led to highly accurate regression models that have gained widespread acceptance. While there are statistical methods available to estimate the global performance of regression models on a test or training dataset, it is often not clear how well this performance transfers to other datasets or how reliable an individual prediction is–a fact that often reduces a user’s trust into a computational method. In analogy to the concept of an experimental error, we sketch how estimators for individual prediction errors can be used to provide confidence intervals for individual predictions. Two novel statistical methods, named CONFINE and CONFIVE, can estimate the reliability of an individual prediction based on the local properties of nearby training data. The methods can be applied equally to linear and non-linear regression methods with very little computational overhead. We compare our confidence estimators with other existing confidence and applicability domain estimators on two biologically relevant problems (MHC–peptide binding prediction and quantitative structure-activity relationship (QSAR)). Our results suggest that the proposed confidence estimators perform comparable to or better than previously proposed estimation methods. Given a sufficient amount of training data, the estimators exhibit error estimates of high quality. In addition, we observed that the quality of estimated confidence intervals is predictable. We discuss how confidence estimation is influenced by noise, the number of features, and the dataset size. Estimating the confidence in individual prediction in terms of error intervals represents an important step from plain, non-informative predictions towards transparent and interpretable predictions that will help to improve the acceptance of computational methods in the biological community.     

Genome-wide identification of specific oligonucleotides using artificial neural network and computational genomic analysis

Background  

Genome-wide identification of specific oligonucleotides (oligos) is a computationally-intensive task and is a requirement for designing microarray probes, primers, and siRNAs. An artificial neural network (ANN) is a machine learning technique that can effectively process complex and high noise data. Here, ANNs are applied to process the unique subsequence distribution for prediction of specific oligos.     

Total nitrogen and ammonia removal prediction in horizontal subsurface flow constructed wetlands: use of artificial neural networks and development of a design equation

The aim of this paper is to examine if artificial neural networks (ANNs) can predict nitrogen removal in horizontal subsurface flow (HSF) constructed wetlands (CWs). ANN development was based on experimental data from five pilot-scale CW units. The proper selection of the components entering the ANN was achieved using principal component analysis (PCA), which identified the main factors affecting TN removal, i.e., porous media porosity, wastewater temperature and hydraulic residence time. Two neural networks were examined: the first included only the three factors selected from the PCA, and the second included in addition meteorological parameters (i.e., barometric pressure, rainfall, wind speed, solar radiation and humidity). The first model could predict TN removal rather satisfactorily (R(2)=0.53), and the second resulted in even better predictions (R(2)=0.69). From the application of the ANNs, a design equation was derived for TN removal prediction, resulting in predictions comparable to those of the ANNs (R(2)=0.47). For the validation of the results of the ANNs and of the design equation, available data from the literature were used and showed a rather satisfactory performance.     

Protocols for listing threatened species can forecast extinction

Risk‐ranking protocols are used widely to classify the conservation status of the world's species. Here we report on the first empirical assessment of their reliability by using a retrospective study of 18 pairs of bird and mammal species (one species extinct and the other extant) with eight different assessors. The performance of individual assessors varied substantially, but performance was improved by incorporating uncertainty in parameter estimates and consensus among the assessors. When this was done, the ranks from the protocols were consistent with the extinction outcome in 70–80% of pairs and there were mismatches in only 10–20% of cases. This performance was similar to the subjective judgements of the assessors after they had estimated the range and population parameters required by the protocols, and better than any single parameter. When used to inform subjective judgement, the protocols therefore offer a means of reducing unpredictable biases that may be associated with expert input and have the advantage of making the logic behind assessments explicit. We conclude that the protocols are useful for forecasting extinctions, although they are prone to some errors that have implications for conservation. Some level of error is to be expected, however, given the influence of chance on extinction. The performance of risk assessment protocols may be improved by providing training in the application of the protocols, incorporating uncertainty in parameter estimates and using consensus among multiple assessors, including some who are experts in the application of the protocols. Continued testing and refinement of the protocols may help to provide better absolute estimates of risk, particularly by re‐evaluating how the protocols accommodate missing data.     

Applications of artificial neural networks predicting macroinvertebrates in freshwaters

To facilitate decision support in freshwater ecosystem protection and restoration management, habitat suitability models can be very valuable. Data driven methods such as artificial neural networks (ANNs) are particularly useful in this context, seen their time-efficient development and relatively high reliability. However, specialized and technical literature on neural network modelling offers a variety of model development criteria to select model architecture, training procedure, etc. This may lead to confusion among ecosystem modellers and managers regarding the optimal training and validation methodology. This paper focuses on the analysis of ANN development and application for predicting macroinvertebrate communities, a species group commonly used in freshwater assessment worldwide. This review reflects on the different aspects regarding model development and application based on a selection of 26 papers reporting the use of ANN models for the prediction of macroinvertebrates. This analysis revealed that the applied model training and validation methodologies can often be improved and moreover crucial steps in the modelling process are often poorly documented. Therefore, suggestions to improve model development, assessment and application in ecological river management are presented. In particular, data pre-processing determines to a high extent the reliability of the induced models and their predictive relevance. This also counts for the validation criteria, that need to be better tuned to the practical simulation requirements. Moreover, the use of sensitivity methods can help to extract knowledge on the habitat preference of species and allow peer-review by ecological experts. The selection of relevant input variables remains a critical challenge as well. Model coupling is a missing crucial step to link human activities, hydrology, physical habitat conditions, water quality and ecosystem status. This last aspect is probably the most valuable aspect to enable decision support in water management based on ANN models.     

Issues regarding artificial neural network modeling for reactors and fermenters

In recent years researchers in many areas have used artificial neural networks (ANNs) to model a variety of physical relationships. While in many cases this selection appears sound and reasonable, one must remember than ANN modeling is an empirical modeling technique (based on data) and is subject to the limitations of such techniques. Poor prediction occurs when the training data set does not contain adequate "information" to model a dynamic process. Using data from a simulated continuous-stirred tank reactor, this paper illustrates four scenarios: (1) steady state, (2) large process time constant, (3) infrequent sampling, and (4) variable sampling rate. The first scenario is typical of simulation studies while the other three incorporate attributes found in real plant data. For the cases in which ANNs predicted well, linear regression (LR), one of the oldest empirical modeling techniques, predicted equally well, and when LR failed to accurately model/predict the data, ANNs predicted poorly. Since real plant data would resemble a combination of situations (2), (3), and (4), it is important to understand that empirical models are not necessarily appropriate for predictively modeling dynamic processes in practice.