Applications of neural networks to recovery of biological products

Artificial neural networks (ANN) are being applied to recovery of products from fermentation broths. Recovery methods for which mathematical models are complex or non-existent are particularly suitable for control and analysis by ANNs. Use and potential of artificial neural networks for product recovery applications are reviewed.     

Use of artificial neural networks and a gamma-concept-based approach to model growth of and bacteriocin production by Streptococcus macedonicus ACA-DC 198 under simulated conditions of Kasseri cheese production

Growth of and bacteriocin production by Streptococcus macedonicus ACA-DC 198 were assessed and modeled under conditions simulating Kasseri cheese production. Controlled fermentations were performed in milk supplemented with yeast extract at different combinations of temperature (25, 40, and 55 degrees C), constant pH (pHs 5 and 6), and added NaCl (at concentrations of 0, 2, and 4%, wt/vol). The data obtained were used to construct two types of predictive models, namely, a modeling approach based on the gamma concept, as well as a model based on artificial neural networks (ANNs). The latter computational methods were used on 36 control fermentations to quantify the complex relationships between the conditions applied (temperature, pH, and NaCl) and population behavior and to calculate the associated biokinetic parameters, i.e., maximum specific growth and cell count decrease rates and specific bacteriocin production. The functions obtained were able to estimate these biokinetic parameters for four validation fermentation experiments and obtained good agreement between modeled and experimental values. Overall, these experiments show that both methods can be successfully used to unravel complex kinetic patterns within biological data of this kind and to predict population kinetics. Whereas ANNs yield a better correlation between experimental and predicted results, the gamma-concept-based model is more suitable for biological interpretation. Also, while the gamma-concept-based model has not been designed for modeling of other biokinetic parameters than the specific growth rate, ANNs are able to deal with any parameter of relevance, including specific bacteriocin production.     

Kinetic models of odor transduction implemented as artificial neural networks

We present a formal model of olfactory transduction corresponding to the biochemical reaction cascade found in chemosensory neurons. It assumes that odorants bind to receptor proteins which, in turn, activate transducer mechanisms corresponding to second messenger-mediated processes. The model is reformulated as a mathematically equivalent artificial neural network (ANN). To enable comparison of the computational power of our model, previously suggested models of chemosensory transduction are also presented in ANN versions. In ANNs, certain biological parameters, such as rate constants and affinities, are transformed into weights that can be fitted by training with a given experimental data set. After training, these weights do not necessarily equal the real biological parameters, but represent a set of values that is sufficient to simulate an experimental set of data. We used ANNs to simulate data recorded from bee subplacodes and compare the capacity of our model with ANN versions of other models. Receptor neurons of the nonpheromonal, general odor-processing subsystem of the honeybee are broadly tuned, have overlapping response spectra, and show highly nonlinear concentration dependencies and mixture interactions, i.e., synergistic and inhibitory effects. Our full model alone has the necessary complexity to simulate these complex response characteristics. To account for the complex response characteristics of honeybee receptor neurons, we suggest that several different receptor protein types and at least two second messenger systems are necessary that may interact at various levels of the transduction cascade and may eventually have opposing effects on receptor neuron excitability.     

Use of Artificial Neural Networks and a Gamma-Concept-Based Approach To Model Growth of and Bacteriocin Production by Streptococcus macedonicus ACA-DC 198 under Simulated Conditions of Kasseri Cheese Production

Growth of and bacteriocin production by Streptococcus macedonicus ACA-DC 198 were assessed and modeled under conditions simulating Kasseri cheese production. Controlled fermentations were performed in milk supplemented with yeast extract at different combinations of temperature (25, 40, and 55°C), constant pH (pHs 5 and 6), and added NaCl (at concentrations of 0, 2, and 4%, wt/vol). The data obtained were used to construct two types of predictive models, namely, a modeling approach based on the gamma concept, as well as a model based on artificial neural networks (ANNs). The latter computational methods were used on 36 control fermentations to quantify the complex relationships between the conditions applied (temperature, pH, and NaCl) and population behavior and to calculate the associated biokinetic parameters, i.e., maximum specific growth and cell count decrease rates and specific bacteriocin production. The functions obtained were able to estimate these biokinetic parameters for four validation fermentation experiments and obtained good agreement between modeled and experimental values. Overall, these experiments show that both methods can be successfully used to unravel complex kinetic patterns within biological data of this kind and to predict population kinetics. Whereas ANNs yield a better correlation between experimental and predicted results, the gamma-concept-based model is more suitable for biological interpretation. Also, while the gamma-concept-based model has not been designed for modeling of other biokinetic parameters than the specific growth rate, ANNs are able to deal with any parameter of relevance, including specific bacteriocin production.     

Importance of feature selection in decision-tree and artificial-neural-network ecological applications. Alburnus alburnus alborella: A practical example

Recent advances in computing technology have increased interest in applying data mining to ecology. Machine learning is one of the methods used in most of these data mining applications. As is well known, approximately 80% of the resources in most data mining applications are devoted to cleaning and preprocessing the data. However, there are few studies on preprocessing the ecological data used as the input in these data mining systems. In this study, we use four different feature selection methods (χ², Information Gain, Gain Ratio, and Symmetrical Uncertainty) and evaluate their effectiveness in preprocessing the input data to be used for inducing artificial neural networks (ANNs) and decision trees (DTs). The presence/absence of fish is the data item used to illustrate our models. Feature selection is fundamental in order to increase the performances of the models obtained. Accuracy of classification improves when a small set of optimally selected features is used. DTs and ANNs are very useful tools when applied to modeling presence/absence of Alburnus alburnus alborella. ANNs generally performed better than DT models.     

Total nitrogen and ammonia removal prediction in horizontal subsurface flow constructed wetlands: use of artificial neural networks and development of a design equation

The aim of this paper is to examine if artificial neural networks (ANNs) can predict nitrogen removal in horizontal subsurface flow (HSF) constructed wetlands (CWs). ANN development was based on experimental data from five pilot-scale CW units. The proper selection of the components entering the ANN was achieved using principal component analysis (PCA), which identified the main factors affecting TN removal, i.e., porous media porosity, wastewater temperature and hydraulic residence time. Two neural networks were examined: the first included only the three factors selected from the PCA, and the second included in addition meteorological parameters (i.e., barometric pressure, rainfall, wind speed, solar radiation and humidity). The first model could predict TN removal rather satisfactorily (R(2)=0.53), and the second resulted in even better predictions (R(2)=0.69). From the application of the ANNs, a design equation was derived for TN removal prediction, resulting in predictions comparable to those of the ANNs (R(2)=0.47). For the validation of the results of the ANNs and of the design equation, available data from the literature were used and showed a rather satisfactory performance.     

Genome-wide identification of specific oligonucleotides using artificial neural network and computational genomic analysis

Background  

Genome-wide identification of specific oligonucleotides (oligos) is a computationally-intensive task and is a requirement for designing microarray probes, primers, and siRNAs. An artificial neural network (ANN) is a machine learning technique that can effectively process complex and high noise data. Here, ANNs are applied to process the unique subsequence distribution for prediction of specific oligos.     

A simulation study comparing the impact of experimental error on the performance of experimental designs and artificial neural networks used for process screening

Many variables and their interactions can affect a biotechnological process. Testing a large number of variables and all their possible interactions is a cumbersome task and its cost can be prohibitive. Several screening strategies, with a relatively low number of experiments, can be used to find which variables have the largest impact on the process and estimate the magnitude of their effect. One approach for process screening is the use of experimental designs, among which fractional factorial and Plackett–Burman designs are frequent choices. Other screening strategies involve the use of artificial neural networks (ANNs). The advantage of ANNs is that they have fewer assumptions than experimental designs, but they render black-box models (i.e., little information can be extracted about the process mechanics). In this paper, we simulate a biotechnological process (fed-batch growth of bakers yeast) to analyze and compare the effect of random experimental errors of different magnitudes and statistical distributions on experimental designs and ANNs. Except for the situation in which the error has a normal distribution and the standard deviation is constant, it was not possible to determine a clear-cut rule for favoring one screening strategy over the other. Instead, we found that the data can be better analyzed using both strategies simultaneously.     

What are artificial neural networks and what they can do?

The artificial neural networks (ANN) are very often applied in many areas of toxicology for the solving of complex problems, such as the prediction of chemical compound properties and quantitative structure-activity relationship. The aim of this contribution is to give the basic knowledge about conception of ANN, theirs division and finally, the typical application of ANN will be discussed. Due to the diversity of architectures and adaptation algorithms, the ANNs are used in the broad spectrum of applications from the environmental processes modeling, through the optimization to quantitative structure-activity relationship (QSAR) methods. In addition, especially ANNs with Kohonen learning are very effective classification tool. The ANNs are mostly applied in cases, where the commonly used methods does not work.     

A Lempel-Ziv complexity-based neural network pruning algorithm

This paper presents a pruning method for artificial neural networks (ANNs) based on the 'Lempel-Ziv complexity' (LZC) measure. We call this method the 'silent pruning algorithm' (SPA). The term 'silent' is used in the sense that SPA prunes ANNs without causing much disturbance during the network training. SPA prunes hidden units during the training process according to their ranks computed from LZC. LZC extracts the number of unique patterns in a time sequence obtained from the output of a hidden unit and a smaller value of LZC indicates higher redundancy of a hidden unit. SPA has a great resemblance to biological brains since it encourages higher complexity during the training process. SPA is similar to, yet different from, existing pruning algorithms. The algorithm has been tested on a number of challenging benchmark problems in machine learning, including cancer, diabetes, heart, card, iris, glass, thyroid, and hepatitis problems. We compared SPA with other pruning algorithms and we found that SPA is better than the 'random deletion algorithm' (RDA) which prunes hidden units randomly. Our experimental results show that SPA can simplify ANNs with good generalization ability.     

Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks

The purpose of this study was to develop a method of classifying cancers to specific diagnostic categories based on their gene expression signatures using artificial neural networks (ANNs). We trained the ANNs using the small, round blue-cell tumors (SRBCTs) as a model. These cancers belong to four distinct diagnostic categories and often present diagnostic dilemmas in clinical practice. The ANNs correctly classified all samples and identified the genes most relevant to the classification. Expression of several of these genes has been reported in SRBCTs, but most have not been associated with these cancers. To test the ability of the trained ANN models to recognize SRBCTs, we analyzed additional blinded samples that were not previously used for the training procedure, and correctly classified them in all cases. This study demonstrates the potential applications of these methods for tumor diagnosis and the identification of candidate targets for therapy.     

String matching artificial neural networks.

Three artificial neural networks (ANNs) are proposed for solving a variety of on- and off-line string matching problems. The ANN structure employed as the building block of these ANNs is derived from the harmony theory (HT) ANN, whereby the resulting string matching ANNs are characterized by fast match-mismatch decisions, low computational complexity, and activation values of the ANN output nodes that can be used as indicators of substitution, insertion (addition) and deletion spelling errors.     

Assessment of respiratory system mechanics by artificial neural networks: an exploratory study.

We evaluated 1) the performance of an artificial neural network (ANN)-based technology in assessing the respiratory system resistance (Rrs) and compliance (Crs) in a porcine model of acute lung injury and 2) the possibility of using, for ANN training, signals coming from an electrical analog (EA) of the lung. Two differently experienced ANNs were compared. One ANN (ANN(BIO)) was trained on tracings recorded at different time points after the administration of oleic acid in 10 anesthetized and paralyzed pigs during constant-flow mechanical ventilation. A second ANN (ANN(MOD)) was trained on EA simulations. Both ANNs were evaluated prospectively on data coming from four different pigs. Linear regression between ANN output and manually computed mechanics showed a regression coefficient (R) of 0.98 for both ANNs in assessing Crs. On Rrs, ANN(BIO) showed a performance expressed by R = 0.40 and ANN(MOD) by R = 0.61. These results suggest that ANNs can learn to assess the respiratory system mechanics during mechanical ventilation but that the assessment of resistance and compliance by ANNs may require different approaches.     

Issues regarding artificial neural network modeling for reactors and fermenters

In recent years researchers in many areas have used artificial neural networks (ANNs) to model a variety of physical relationships. While in many cases this selection appears sound and reasonable, one must remember than ANN modeling is an empirical modeling technique (based on data) and is subject to the limitations of such techniques. Poor prediction occurs when the training data set does not contain adequate "information" to model a dynamic process. Using data from a simulated continuous-stirred tank reactor, this paper illustrates four scenarios: (1) steady state, (2) large process time constant, (3) infrequent sampling, and (4) variable sampling rate. The first scenario is typical of simulation studies while the other three incorporate attributes found in real plant data. For the cases in which ANNs predicted well, linear regression (LR), one of the oldest empirical modeling techniques, predicted equally well, and when LR failed to accurately model/predict the data, ANNs predicted poorly. Since real plant data would resemble a combination of situations (2), (3), and (4), it is important to understand that empirical models are not necessarily appropriate for predictively modeling dynamic processes in practice.     

Evaluation and amelioration of computer-aided diagnosis with artificial neural networks utilizing small-sized sample sets

Concerns about the specificity and reliability of artificial neural networks (ANNs) impede further application of ANNs in medicine. This is particularly true when developing computer-aided diagnosis (CAD) tools using ANNs for orphan diseases and emerging research areas where only a small-sized sample set is available. It is unreasonable to claim one ANN's performance as better than another simply on the basis of a single output without considering possible output variability due to factors including data noise and ANN training protocols. In this paper, a bootstrap resampling method is proposed to quantitatively analyze ANN output reliability and changing performance as the sample data and training protocols are varied. The method is tested in the area of feature classification for analysis of masses detected on mammograms. Our experiments show that ANNs performance, measured in terms of the area under the receiver operating characteristic (ROC) curve, is not a fixed value, but follows a distribution function sensitive to many factors. We demonstrate that our approach to determining the bootstrap estimates of confidence intervals (CIs) and prediction intervals (PIs) can be used to assure optimal performance in terms of ANN model configuration. We also show that the unintentional inclusion of data noise, which biases ANN results in small task-specific databases, can be accurately detected via the bootstrap estimates.     

Constraints on adaptation: explaining deviation from optimal sex ratio using artificial neural networks

Determining processes constraining adaptation is a major challenge facing evolutionary biology, and sex allocation has proved a useful model system for exploring different constraints. We investigate the evolution of suboptimal sex allocation in a solitary parasitoid wasp system by modelling information acquisition and processing using artificial neural networks (ANNs) evolving according to a genetic algorithm. Theory predicts an instantaneous switch from the production of male to female offspring with increasing host size, whereas data show gradual changes. We found that simple ANNs evolved towards producing sharp switches in sex ratio, but additional biologically reasonable assumptions of costs of synapse maintenance, and simplification of the ANNs, led to more gradual adjustment. Switch sharpness was robust to uncertainty in fitness consequences of host size, challenging interpretations of previous empirical findings. Our results also question some intuitive hypotheses concerning the evolution of threshold traits and confirm how neural processing may constrain adaptive behaviour.     

Quality by Design Approach: Application of Artificial Intelligence Techniques of Tablets Manufactured by Direct Compression

The publication of the International Conference of Harmonization (ICH) Q8, Q9, and Q10 guidelines paved the way for the standardization of quality after the Food and Drug Administration issued current Good Manufacturing Practices guidelines in 2003. “Quality by Design”, mentioned in the ICH Q8 guideline, offers a better scientific understanding of critical process and product qualities using knowledge obtained during the life cycle of a product. In this scope, the “knowledge space” is a summary of all process knowledge obtained during product development, and the “design space” is the area in which a product can be manufactured within acceptable limits. To create the spaces, artificial neural networks (ANNs) can be used to emphasize the multidimensional interactions of input variables and to closely bind these variables to a design space. This helps guide the experimental design process to include interactions among the input variables, along with modeling and optimization of pharmaceutical formulations. The objective of this study was to develop an integrated multivariate approach to obtain a quality product based on an understanding of the cause–effect relationships between formulation ingredients and product properties with ANNs and genetic programming on the ramipril tablets prepared by the direct compression method. In this study, the data are generated through the systematic application of the design of experiments (DoE) principles and optimization studies using artificial neural networks and neurofuzzy logic programs.KEY WORDS: artificial neural networks (ANNs), gene expression programming (GEP), optimization, quality by design (QbD)