Issues regarding artificial neural network modeling for reactors and fermenters

In recent years researchers in many areas have used artificial neural networks (ANNs) to model a variety of physical relationships. While in many cases this selection appears sound and reasonable, one must remember than ANN modeling is an empirical modeling technique (based on data) and is subject to the limitations of such techniques. Poor prediction occurs when the training data set does not contain adequate "information" to model a dynamic process. Using data from a simulated continuous-stirred tank reactor, this paper illustrates four scenarios: (1) steady state, (2) large process time constant, (3) infrequent sampling, and (4) variable sampling rate. The first scenario is typical of simulation studies while the other three incorporate attributes found in real plant data. For the cases in which ANNs predicted well, linear regression (LR), one of the oldest empirical modeling techniques, predicted equally well, and when LR failed to accurately model/predict the data, ANNs predicted poorly. Since real plant data would resemble a combination of situations (2), (3), and (4), it is important to understand that empirical models are not necessarily appropriate for predictively modeling dynamic processes in practice.     

Artificial neural network based soft estimator for estimation of transducer static nonlinearity

In this work, the development of an Artificial Neural Network (ANN) based soft estimator is reported for the estimation of static-nonlinearity associated with the transducers. Under the realm of ANN based transducer modeling, only two neural models have been suggested to estimate the static-nonlinearity associated with the transducers with quite successful results. The first existing model is based on the concept of a functional link artificial neural network (FLANN) trained with mu-LMS (Least Mean Squares) learning algorithm. The second one is based on the architecture of a single layer linear ANN trained with alpha-LMS learning algorithm. However, both these models suffer from the problem of slow convergence (learning). In order to circumvent this problem, it is proposed to synthesize the direct model of transducers using the concept of a Polynomial-ANN (polynomial artificial neural network) trained with Levenberg-Marquardt (LM) learning algorithm. The proposed Polynomial-ANN oriented transducer model is implemented based on the topology of a single-layer feed-forward back-propagation-ANN. The proposed neural modeling technique provided an extremely fast convergence speed with increased accuracy for the estimation of transducer static nonlinearity. The results of convergence are very stimulating with the LM learning algorithm.     

利用神经网络对蛋白质家族进行预测

提出了一种利用神经网络为蛋白质家族建立模型的方法,这一方法的理论出发点是利用神经网络从一组同家族蛋白质序列中识别出共同的特征模式,建好的模型可用于预测蛋白质家族,使用这一方法。所能识别的模式在长度、位点等方面都不受限制。而且建模及预测过程中输入神经网络的蛋白质序列不需要作预对齐。对Pfam蛋白质库中的二十个家族运用此方法,预测的平均正确率达到了95．5％。     

An ensemble simulation approach for artificial neural network: An example from chlorophyll a simulation in Lake Poyang,China

Artificial neural network (ANN) models have been widely used in environmental modeling with considerable success. To improve the reliability of ANN models, ensemble simulations were applied in this study to develop four ANN ensemble models for chlorophyll a simulation in the largest freshwater lake (Lake Poyang) in China. Reliability (evaluated by model fit and stability) of these ANN ensemble models was compared with that of single ANN models from ensemble members. The model fit of these single ANN models varied significantly over repeated runs, indicating the unstable performance of the single ANN models. Comparing with the single ANN models, the ANN ensemble models showed a better model fit and stability, implying the potential of ensemble simulation in achieving a more reliable model. An ensemble size of 30 was adequate for the ANN ensemble models to achieve a good model fit, while an ensemble size of 50 was adequate to achieve good stability. This case study highlighted both the necessity and potential of the ensemble simulation approach to achieve a reliable ANN model with good model fit and stability.     

Modeling Healthcare Processes Using Commitments: An Empirical Evaluation

The two primary objectives of this paper are: (a) to demonstrate how Comma, a business modeling methodology based on commitments, can be applied in healthcare process modeling, and (b) to evaluate the effectiveness of such an approach in producing healthcare process models. We apply the Comma approach on a breast cancer diagnosis process adapted from an HHS committee report, and presents the results of an empirical study that compares Comma with a traditional approach based on the HL7 Messaging Standard (Traditional-HL7). Our empirical study involved 47 subjects, and two phases. In the first phase, we partitioned the subjects into two approximately equal groups. We gave each group the same requirements based on a process scenario for breast cancer diagnosis. Members of one group first applied Traditional-HL7 and then Comma whereas members of the second group first applied Comma and then Traditional-HL7—each on the above-mentioned requirements. Thus, each subject produced two models, each model being a set of UML Sequence Diagrams. In the second phase, we repartitioned the subjects into two groups with approximately equal distributions from both original groups. We developed exemplar Traditional-HL7 and Comma models; we gave one repartitioned group our Traditional-HL7 model and the other repartitioned group our Comma model. We provided the same changed set of requirements to all subjects and asked them to modify the provided exemplar model to satisfy the new requirements. We assessed solutions produced by subjects in both phases with respect to measures of flexibility, time, difficulty, objective quality, and subjective quality. Our study found that Comma is superior to Traditional-HL7 in flexibility and objective quality as validated via Student’s t-test to the 10% level of significance. Comma is a promising new approach for modeling healthcare processes. Further gains could be made through improved tooling and enhanced training of modeling personnel.     

Sequential modelling of a full-scale wastewater treatment plant using an artificial neural network

This work proposes a sequential modelling approach using an artificial neural network (ANN) to develop four independent multivariate models that are able to predict the dynamics of biochemical oxygen demand (BOD), chemical oxygen demand (COD), suspended solid (SS), and total nitrogen (TN) removal in a wastewater treatment plant (WWTP). Suitable structures of ANN models were automatically and conveniently optimized by a genetic algorithm rather than the conventional trial and error method. The sequential modelling approach, which is composed of two parts, a process disturbance estimator and a process behaviour predictor, was also presented to develop multivariate dynamic models. In particular, the process disturbance estimator was first employed to estimate the influent quality. The process behaviour predictor then sequentially predicted the effluent quality based on the estimated influent quality from the process disturbance estimator with other process variables. The efficiencies of the developed ANN models with a sequential modelling approach were demonstrated with a practical application using a data set collected from a full-scale WWTP during 2 years. The results show that the ANN with the sequential modelling approach successfully developed multivariate dynamic models of BOD, COD, SS, and TN removal with satisfactory estimation and prediction capability. Thus, the proposed method could be used as a powerful tool for the prediction of complex and nonlinear WWTP performance.     

A Neuro-Muscular Elasto-Dynamic Model of the Human Arm Part 2: Musculotendon Dynamics and Related Stress Effects

In this paper we develop an elasto-dynamic model of the human arm that includes effects of neuro-muscular control uponelastic deformation in the limb.The elasto-dynamic model of the arm is based on hybrid parameter multiple body systemvariational projection principles presented in the companion paper.Though the technique is suitable for detailed bone and jointmodeling,we present simulations for simplified geometry of the bones,discretized as Rayleigh beams with elongation,whileallowing for large deflections.Motion of the upper extremity is simulated by incorporating muscle forces derived from aHill-type model of musculotendon dynamics.The effects of muscle force are modeled in two ways.In one approach,aneffective joint torque is calculated by multiplying the muscle force by a joint moment ann.A second approach models themuscle as acting along a straight line between the origin and insertion sites of the tendon.Simple arm motion is simulated byutilizing neural feedback and feedforward control.Simulations illustrate the combined effects of neural control strategies,models of muscle force inclusion,and elastic assumptions on joint trajectories and stress and strain development in the bone andtendon.     

An efficient model development strategy for bioprocesses based on neural networks in macroscopic balances: Part II

There is a need for efficient modeling strategies which quickly lead to reliable mathematical models that can be applied for design and optimization of (bio)-chemical processes. The serial gray box modeling strategy is potentially very efficient because no detailed knowledge is needed to construct the white box part of the model and because covenient black box modeling techniques like neural networks can be used for the black box part of the model. This paper shows for a typical biochemical conversion how the serial gray box modeling strategy can be applied efficiently to obtain a model with good frequency extrapolation properties. Models with good frequency extrapolation properties can be applied under dynamic conditions that were not present during the identification experiments. For a given application domain of a model, this property can be used to considerably reduce the number of identification experiments. The serial gray box modeling strategy is demonstrated to be successful for the modeling of the enzymatic conversion of penicillin G In the concentration range of 10-100 mM and temperature range of 298-335 K. Frequency extrapolation is shown by using only constant temperatures in the (batch) identification experiments, while the model can be used reliable with varying temperatures during the (batch) validation experiments. No reliable frequency extrapolation properties could be obtained for a black box model, and for a more knowledge-driven white box model reliable frequency extrapolation properties could only be obtained by incorporating more knowledge in the model. Copyright 1999 John Wiley & Sons, Inc.     

Modeling the Afferent Dynamics of the Baroreflex Control System

In this study we develop a modeling framework for predicting baroreceptor firing rate as a function of blood pressure. We test models within this framework both quantitatively and qualitatively using data from rats. The models describe three components: arterial wall deformation, stimulation of mechanoreceptors located in the BR nerve-endings, and modulation of the action potential frequency. The three sub-systems are modeled individually following well-established biological principles. The first submodel, predicting arterial wall deformation, uses blood pressure as an input and outputs circumferential strain. The mechanoreceptor stimulation model, uses circumferential strain as an input, predicting receptor deformation as an output. Finally, the neural model takes receptor deformation as an input predicting the BR firing rate as an output. Our results show that nonlinear dependence of firing rate on pressure can be accounted for by taking into account the nonlinear elastic properties of the artery wall. This was observed when testing the models using multiple experiments with a single set of parameters. We find that to model the response to a square pressure stimulus, giving rise to post-excitatory depression, it is necessary to include an integrate-and-fire model, which allows the firing rate to cease when the stimulus falls below a given threshold. We show that our modeling framework in combination with sensitivity analysis and parameter estimation can be used to test and compare models. Finally, we demonstrate that our preferred model can exhibit all known dynamics and that it is advantageous to combine qualitative and quantitative analysis methods.     

A computational model of limb impedance control based on principles of internal model uncertainty

Efficient human motor control is characterized by an extensive use of joint impedance modulation, which is achieved by co-contracting antagonistic muscles in a way that is beneficial to the specific task. While there is much experimental evidence available that the nervous system employs such strategies, no generally-valid computational model of impedance control derived from first principles has been proposed so far. Here we develop a new impedance control model for antagonistic limb systems which is based on a minimization of uncertainties in the internal model predictions. In contrast to previously proposed models, our framework predicts a wide range of impedance control patterns, during stationary and adaptive tasks. This indicates that many well-known impedance control phenomena naturally emerge from the first principles of a stochastic optimization process that minimizes for internal model prediction uncertainties, along with energy and accuracy demands. The insights from this computational model could be used to interpret existing experimental impedance control data from the viewpoint of optimality or could even govern the design of future experiments based on principles of internal model uncertainty.     

Physics and chemistry-driven artificial neural network for predicting bioactivity of peptides and proteins and their design

Predicting the bioactivity of peptides and proteins is an important challenge in drug development and protein engineering. In this study we introduce a novel approach, the so-called “physics and chemistry-driven artificial neural network (Phys-Chem ANN)”, to deal with such a problem. Unlike the existing ANN approaches, which were designed under the inspiration of biological neural system, the Phys-Chem ANN approach is based on the physical and chemical principles, as well as the structural features of proteins. In the Phys-Chem ANN model the “hidden layers” are no longer virtual “neurons”, but real structural units of proteins and peptides. It is a hybridization approach, which combines the linear free energy concept of quantitative structure-activity relationship (QSAR) with the advanced mathematical technique of ANN. The Phys-Chem ANN approach has adopted an iterative and feedback procedure, incorporating both machine-learning and artificial intelligence capabilities. In addition to making more accurate predictions for the bioactivities of proteins and peptides than is possible with the traditional QSAR approach, the Phys-Chem ANN approach can also provide more insights about the relationship between bioactivities and the structures involved than the ANN approach does. As an example of the application of the Phys-Chem ANN approach, a predictive model for the conformational stability of human lysozyme is presented.     

Background sampling for multi-scale ensemble habitat selection modeling: Does the number of points matter?

Ensemble habitat selection modeling is becoming a popular approach among ecologists to answer different questions. Since we are still in the early stages of development and application of ensemble modeling, there remain many questions regarding performance and parameterization. One important gap, which this paper addresses, is how the number of background points used to train models influences the performance of the ensemble model. We used an empirical presence-only dataset and three different selections of background points to train scale-optimized habitat selection models using six modeling algorithms (GLM, GAM, MARS, ANN, Random Forest, and MaxEnt). We tested four ensemble models using different combinations of the component models: (a) equal numbers of background points and presences, (b) background points equaled ten times the number of presences, (c) 10,000 background points, and (d) optimized background points for each component model. Among regression-based approaches, MARS performed best when built with 10,000 background points. Among machine learning models, RF performed the best when built with equal presences and background points. Among the four ensemble models, AUC indicated that the best performing model was the ensemble with each component model including the optimized number of background points, while TSS increased as the number of background points models increased. We found that an ensemble of models, each trained with an optimal number of background points, outperformed ensembles of models trained with the same number of background points, although differences in performance were slight. When using a single modeling method, RF with equal number of presences and background points can perform better than an ensemble model, but the performance fluctuates when the number of background points is not properly selected. On the other hand, ensemble modeling provides consistently high accuracy regardless of background point sampling approach. Further, optimizing the number of background points for each component model within an ensemble model can provide the best model improvement. We suggest evaluating more models across multiple species to investigate how background point selection might affect ensemble models in different scenarios.     

An Automated System for Skeletal Maturity Assessment by Extreme Learning Machines

Assessing skeletal age is a subjective and tedious examination process. Hence, automated assessment methods have been developed to replace manual evaluation in medical applications. In this study, a new fully automated method based on content-based image retrieval and using extreme learning machines (ELM) is designed and adapted to assess skeletal maturity. The main novelty of this approach is it overcomes the segmentation problem as suffered by existing systems. The estimation results of ELM models are compared with those of genetic programming (GP) and artificial neural networks (ANNs) models. The experimental results signify improvement in assessment accuracy over GP and ANN, while generalization capability is possible with the ELM approach. Moreover, the results are indicated that the ELM model developed can be used confidently in further work on formulating novel models of skeletal age assessment strategies. According to the experimental results, the new presented method has the capacity to learn many hundreds of times faster than traditional learning methods and it has sufficient overall performance in many aspects. It has conclusively been found that applying ELM is particularly promising as an alternative method for evaluating skeletal age.     

An ensemble model for predicting Rhopalosiphum padi abundance

A novel modeling method is proposed to predict the abundance of the main vector of barley yellow dwarf virus in autumn sown cereal crops. An ensemble model based on artificial neural networks (ANN) was developed to predict the number of Rhopalosiphum padi (L.) (Homoptera: Aphididae) caught in traps during the autumn flight period at Lincoln, Canterbury, New Zealand, over the period 1982–2003. Artificial neural networks were trained using historical weather data and aphid data collected from a suction trap. Model results were compared with those obtained using multiple regression (MR) models using the same independent variables. Both ANN and MR models were validated by leave‐one‐out validation, in other words, by sequentially jackknifing each year out of the data set, fitting a model to the remaining data, then using that model to predict the number of aphids for each jackknifed year. A linear ensemble of ANN models further improved the predictions and represented the trends in the number of aphids over the 22‐year period very well. The r² between the predicted and observed numbers of aphids for the ANN models changed from 0.68 to 0.83 using the linear ensemble model, but the ensemble approach did not change the prediction for the MR models. The absolute mean error (ABSME) of prediction was much lower for the ANN ensemble predictions compared to that for the MR models. The ABMSE for the ANN models dropped from 109 to 86 aphids compared to an ABMSE reduction from 245 to 220 aphids for the MR models. We discuss the potential for ensemble models for predicting insect abundance when long‐term historical data are available.     

Construction of Predictive Models to Describe Apparent and Complex Viscosity Values of O/W Model System Meat Emulsions Using Adaptive Neuro – Fuzzy Inference System (ANFIS) and Artificial Neural Networks (ANN)

This paper introduces an adaptive neuro ?C fuzzy inference system (ANFIS) and artificial neural networks (ANN) models to predict the apparent and complex viscosity values of model system meat emulsions. Constructed models were compared with multiple linear regression (MLR) modeling based on their estimation performance. The root mean square error (RMSE), mean absolute error (MAE) and determination coefficient (R ²) statistics were performed to evaluate the accuracy of the models tested. Comparison of the models showed that the ANFIS model performed better than the ANN and MLR models to estimate the apparent and complex viscosity values of the model system meat emulsions. Coefficients of determination (R ²) calculated for estimation performance of ANFIS modeling to predict apparent and complex viscosity of the emulsions were 0.996 and 0.992, respectively. Similar R ² values (0.991 and 0.985) were obtained when estimating the performance of the ANN model. In the present study, use of the constructed ANFIS models can be suggested to effectively predict the apparent and complex viscosity values of model system meat emulsions.     

Teaching from classic papers: Hill's model of muscle contraction

A. V. Hill's 1938 paper "The heat of shortening and the dynamic constants of muscle" is an enduring classic, presenting detailed methods, meticulous experiments, and the model of muscle contraction that now bears Hill's name. Pairing a simulation based on Hill's model with a reading of his paper allows students to follow his thought process to discover key principles of muscle physiology and gain insight into how to develop quantitative models of physiological processes. In this article, the experience of the author using this approach in a graduate biomedical engineering course is outlined, along with suggestions for adapting this approach to other audiences.     

An Approach to Improve the Performance of PM Forecasters

The particulate matter (PM) concentration has been one of the most relevant environmental concerns in recent decades due to its prejudicial effects on living beings and the earth’s atmosphere. High PM concentration affects the human health in several ways leading to short and long term diseases. Thus, forecasting systems have been developed to support decisions of the organizations and governments to alert the population. Forecasting systems based on Artificial Neural Networks (ANNs) have been highlighted in the literature due to their performances. In general, three ANN-based approaches have been found for this task: ANN trained via learning algorithms, hybrid systems that combine search algorithms with ANNs, and hybrid systems that combine ANN with other forecasters. Independent of the approach, it is common to suppose that the residuals (error series), obtained from the difference between actual series and forecasting, have a white noise behavior. However, it is possible that this assumption is infringed due to: misspecification of the forecasting model, complexity of the time series or temporal patterns of the phenomenon not captured by the forecaster. This paper proposes an approach to improve the performance of PM forecasters from residuals modeling. The approach analyzes the remaining residuals recursively in search of temporal patterns. At each iteration, if there are temporal patterns in the residuals, the approach generates the forecasting of the residuals in order to improve the forecasting of the PM time series. The proposed approach can be used with either only one forecaster or by combining two or more forecasting models. In this study, the approach is used to improve the performance of a hybrid system (HS) composed by genetic algorithm (GA) and ANN from residuals modeling performed by two methods, namely, ANN and own hybrid system. Experiments were performed for PM_2.5 and PM₁₀ concentration series in Kallio and Vallila stations in Helsinki and evaluated from six metrics. Experimental results show that the proposed approach improves the accuracy of the forecasting method in terms of fitness function for all cases, when compared with the method without correction. The correction via HS obtained a superior performance, reaching the best results in terms of fitness function and in five out of six metrics. These results also were found when a sensitivity analysis was performed varying the proportions of the sets of training, validation and test. The proposed approach reached consistent results when compared with the forecasting method without correction, showing that it can be an interesting tool for correction of PM forecasters.     

Inulinase production in a batch bioreactor using agroindustrial residues as the substrate: experimental data and modeling

The production of inulinase employing agroindustrial residues as the substrate is a good alternative to reduce production costs and to minimize the environmental impact of disposing these residues in the environment. This study focused on the use of a phenomenological model and an artificial neural network (ANN) to simulate the inulinase production during the batch cultivation of the yeast Kluyveromyces marxianus NRRL Y-7571, employing a medium containing agroindustrial residues such as molasses, corn steep liquor and yeast extract. It was concluded that due to the complexity of the medium composition it was rather difficult to use a phenomenological model with sufficient accuracy. For this reason, an alternative and more cost-effective methodology based on ANN was adopted. The predictive capacity of the ANN was superior to that of the phenomenological model, indicating that the neural network approach could be used as an alternative in the predictive modeling of complex batch cultivations.     

Artificial neural network prediction of viruses in shellfish

A database was probed with artificial neural network (ANN) and multivariate logistic regression (MLR) models to investigate the efficacy of predicting PCR-identified human adenovirus (ADV), Norwalk-like virus (NLV), and enterovirus (EV) presence or absence in shellfish harvested from diverse countries in Europe (Spain, Sweden, Greece, and the United Kingdom). The relative importance of numerical and heuristic input variables to the ANN model for each country and for the combined data was analyzed with a newly defined relative strength effect, which illuminated the importance of bacteriophages as potential viral indicators. The results of this analysis showed that ANN models predicted all types of viral presence and absence in shellfish with better precision than MLR models for a multicountry database. For overall presence/absence classification accuracy, ANN modeling had a performance rate of 95.9%, 98.9%, and 95.7% versus 60.5%, 75.0%, and 64.6% for the MLR for ADV, NLV, and EV, respectively. The selectivity (prediction of viral negatives) was greater than the sensitivity (prediction of viral positives) for both models and with all virus types, with the ANN model performing with greater sensitivity than the MLR. ANN models were able to illuminate site-specific relationships between microbial indicators chosen as model inputs and human virus presence. A validation study on ADV demonstrated that the MLR and ANN models differed in sensitivity and selectivity, with the ANN model correctly identifying ADV presence with greater precision.