Comparison of Arrhenius-type and Bêlehrádek-type models for prediction of bacterial growth in foods

D.A. RATKOWSKY, T. ROSS, T.A. WCMEEKIN AND J. OLLEY. 1991. The development of Arrhenius-type ('Schoolfield') and Bêlehrádek-type (square root) models that describe microbial growth rates is briefly described. Both types of model have been advocated for use in predictive microbiology. On the basis of published data sets for the growth of bacteria, the consequences of mathematical transformation of data and the use of invalid stochastic assumptions upon model predictions are demonstrated. Mean square error is shown to be an inappropriate criterion by which to compare the performance of predictive models. The data show that bacterial growth responses such as generation time and lag time become more variable as their mean magnitude increases. The practical consequences of such variability for predictive microbiology are discussed.     

食品微生物生长预测模型研究新进展

为了更好的了解食品微生物学预测模型的基本内容,探讨数学模型在预测微生物学中的作用,达到提高食品卫生检测效率,保证食品质量安全的目的,本文以文献综述形式,简要概述了预测微生物学一级、二级和三级模型的含义与内容。并在此基础上,着重介绍了全球范围内已经成功推广使用的多种三级模型,阐述了它们的研究背景、研究进展、使用方法,分析了各种模型存在的优缺点,可为实际应用中选择合适的模型提供参考。在比较使用不同种类模型后,发现Baranyi＆Roberts、响应面和ComBase模型在各级模型中具有更好的使用价值。     

食品微生物生长预测模型研究新进展

为了更好的了解食品微生物学预测模型的基本内容, 探讨数学模型在预测微生物学中的作用, 达到提高食品卫生检测效率, 保证食品质量安全的目的, 本文以文献综述形式, 简要概述了预测微生物学一级、二级和三级模型的含义与内容。并在此基础上, 着重介绍了全球范围内已经成功推广使用的多种三级模型, 阐述了它们的研究背景、研究进展、使用方法, 分析了各种模型存在的优缺点, 可为实际应用中选择合适的模型提供参考。在比较使用不同种类模型后, 发现Baranyi & Roberts、响应面和ComBase模型在各级模型中具有更好的使用价值。     

A prototype model structure for mixed microbial populations in homogeneous food products

An important factor which has not been included in many models in the field of predictive microbiology is the influence of a background of microflora in a food product. It is however generally known that the growth of a microorganism as a pure culture can be substantially different from its growth in a mixed culture, due to microbial interactions. Because of the importance of these interactions and the lack of suitable modeling techniques in the field of predictive microbiology to describe them, the potential of models in other research fields-namely ecology-to deal with interactions is explored in previous work of the authors. However, a model structure for microbial growth in food products cannot simply be copied from those elaborated in ecology. The structure of a predictive growth model is indeed typical, primarily due to the explicit modeling of a lag phase. The current paper proposes a prototype model structure for growth of mixed microbial populations in homogeneous food products. The model is able to describe a lag phase and reduces to a classical predictive growth model in the special case of single-species growth.     

Improved prediction of smoking status via isoform-aware RNA-seq deep learning models

Most predictive models based on gene expression data do not leverage information related to gene splicing, despite the fact that splicing is a fundamental feature of eukaryotic gene expression. Cigarette smoking is an important environmental risk factor for many diseases, and it has profound effects on gene expression. Using smoking status as a prediction target, we developed deep neural network predictive models using gene, exon, and isoform level quantifications from RNA sequencing data in 2,557 subjects in the COPDGene Study. We observed that models using exon and isoform quantifications clearly outperformed gene-level models when using data from 5 genes from a previously published prediction model. Whereas the test set performance of the previously published model was 0.82 in the original publication, our exon-based models including an exon-to-isoform mapping layer achieved a test set AUC (area under the receiver operating characteristic) of 0.88, which improved to an AUC of 0.94 using exon quantifications from a larger set of genes. Isoform variability is an important source of latent information in RNA-seq data that can be used to improve clinical prediction models.     

Bělehrádek-type models

Summary Square-root (or Ratkowsky) models are a special case of Blehrádek's temperature rate-relationship first published in 1926 and widely used in several fields of biology. Blehrádek-type models also describe microbial growth, and have been extended for use in food microbiology by the inclusion of terms for water activity and pH. The parameters of the square root-type models are defined and their determination described. Favorable features of square root-type models include parsimony, parameter estimation properties, and ease of use. Square root-type models have been developed for a number of organisms of concern to the food industry and have also been adopted for use in a number of electronic devices used in predictive microbiology. Criticisms of square root-type models are also considered.Mention of brand or firm names does not constitute an endorsement by the US Department of Agriculture over others of a similar nature not mentioned.     

Modelling the bacterial growth/no growth interface

A logistic regression model is proposed which enables one to model the boundary between growth and no growth for bacterial strains in the presence of one or more growth controlling factors such as temperature, pH and additives such as salt and sodium nitrite. The form of the expression containing the growth limiting factors may be suggested by a kinetic model, while the response at a given combination of factors may either be presence/absence (i.e. growth/no growth) or probabilistic (i.e. r successes in n trials). The approach described represents an integration of the probability and kinetic aspects of predictive microbiology, and a unification of predictive microbiology and the hurdle concept. The model is illustrated using data for Shigella flexneri.     

Modelling the growth,survival and death of Listeria monocytogenes

In this paper, the predictive microbiology approach has been generalized to the study of growth, survival and death of Listeria monocytogenes. As this micro-organism is involved in food poisoning, its growth, survival and death were studied as functions of low temperatures, NaCl and phenol compounds, in a synthetic medium, by a factorially designed experiment. A significant inactivation of L. monocytogenes was obtained with 20 ppm of phenol and 4% (w/v) NaCl at temperatures from 4 to 12 degrees C. An empirical model is proposed to describe, in a single step, the biomass profile vs studied factors. Thereby, the influence of temperature, NaCl and phenol concentration on L. monocytogenes biomass quantity (0.5-8 log cfu ml(-1)) are presented as a function of storage duration. The comparisons of the proposed model with existing models (Gompertz for growth, vitalistic for survival and death) were performed. The use of a single equation allows the prediction of contamination levels in all experimental conditions without knowledge a priori. The model offers considerable prospects for its use in food microbiology.     

Predicting germination response to temperature. III. Model validation under field-variable temperature conditions

BACKGROUND AND AIMS: Two previous papers in this series evaluated model fit of eight thermal-germination models parameterized from constant-temperature germination data. The previous studies determined that model formulations with the fewest shape assumptions provided the best estimates of both germination rate and germination time. The purpose of this latest study was to evaluate the accuracy and efficiency of these same models in predicting germination time and relative seedlot performance under field-variable temperature scenarios. METHODS: The seeds of four rangeland grass species were germinated under 104 variable-temperature treatments simulating six planting dates at three field sites in south-western Idaho. Measured and estimated germination times for all subpopulations were compared for all models, species and temperature treatments. KEY RESULTS: All models showed similar, and relatively high, predictive accuracy for field-temperature simulations except for the iterative-probit-optimization (IPO) model, which exhibited systematic errors as a function of subpopulation. Highest efficiency was obtained with the statistical-gridding (SG) model, which could be directly parameterized by measured subpopulation rate data. Relative seedlot response predicted by thermal time coefficients was somewhat different from that estimated from mean field-variable temperature response as a function of subpopulation. CONCLUSIONS: All germination response models tested performed relatively well in estimating field-variable temperature response. IPO caused systematic errors in predictions of germination time, and may have degraded the physiological relevance of resultant cardinal-temperature parameters. Comparative indices based on expected field performance may be more ecologically relevant than indices derived from a broader range of potential thermal conditions.     

Development and validation of experimental protocols for use of cardinal models for prediction of microorganism growth in food products

An experimental protocol to validate secondary-model application to foods was suggested. Escherichia coli, Listeria monocytogenes, Bacillus cereus, Clostridium perfringens, and Salmonella were observed in various food categories, such as meat, dairy, egg, or seafood products. The secondary model validated in this study was based on the gamma concept, in which the environmental factors temperature, pH, and water activity (aw) were introduced as individual terms with microbe-dependent parameters, and the effect of foodstuffs on the growth rates of these species was described with a food- and microbe-dependent parameter. This food-oriented approach was carried out by challenge testing, generally at 15 and 10 degrees C for L. monocytogenes, E. coli, B. cereus, and Salmonella and at 25 and 20 degrees C for C. perfringens. About 222 kinetics in foods were generated. The results were compared to simulations generated by existing software, such as PMP. The bias factor was also calculated. The methodology to obtain a food-dependent parameter (fitting step) and therefore to compare results given by models with new independent data (validation step) is discussed in regard to its food safety application. The proposed methods were used within the French national program of predictive microbiology, Sym'Previus, to include challenge test results in the database and to obtain predictive models designed for microbial growth in food products.     

Bayesian model adequacy and choice in phylogenetics

Bayesian inference is becoming a common statistical approach to phylogenetic estimation because, among other reasons, it allows for rapid analysis of large data sets with complex evolutionary models. Conveniently, Bayesian phylogenetic methods use currently available stochastic models of sequence evolution. However, as with other model-based approaches, the results of Bayesian inference are conditional on the assumed model of evolution: inadequate models (models that poorly fit the data) may result in erroneous inferences. In this article, I present a Bayesian phylogenetic method that evaluates the adequacy of evolutionary models using posterior predictive distributions. By evaluating a model's posterior predictive performance, an adequate model can be selected for a Bayesian phylogenetic study. Although I present a single test statistic that assesses the overall (global) performance of a phylogenetic model, a variety of test statistics can be tailored to evaluate specific features (local performance) of evolutionary models to identify sources failure. The method presented here, unlike the likelihood-ratio test and parametric bootstrap, accounts for uncertainty in the phylogeny and model parameters.     

Comparison of logistic regression and linear regression in modeling percentage data

Percentage is widely used to describe different results in food microbiology, e.g., probability of microbial growth, percent inactivated, and percent of positive samples. Four sets of percentage data, percent-growth-positive, germination extent, probability for one cell to grow, and maximum fraction of positive tubes, were obtained from our own experiments and the literature. These data were modeled using linear and logistic regression. Five methods were used to compare the goodness of fit of the two models: percentage of predictions closer to observations, range of the differences (predicted value minus observed value), deviation of the model, linear regression between the observed and predicted values, and bias and accuracy factors. Logistic regression was a better predictor of at least 78% of the observations in all four data sets. In all cases, the deviation of logistic models was much smaller. The linear correlation between observations and logistic predictions was always stronger. Validation (accomplished using part of one data set) also demonstrated that the logistic model was more accurate in predicting new data points. Bias and accuracy factors were found to be less informative when evaluating models developed for percentage data, since neither of these indices can compare predictions at zero. Model simplification for the logistic model was demonstrated with one data set. The simplified model was as powerful in making predictions as the full linear model, and it also gave clearer insight in determining the key experimental factors.     

Comparison of Logistic Regression and Linear Regression in Modeling Percentage Data

Percentage is widely used to describe different results in food microbiology, e.g., probability of microbial growth, percent inactivated, and percent of positive samples. Four sets of percentage data, percent-growth-positive, germination extent, probability for one cell to grow, and maximum fraction of positive tubes, were obtained from our own experiments and the literature. These data were modeled using linear and logistic regression. Five methods were used to compare the goodness of fit of the two models: percentage of predictions closer to observations, range of the differences (predicted value minus observed value), deviation of the model, linear regression between the observed and predicted values, and bias and accuracy factors. Logistic regression was a better predictor of at least 78% of the observations in all four data sets. In all cases, the deviation of logistic models was much smaller. The linear correlation between observations and logistic predictions was always stronger. Validation (accomplished using part of one data set) also demonstrated that the logistic model was more accurate in predicting new data points. Bias and accuracy factors were found to be less informative when evaluating models developed for percentage data, since neither of these indices can compare predictions at zero. Model simplification for the logistic model was demonstrated with one data set. The simplified model was as powerful in making predictions as the full linear model, and it also gave clearer insight in determining the key experimental factors.     

Comparison of artificial neural network and logistic regression models for predicting in-hospital mortality after primary liver cancer surgery

Background

Since most published articles comparing the performance of artificial neural network (ANN) models and logistic regression (LR) models for predicting hepatocellular carcinoma (HCC) outcomes used only a single dataset, the essential issue of internal validity (reproducibility) of the models has not been addressed. The study purposes to validate the use of ANN model for predicting in-hospital mortality in HCC surgery patients in Taiwan and to compare the predictive accuracy of ANN with that of LR model.

Methodology/Principal Findings

Patients who underwent a HCC surgery during the period from 1998 to 2009 were included in the study. This study retrospectively compared 1,000 pairs of LR and ANN models based on initial clinical data for 22,926 HCC surgery patients. For each pair of ANN and LR models, the area under the receiver operating characteristic (AUROC) curves, Hosmer-Lemeshow (H-L) statistics and accuracy rate were calculated and compared using paired T-tests. A global sensitivity analysis was also performed to assess the relative significance of input parameters in the system model and the relative importance of variables. Compared to the LR models, the ANN models had a better accuracy rate in 97.28% of cases, a better H-L statistic in 41.18% of cases, and a better AUROC curve in 84.67% of cases. Surgeon volume was the most influential (sensitive) parameter affecting in-hospital mortality followed by age and lengths of stay.

Conclusions/Significance

In comparison with the conventional LR model, the ANN model in the study was more accurate in predicting in-hospital mortality and had higher overall performance indices. Further studies of this model may consider the effect of a more detailed database that includes complications and clinical examination findings as well as more detailed outcome data.     

Development of CYP3A4 inhibition models: comparisons of machine-learning techniques and molecular descriptors

Computational models of cytochrome P450 3A4 inhibition were developed based on high-throughput screening data for 4470 proprietary compounds. Multiple models differentiating inhibitors (IC(50) <3 microM) and noninhibitors were generated using various machine-learning algorithms (recursive partitioning [RP], Bayesian classifier, logistic regression, k-nearest-neighbor, and support vector machine [SVM]) with structural fingerprints and topological indices. Nineteen models were evaluated by internal 10-fold cross-validation and also by an independent test set. Three most predictive models, Barnard Chemical Information (BCI)-fingerprint/SVM, MDL-keyset/SVM, and topological indices/RP, correctly classified 249, 248, and 236 compounds of 291 noninhibitors and 135, 137, and 147 compounds of 179 inhibitors in the validation set. Their overall accuracies were 82%, 82%, and 81%, respectively. Investigating applicability of the BCI/SVM model found a strong correlation between the predictive performance and the structural similarity to the training set. Using Tanimoto similarity index as a confidence measurement for the predictions, the limitation of the extrapolation was 0.7 in the case of the BCI/SVM model. Taking consensus of the 3 best models yielded a further improvement in predictive capability, kappa = 0.65 and accuracy = 83%. The consensus model could also be tuned to minimize either false positives or false negatives depending on the emphasis of the screening.     

Developing risk models for multicenter data using standard logistic regression produced suboptimal predictions: A simulation study

Although multicenter data are common, many prediction model studies ignore this during model development. The objective of this study is to evaluate the predictive performance of regression methods for developing clinical risk prediction models using multicenter data, and provide guidelines for practice. We compared the predictive performance of standard logistic regression, generalized estimating equations, random intercept logistic regression, and fixed effects logistic regression. First, we presented a case study on the diagnosis of ovarian cancer. Subsequently, a simulation study investigated the performance of the different models as a function of the amount of clustering, development sample size, distribution of center-specific intercepts, the presence of a center-predictor interaction, and the presence of a dependency between center effects and predictors. The results showed that when sample sizes were sufficiently large, conditional models yielded calibrated predictions, whereas marginal models yielded miscalibrated predictions. Small sample sizes led to overfitting and unreliable predictions. This miscalibration was worse with more heavily clustered data. Calibration of random intercept logistic regression was better than that of standard logistic regression even when center-specific intercepts were not normally distributed, a center-predictor interaction was present, center effects and predictors were dependent, or when the model was applied in a new center. Therefore, to make reliable predictions in a specific center, we recommend random intercept logistic regression.     

Modeling Dynamic Systems with Efficient Ensembles of Process-Based Models

Ensembles are a well established machine learning paradigm, leading to accurate and robust models, predominantly applied to predictive modeling tasks. Ensemble models comprise a finite set of diverse predictive models whose combined output is expected to yield an improved predictive performance as compared to an individual model. In this paper, we propose a new method for learning ensembles of process-based models of dynamic systems. The process-based modeling paradigm employs domain-specific knowledge to automatically learn models of dynamic systems from time-series observational data. Previous work has shown that ensembles based on sampling observational data (i.e., bagging and boosting), significantly improve predictive performance of process-based models. However, this improvement comes at the cost of a substantial increase of the computational time needed for learning. To address this problem, the paper proposes a method that aims at efficiently learning ensembles of process-based models, while maintaining their accurate long-term predictive performance. This is achieved by constructing ensembles with sampling domain-specific knowledge instead of sampling data. We apply the proposed method to and evaluate its performance on a set of problems of automated predictive modeling in three lake ecosystems using a library of process-based knowledge for modeling population dynamics. The experimental results identify the optimal design decisions regarding the learning algorithm. The results also show that the proposed ensembles yield significantly more accurate predictions of population dynamics as compared to individual process-based models. Finally, while their predictive performance is comparable to the one of ensembles obtained with the state-of-the-art methods of bagging and boosting, they are substantially more efficient.     

Developing and distributing user-friendly application software

Summary The adoption of new techniques in predictive microbiology by the food industry will ultimately be dependent on the development of user-friendly application software that makes it easy for non-research personnel to employ the mathematical models. Such applications should be an integral part of projects in predictive microbiology. Recommendations related to the architecture, speed, protection, testing, and distribution of application software are presented based on our experience in developing and distributing the Microbial Food Safety Pathogen Modeling Program.Reference to a brand or firm name does not constitute an endorsement by the US Department of Agriculture over others of a similar nature not mentioned.     

Validation of species–climate impact models under climate change

Increasing concern over the implications of climate change for biodiversity has led to the use of species–climate envelope models to project species extinction risk under climate‐change scenarios. However, recent studies have demonstrated significant variability in model predictions and there remains a pressing need to validate models and to reduce uncertainties. Model validation is problematic as predictions are made for events that have not yet occurred. Resubstituition and data partitioning of present‐day data sets are, therefore, commonly used to test the predictive performance of models. However, these approaches suffer from the problems of spatial and temporal autocorrelation in the calibration and validation sets. Using observed distribution shifts among 116 British breeding‐bird species over the past ～20 years, we are able to provide a first independent validation of four envelope modelling techniques under climate change. Results showed good to fair predictive performance on independent validation, although rules used to assess model performance are difficult to interpret in a decision‐planning context. We also showed that measures of performance on nonindependent data provided optimistic estimates of models' predictive ability on independent data. Artificial neural networks and generalized additive models provided generally more accurate predictions of species range shifts than generalized linear models or classification tree analysis. Data for independent model validation and replication of this study are rare and we argue that perfect validation may not in fact be conceptually possible. We also note that usefulness of models is contingent on both the questions being asked and the techniques used. Implementations of species–climate envelope models for testing hypotheses and predicting future events may prove wrong, while being potentially useful if put into appropriate context.     

Choosing the best method for stream bioassessment using macrophyte communities: Indices and predictive models

The bioassessment and monitoring of the ecological status of rivers using macrophytes has gained new momentum since macrophytes were recognised as biological quality elements for the implementation of the European Water Framework Directive (WFD; EU/2000/60).Our objectives were to test the suitability of two predictive modelling approaches to macrophyte communities as a tool for water quality assessment, and to compare their performance with other more common approaches—the use of macrophytes as indicators of the trophic status of rivers and multimetric indices. We used floristic and environmental data that were collected in the spring of 2004 and 2005 from around 400 sites on rivers across mainland Portugal, western Iberia.We build two predictive models: MACPACS (MACrophyte Prediction And Classification System) and MAC (Macrophyte Assessment and Classification) based on RIVPACS and the BEAST methods, respectively. Whereas MACPACS is derived from taxa occurrence data, MAC uses a quantitative measure of taxa abundance. Both models showed good performance in predicting reference sites to the correct group and low rate of misclassification errors. However, they performed differently. MAC depicts a reliable response to the overall human-mediated degradation of fluvial systems, as does the multimetric index (RVI, Riparian Vegetation Index), but MACPACS presented only a poor correlation with the Global Human Disturbance Index and with the nutrients input. The incorporation of abundance data in vegetation predictive models appears to be particularly important to the detection of high levels of degradation. The values for correlations with physical–chemical pressure variables were lower than expected for MTR (Mean Trophic Rank) due to an insufficient number of scoring species found in Portuguese fluvial systems. Our results suggest that the most effective methods for bioassessment in Mediterranean-type rivers are either the RVI or the MAC predictive model.