Hybrid physics-based and data-driven modeling for bioprocess online simulation and optimization

Model-based online optimization has not been widely applied to bioprocesses due to the challenges of modeling complex biological behaviors, low-quality industrial measurements, and lack of visualization techniques for ongoing processes. This study proposes an innovative hybrid modeling framework which takes advantages of both physics-based and data-driven modeling for bioprocess online monitoring, prediction, and optimization. The framework initially generates high-quality data by correcting raw process measurements via a physics-based noise filter (a generally available simple kinetic model with high fitting but low predictive performance); then constructs a predictive data-driven model to identify optimal control actions and predict discrete future bioprocess behaviors. Continuous future process trajectories are subsequently visualized by re-fitting the simple kinetic model (soft sensor) using the data-driven model predicted discrete future data points, enabling the accurate monitoring of ongoing processes at any operating time. This framework was tested to maximize fed-batch microalgal lutein production by combining with different online optimization schemes and compared against the conventional open-loop optimization technique. The optimal results using the proposed framework were found to be comparable to the theoretically best production, demonstrating its high predictive and flexible capabilities as well as its potential for industrial application.     

A computational modeling and analysis in cell biological dynamics using electric cell-substrate impedance sensing (ECIS)

In this paper, a study of computational modeling and multi-scale analysis in cell dynamics is presented. Our study aims at: (1) deriving and validating a mathematical model for cell growth, and (2) quantitatively detecting and analyzing the biological interdependencies across multiple observational scales with a variety of time and frequency resolutions. This research was conducted using the time series data practically measured from a novel on-line cell monitoring technique, referred to as electric cell-substrate impedance sensing (ECIS), which allows continuously tracking the cellular behavior such as adhesion, proliferation, spreading and micromotion. First, comparing our ECIS-based cellular growth modeling analysis results with those determined by hematocytometer measurement using different time intervals, we found that the results obtained from both experimental methods consistently agreed. However, our study demonstrated that it is much easier and more convenient to operate with the ECIS system for on-line cellular growth monitoring. Secondly, for multi-scale analysis our results showed that the proposed wavelet-based methodology can effectively quantify the fluctuations associated with cell micromotions and quantitatively capture the biological interdependencies across multiple observational scales. Note that although the wavelet method is well known, its application into the ECIS time series analysis is novel and unprecedented in computational cell biology. Our analyses indicated that the proposed study on ECIS time series could provide a hopeful start and great potentials in both modeling and elucidating the complex mechanisms of cell biological systems.     

A general approach for two-stage analysis of multilevel clustered non-Gaussian data

In this article, we propose a two-stage approach to modeling multilevel clustered non-Gaussian data with sufficiently large numbers of continuous measures per cluster. Such data are common in biological and medical studies utilizing monitoring or image-processing equipment. We consider a general class of hierarchical models that generalizes the model in the global two-stage (GTS) method for nonlinear mixed effects models by using any square-root-n-consistent and asymptotically normal estimators from stage 1 as pseudodata in the stage 2 model, and by extending the stage 2 model to accommodate random effects from multiple levels of clustering. The second-stage model is a standard linear mixed effects model with normal random effects, but the cluster-specific distributions, conditional on random effects, can be non-Gaussian. This methodology provides a flexible framework for modeling not only a location parameter but also other characteristics of conditional distributions that may be of specific interest. For estimation of the population parameters, we propose a conditional restricted maximum likelihood (CREML) approach and establish the asymptotic properties of the CREML estimators. The proposed general approach is illustrated using quartiles as cluster-specific parameters estimated in the first stage, and applied to the data example from a collagen fibril development study. We demonstrate using simulations that in samples with small numbers of independent clusters, the CREML estimators may perform better than conditional maximum likelihood estimators, which are a direct extension of the estimators from the GTS method.     

Nonlinear modeling and adaptive monitoring with fuzzy and multivariate statistical methods in biological wastewater treatment plants

A new approach to nonlinear modeling and adaptive monitoring using fuzzy principal component regression (FPCR) is proposed and then applied to a real wastewater treatment plant (WWTP) data set. First, principal component analysis (PCA) is used to reduce the dimensionality of data and to remove collinearity. Second, the adaptive credibilistic fuzzy-c-means method is used to appropriately monitor diverse operating conditions based on the PCA score values. Then a new adaptive discrimination monitoring method is proposed to distinguish between a large process change and a simple fault. Third, a FPCR method is proposed, where the Takagi-Sugeno-Kang (TSK) fuzzy model is employed to model the relation between the PCA score values and the target output to avoid the over-fitting problem with original variables. Here, the rule bases, the centers and the widths of TSK fuzzy model are found by heuristic methods. The proposed FPCR method is applied to predict the output variable, the reduction of chemical oxygen demand in the full-scale WWTP. The result shows that it has the ability to model the nonlinear process and multiple operating conditions and is able to identify various operating regions and discriminate between a sustained fault and a simple fault (or abnormalities) occurring within the process data.     

Nonlinear biological batch process monitoring and fault identification based on kernel fisher discriminant analysis

A novel nonlinear biological batch process monitoring and fault identification approach based on kernel Fisher discriminant analysis (kernel FDA) is proposed. This method has a powerful ability to deal with nonlinear data and does not need to predict the future observations of variables. So it is more sensitive to fault detection. In order to improve the monitoring performance, variable trajectories of the batch processes are separated into several blocks. Then data in the original space is mapped into high-dimensional feature space via nonlinear kernel function and the optimal kernel Fisher feature vector and discriminant vector are extracted to perform process monitoring and fault identification. The key to the proposed approach is to calculate the distance of block data which are projected to the optimal kernel Fisher discriminant vector between new batch and reference batch. Through comparing distance with the predefined threshold, it can be considered whether the batch is normal or abnormal. Similar degree between the present discriminant vector and the optimal discriminant vector of fault in historical data set is used to perform fault diagnosis. The proposed method is applied to the process of fed-batch penicillin fermentation simulator benchmark and shows that it can effectively capture nonlinear relationships among process variables and is more efficient than MPCA approach.     

Explainable deep transfer learning model for disease risk prediction using high-dimensional genomic data

Building an accurate disease risk prediction model is an essential step in the modern quest for precision medicine. While high-dimensional genomic data provides valuable data resources for the investigations of disease risk, their huge amount of noise and complex relationships between predictors and outcomes have brought tremendous analytical challenges. Deep learning model is the state-of-the-art methods for many prediction tasks, and it is a promising framework for the analysis of genomic data. However, deep learning models generally suffer from the curse of dimensionality and the lack of biological interpretability, both of which have greatly limited their applications. In this work, we have developed a deep neural network (DNN) based prediction modeling framework. We first proposed a group-wise feature importance score for feature selection, where genes harboring genetic variants with both linear and non-linear effects are efficiently detected. We then designed an explainable transfer-learning based DNN method, which can directly incorporate information from feature selection and accurately capture complex predictive effects. The proposed DNN-framework is biologically interpretable, as it is built based on the selected predictive genes. It is also computationally efficient and can be applied to genome-wide data. Through extensive simulations and real data analyses, we have demonstrated that our proposed method can not only efficiently detect predictive features, but also accurately predict disease risk, as compared to many existing methods.     

Network-based auto-probit modeling for protein function prediction

Predicting the functional roles of proteins based on various genome-wide data, such as protein-protein association networks, has become a canonical problem in computational biology. Approaching this task as a binary classification problem, we develop a network-based extension of the spatial auto-probit model. In particular, we develop a hierarchical Bayesian probit-based framework for modeling binary network-indexed processes, with a latent multivariate conditional autoregressive Gaussian process. The latter allows for the easy incorporation of protein-protein association network topologies-either binary or weighted-in modeling protein functional similarity. We use this framework to predict protein functions, for functions defined as terms in the Gene Ontology (GO) database, a popular rigorous vocabulary for biological functionality. Furthermore, we show how a natural extension of this framework can be used to model and correct for the high percentage of false negative labels in training data derived from GO, a serious shortcoming endemic to biological databases of this type. Our method performance is evaluated and compared with standard algorithms on weighted yeast protein-protein association networks, extracted from a recently developed integrative database called Search Tool for the Retrieval of INteracting Genes/proteins (STRING). Results show that our basic method is competitive with these other methods, and that the extended method-incorporating the uncertainty in negative labels among the training data-can yield nontrivial improvements in predictive accuracy.     

Protein significance analysis in selected reaction monitoring (SRM) measurements

Selected reaction monitoring (SRM) is a targeted mass spectrometry technique that provides sensitive and accurate protein detection and quantification in complex biological mixtures. Statistical and computational tools are essential for the design and analysis of SRM experiments, particularly in studies with large sample throughput. Currently, most such tools focus on the selection of optimized transitions and on processing signals from SRM assays. Little attention is devoted to protein significance analysis, which combines the quantitative measurements for a protein across isotopic labels, peptides, charge states, transitions, samples, and conditions, and detects proteins that change in abundance between conditions while controlling the false discovery rate. We propose a statistical modeling framework for protein significance analysis. It is based on linear mixed-effects models and is applicable to most experimental designs for both isotope label-based and label-free SRM workflows. We illustrate the utility of the framework in two studies: one with a group comparison experimental design and the other with a time course experimental design. We further verify the accuracy of the framework in two controlled data sets, one from the NCI-CPTAC reproducibility investigation and the other from an in-house spike-in study. The proposed framework is sensitive and specific, produces accurate results in broad experimental circumstances, and helps to optimally design future SRM experiments. The statistical framework is implemented in an open-source R-based software package SRMstats, and can be used by researchers with a limited statistics background as a stand-alone tool or in integration with the existing computational pipelines.     

The clustering of regression models method with applications in gene expression data

Identification of differentially expressed genes and clustering of genes are two important and complementary objectives addressed with gene expression data. For the differential expression question, many "per-gene" analytic methods have been proposed. These methods can generally be characterized as using a regression function to independently model the observations for each gene; various adjustments for multiplicity are then used to interpret the statistical significance of these per-gene regression models over the collection of genes analyzed. Motivated by this common structure of per-gene models, we proposed a new model-based clustering method--the clustering of regression models method, which groups genes that share a similar relationship to the covariate(s). This method provides a unified approach for a family of clustering procedures and can be applied for data collected with various experimental designs. In addition, when combined with per-gene methods for assessing differential expression that employ the same regression modeling structure, an integrated framework for the analysis of microarray data is obtained. The proposed methodology was applied to two microarray data sets, one from a breast cancer study and the other from a yeast cell cycle study.     

Modeling of chromosome motility during mitosis

Chromosome motility is a highly regulated and complex process that ultimately achieves proper segregation of the replicated genome. Recent modeling studies provide a computational framework for investigating how microtubule assembly dynamics, motor protein activity and mitotic spindle mechanical properties are integrated to drive chromosome motility. Among other things, these studies show that metaphase chromosome oscillations can be explained by a range of assumptions, and that non-oscillatory states can be achieved with modest changes to the model parameters. In addition, recent microscopy studies provide new insight into the nature of the coupling between force on the kinetochore and kinetochore-microtubule assembly/disassembly. Together, these studies facilitate advancement toward a unified model that quantitatively predicts chromosome motility.     

Identifying miRNA-disease association based on integrating miRNA topological similarity and functional similarity

Background: MicroRNAs (miRNAs) are a significant type of non-coding RNAs, which usually were encoded by endogenous genes with about ~22 nt nucleotides. Accumulating biological experiments have shown that miRNAs have close associations with various human diseases. Although traditional experimental methods achieve great successes in miRNA-disease interaction identification, these methods also have some limitations. Therefore, it is necessary to develop computational method to predict miRNA-disease interactions. Methods: Here, we propose a computational framework (MDVSI) to predict interactions between miRNAs and diseases by integrating miRNA topological similarity and functional similarity. Firstly, the CosRA index is utilized to measure miRNA similarity based on network topological feature. Then, in order to enhance the reliability of miRNA similarity, the functional similarity and CosRA similarity are integrated based on linear weight method. Further, the potential miRNA-disease associations are predicted by using recommendation method. In addition, in order to overcome limitation of recommendation method, for new disease, a new strategy is proposed to predict potential interactions between miRNAs and new disease based on disease functional similarity. Results: To evaluate the performance of different methods, we conduct ten-fold cross validation and de novo test in experiment and compare MDVSI with two the-state-of-art methods. The experimental result shows that MDVSI achieves an AUC of 0.91, which is at least 0.012 higher than other compared methods. Conclusions: In summary, we propose a computational framework (MDSVI) for miRNA-disease interaction prediction. The experiment results demonstrate that it outperforms other the-state-of-the-art methods. Case study shows that it can effectively identify potential miRNA-disease interactions.     

A connectionist model of development.

We present a phenomenological modeling framework for development. Our purpose is to provide a systematic method for discovering and expressing correlations in experimental data on gene expression and other developmental processes. The modeling framework is based on a connectionist or "neural net" dynamics for biochemical regulators, coupled to "grammatical rules" which describe certain features of the birth, growth, and death of cells, synapses and other biological entities. We outline how spatial geometry can be included, although this part of the model is not complete. As an example of the application of our results to a specific biological system, we show in detail how to derive a rigorously testable model of the network of segmentation genes operating in the blastoderm of Drosophila. To further illustrate our methods, we sketch how they could be applied to two other important developmental processes: cell cycle control and cell-cell induction. We also present a simple biochemical model leading to our assumed connectionist dynamics which shows that the dynamics used is at least compatible with known chemical mechanisms.     

Intelligent real-time performance monitoring and quality prediction for batch/fed-batch cultivations

Supervision of batch bioprocess operations in real-time during the progress of a batch run offers many advantages over end-of-batch quality control. Multivariate statistical techniques such as multiway partial least squares (MPLS) provide an efficient modeling and supervision framework. A new type of MPLS modeling technique that is especially suitable for online real-time process monitoring and the multivariate monitoring charts are presented. This online process monitoring technique is also extended to include predictions of end-of-batch quality measurements during the progress of a batch run. Process monitoring, quality estimation and fault diagnosis activities are automated and supervised by embedding them into a real-time knowledge-based system (RTKBS). Interpretation of multivariate charts is also automated through a generic rule-base for efficient alarm handling. The integrated RTKBS and the implementation of MPLS-based process monitoring and quality control are illustrated using a fed-batch penicillin production benchmark process simulator.     

Mathematical modeling as a tool for investigating cell cycle control networks

Although not a traditional experimental "method," mathematical modeling can provide a powerful approach for investigating complex cell signaling networks, such as those that regulate the eukaryotic cell division cycle. We describe here one modeling approach based on expressing the rates of biochemical reactions in terms of nonlinear ordinary differential equations. We discuss the steps and challenges in assigning numerical values to model parameters and the importance of experimental testing of a mathematical model. We illustrate this approach throughout with the simple and well-characterized example of mitotic cell cycles in frog egg extracts. To facilitate new modeling efforts, we describe several publicly available modeling environments, each with a collection of integrated programs for mathematical modeling. This review is intended to justify the place of mathematical modeling as a standard method for studying molecular regulatory networks and to guide the non-expert to initiate modeling projects in order to gain a systems-level perspective for complex control systems.     

森林-农业景观格局变化的社会-生态系统模拟模型方法进展

社会-生态系统（SES）模拟模型是景观格局分析和决策的有效工具,能表征景观格局变化的社会-生态效应及景观决策的复杂反馈机制。文献综述了森林-农业景观格局的SES模型方法进展发现：（1）多数模型对景观过程与社会经济决策的反馈关系分析不足;（2）应集成多种情景模拟和景观效应分析方法,完善现有SES模型的理论方法基础;（3）通过集成格局优化模型和自主体模型会有效改进SES模型功能,具体途径包括：集成情景-生态效应的景观格局模拟方法、完善景观决策的理论基础、加强集成模型的不确定性分析、降低模型复杂性和综合定性-定量数据等。研究结果有助于理解多尺度森林-农业景观格局在社会-生态系统中的重要作用,能更好地支持跨学科集成模型开发与应用。     

Differential equation modeling of HIV viral fitness experiments: model identification, model selection, and multimodel inference

Summary .  Many biological processes and systems can be described by a set of differential equation (DE) models. However, literature in statistical inference for DE models is very sparse. We propose statistical estimation, model selection, and multimodel averaging methods for HIV viral fitness experiments in vitro that can be described by a set of nonlinear ordinary differential equations (ODE). The parameter identifiability of the ODE models is also addressed. We apply the proposed methods and techniques to experimental data of viral fitness for HIV-1 mutant 103N. We expect that the proposed modeling and inference approaches for the DE models can be widely used for a variety of biomedical studies.