A bootstrap-aggregated hybrid semi-parametric modeling framework for bioprocess development

Bioprocess and Biosystems Engineering - Hybrid semi-parametric modeling, combining mechanistic and machine-learning methods, has proven to be a powerful method for process development. This paper...     

Combining model structure identification and hybrid modelling for photo-production process predictive simulation and optimisation

Integrating physical knowledge and machine learning is a critical aspect of developing industrially focused digital twins for monitoring, optimisation, and design of microalgal and cyanobacterial photo-production processes. However, identifying the correct model structure to quantify the complex biological mechanism poses a severe challenge for the construction of kinetic models, while the lack of data due to the time-consuming experiments greatly impedes applications of most data-driven models. This study proposes the use of an innovative hybrid modelling approach that consists of a simple kinetic model to govern the overall process dynamic trajectory and a data-driven model to estimate mismatch between the kinetic equations and the real process. An advanced automatic model structure identification strategy is adopted to simultaneously identify the most physically probable kinetic model structure and minimum number of data-driven model parameters that can accurately represent multiple data sets over a broad spectrum of process operating conditions. Through this hybrid modelling and automatic structure identification framework, a highly accurate mathematical model was constructed to simulate and optimise an algal lutein production process. Performance of this hybrid model for long-term predictive modelling, optimisation, and online self-calibration is demonstrated and thoroughly discussed, indicating its significant potential for future industrial application.     

Hybrid physics-based and data-driven modeling for bioprocess online simulation and optimization

Model-based online optimization has not been widely applied to bioprocesses due to the challenges of modeling complex biological behaviors, low-quality industrial measurements, and lack of visualization techniques for ongoing processes. This study proposes an innovative hybrid modeling framework which takes advantages of both physics-based and data-driven modeling for bioprocess online monitoring, prediction, and optimization. The framework initially generates high-quality data by correcting raw process measurements via a physics-based noise filter (a generally available simple kinetic model with high fitting but low predictive performance); then constructs a predictive data-driven model to identify optimal control actions and predict discrete future bioprocess behaviors. Continuous future process trajectories are subsequently visualized by re-fitting the simple kinetic model (soft sensor) using the data-driven model predicted discrete future data points, enabling the accurate monitoring of ongoing processes at any operating time. This framework was tested to maximize fed-batch microalgal lutein production by combining with different online optimization schemes and compared against the conventional open-loop optimization technique. The optimal results using the proposed framework were found to be comparable to the theoretically best production, demonstrating its high predictive and flexible capabilities as well as its potential for industrial application.     

A hybrid mechanistic-empirical model for in silico mammalian cell bioprocess simulation

In cell culture processes cell growth and metabolism drive changes in the chemical environment of the culture. These environmental changes elicit reactor control actions, cell growth response, and are sensed by cell signaling pathways that influence metabolism. The interplay of these forces shapes the culture dynamics through different stages of cell cultivation and the outcome greatly affects process productivity, product quality, and robustness. Developing a systems model that describes the interactions of those major players in the cell culture system can lead to better process understanding and enhance process robustness. Here we report the construction of a hybrid mechanistic-empirical bioprocess model which integrates a mechanistic metabolic model with subcomponent models for cell growth, signaling regulation, and the bioreactor environment for in silico exploration of process scenarios. Model parameters were optimized by fitting to a dataset of cell culture manufacturing process which exhibits variability in metabolism and productivity. The model fitting process was broken into multiple steps to mitigate the substantial numerical challenges related to the first-principles model components. The optimized model captured the dynamics of metabolism and the variability of the process runs with different kinetic profiles and productivity. The variability of the process was attributed in part to the metabolic state of cell inoculum. The model was then used to identify potential mitigation strategies to reduce process variability by altering the initial process conditions as well as to explore the effect of changing CO₂ removal capacity in different bioreactor scales on process performance. By incorporating a mechanistic model of cell metabolism and appropriately fitting it to a large dataset, the hybrid model can describe the different metabolic phases in culture and the variability in manufacturing runs. This approach of employing a hybrid model has the potential to greatly facilitate process development and reactor scaling.     

A hybrid agent-based approach for modeling microbiological systems

Models for systems biology commonly adopt Differential Equations or Agent-Based modeling approaches for simulating the processes as a whole. Models based on differential equations presuppose phenomenological intracellular behavioral mechanisms, while models based on Multi-Agent approach often use directly translated, and quantitatively less precise if-then logical rule constructs. We propose an extendible systems model based on a hybrid agent-based approach where biological cells are modeled as individuals (agents) while molecules are represented by quantities. This hybridization in entity representation entails a combined modeling strategy with agent-based behavioral rules and differential equations, thereby balancing the requirements of extendible model granularity with computational tractability. We demonstrate the efficacy of this approach with models of chemotaxis involving an assay of 10³ cells and 1.2×10⁶ molecules. The model produces cell migration patterns that are comparable to laboratory observations.     

A new approach for modeling cellulase-cellulose adsorption and the kinetics of the enzymatic hydrolysis of microcrystalline cellulose

Two fractions of substrate in microcrystalline cellulose which differ in their adsorption capacities for the cellulases and their susceptibility to enzymatic attack have been identified. On the basis of a two-substrate hypothesis, mathematical models to describe enzyme adsorption and the kinetics of hydrolysis have been derived. A new nonequilibrium approach was chosen to predict cellulase-cellulose adsorption. A maximum binding capacity of 76 mg protein per gram substrate and a half-maximum saturation constant of 26 filter paper units (FPU) per gram substrate have been calculated, and a linear relationship of hydrolysis rate vs. adsorbed protein has been found. The fraction of substrate more easily hydrolyzed, as calculated from hydrolysis data, represents 19% of the total effective substrate concentration. This fraction is only slightly different from that of other celluloses and has been estimated to be 27% and 30% for NaOH- and H(3)PO(4)-swollen cellulose, respectively. The effective substrate concentration is equal to the maximum amount of the substrate which can be converted during exhaustive hydrolysis. This in turn is determined by the overall degradability of the substrate by the cellulases (85-90% for microcrystalline cellulose) and by the cellobiose concentration during hydrolysis. The kinetic model is based on a summation of two integrated first-order reactions with respect to the effective substrate concentration. Furthermore, it includes the principal factors influencing the reaction rates: the ratio of filter paper and beta-glucosidase units per gram substrate and the initial substrate concentration. (c) 1993 John Wiley & Sons, Inc.     

COLLAPSE: A representation learning framework for identification and characterization of protein structural sites

The identification and characterization of the structural sites which contribute to protein function are crucial for understanding biological mechanisms, evaluating disease risk, and developing targeted therapies. However, the quantity of known protein structures is rapidly outpacing our ability to functionally annotate them. Existing methods for function prediction either do not operate on local sites, suffer from high false positive or false negative rates, or require large site‐specific training datasets, necessitating the development of new computational methods for annotating functional sites at scale. We present COLLAPSE (Compressed Latents Learned from Aligned Protein Structural Environments), a framework for learning deep representations of protein sites. COLLAPSE operates directly on the 3D positions of atoms surrounding a site and uses evolutionary relationships between homologous proteins as a self‐supervision signal, enabling learned embeddings to implicitly capture structure–function relationships within each site. Our representations generalize across disparate tasks in a transfer learning context, achieving state‐of‐the‐art performance on standardized benchmarks (protein–protein interactions and mutation stability) and on the prediction of functional sites from the prosite database. We use COLLAPSE to search for similar sites across large protein datasets and to annotate proteins based on a database of known functional sites. These methods demonstrate that COLLAPSE is computationally efficient, tunable, and interpretable, providing a general‐purpose platform for computational protein analysis.     

A hybrid systems framework for cellular processes

With the availability of technologies that allow us to obtain stimulus-response time series data for modeling and system identification, there is going to be an increasing need for conceptual frameworks in which to formulate and test hypotheses about intra- and inter-cellular dynamics, in general and not just dependent on a particular cell line, cell type, organism, or technology. While the semantics can be quite different, biologists and systems scientists use in many cases a similar language (notion of feedback, regulation, etc.). A more abstract system-theoretic framework for signals, systems, and control could provide the biologist with an interface between the domains. Apart from recent examples to identify functional elements and describing them in engineering terms, there have been various more abstract developments to describe dynamics at the cell level in the past. This includes Rosen's (M,R)-systems. This paper presents an abstract and general compact mathematical framework of intracellular dynamics, regulation and regime switching inspired by (M,R)-theory and based on hybrid automata.     

Hybrid modeling framework for process analytical technology: application to Bordetella pertussis cultures

In the process analytical technology (PAT) initiative, the application of sensors technology and modeling methods is promoted. The emphasis is on Quality by Design, online monitoring, and closed-loop control with the general aim of building in product quality into manufacturing operations. As a result, online high-throughput process analyzers find increasing application and therewith high amounts of highly correlated data become available online. In this study, an hybrid chemometric/mathematical modeling method is adopted for data analysis, which is shown to be advantageous over the commonly used chemometric techniques in PAT applications. This methodology was applied to the analysis of process data of Bordetella pertussis cultivations, namely online data of near-infrared, (NIR), pH, temperature and dissolved oxygen, and off-line data of biomass, glutamate, and lactate concentrations. The hybrid model structure consisted of macroscopic material balance equations in which the specific reactions rates are modeled by nonlinear partial least square (PLS). This methodology revealed a significant higher statistical confidence in comparison to PLSs, translated in a reduction of mean squared prediction errors (e.g., individual root mean squared prediction errors calibration/validation obtained through the hybrid model for the concentrations of lactate: 0.8699/0.7190 mmol/L; glutamate: 0.6057/0.2917 mmol/L; and biomass: 0.0520/0.0283 OD; and obtained through the PLS model for the concentrations of lactate: 1.3549/1.0087 mmol/L; glutamate: 0.7628/0.3504 mmol/L; and biomass: 0.0949/0.0412 OD). Moreover, the analysis of loadings and scores in the hybrid approach revealed that process features can, as for PLS, be extracted by the hybrid method.     

Dynamical model development and parameter identification for an anaerobic wastewater treatment process.

This paper deals with the development and the parameter identification of an anaerobic digestion process model. A two-step (acidogenesis-methanization) mass-balance model has been considered. The model incorporates electrochemical equilibria in order to include the alkalinity, which has to play a central role in the related monitoring and control strategy of a treatment plant. The identification is based on a set of dynamical experiments designed to cover a wide spectrum of operating conditions that are likely to take place in the practical operation of the plant. A step by step identification procedure to estimate the model parameters is presented. The results of 70 days of experiments in a 1-m(3) fermenter are then used to validate the model.     

A stochastic formulation of the gompertzian growth model for in vitro bactericidal kinetics: parameter estimation and extinction probability

Time-kill curves have frequently been employed to study the antimicrobial effects of antibiotics. The relevance of pharmacodynamic modeling to these investigations has been emphasized in many studies of bactericidal kinetics. Stochastic models are needed that take into account the randomness of the mechanisms of both bacterial growth and bacteria-drug interactions. However, most of the models currently used to describe antibiotic activity against microorganisms are deterministic. In this paper we examine a stochastic differential equation representing a stochastic version of a pharmacodynamic model of bacterial growth undergoing random fluctuations, and derive its solution, mean value and covariance structure. An explicit likelihood function is obtained both when the process is observed continuously over a period of time and when data is sampled at time points, as is the custom in these experimental conditions. Some asymptotic properties of the maximum likelihood estimators for the model parameters are discussed. The model is applied to analyze in vitro time-kill data and to estimate model parameters; the probability of the bacterial population size dropping below some critical threshold is also evaluated. Finally, the relationship between bacterial extinction probability and the pharmacodynamic parameters estimated is discussed.     

A machine learning-based framework for delivery error prediction in proton pencil beam scanning using irradiation log-files

PurposeThis study aims to investigate the use of machine learning models for delivery error prediction in proton pencil beam scanning (PBS) delivery.MethodsA dataset of planned and delivered PBS spot parameters was generated from a set of 20 prostate patient treatments. Planned spot parameters (spot position, MU and energy) were extracted from the treatment planning system (TPS) for each beam. Delivered spot parameters were extracted from irradiation log-files for each beam delivery following treatment. The dataset was used as a training dataset for three machine learning models which were trained to predict delivered spot parameters based on planned parameters. K-fold cross validation was employed for hyper-parameter tuning and model selection where the mean absolute error (MAE) was used as the model evaluation metric. The model with lowest MAE was then selected to generate a predicted dose distribution for a test prostate patient within a commercial TPS.ResultsAnalysis of the spot position delivery error between planned and delivered values resulted in standard deviations of 0.39 mm and 0.44 mm for x and y spot positions respectively. Prediction error standard deviation values of spot positions using the selected model were 0.22 mm and 0.11 mm for x and y spot positions respectively. Finally, a three-way comparison of dose distributions and DVH values for select OARs indicates that the random-forest-predicted dose distribution within the test prostate patient was in closer agreement to the delivered dose distribution than the planned distribution.ConclusionsPBS delivery error can be accurately predicted using machine learning techniques.     

System identification and parameter estimation in mathematical medicine: examples demonstrated for prostate cancer

We review our studies on how to identify the most appropriate models of diseases, and how to determine their parameters in a quantitative manner given a short time series of biomarkers, using intermittent androgen deprivation therapy of prostate cancer as an example. Recently, it has become possible to estimate the specific parameters of individual patients within a reasonable time by employing the information concerning other previous patients as a prior. We discuss the importance of using multiple mathematical methods simultaneously to achieve a solid diagnosis and prognosis in the future practice of personalized medicine.     

A mechanistic model for enzymatic saccharification of cellulose using continuous distribution kinetics II: cooperative enzyme action, solution kinetics, and product inhibition

The projected cost for the enzymatic hydrolysis of cellulosic biomass continues to be a barrier for the commercial production of liquid transportation fuels from renewable feedstocks. Predictive models for the kinetics of the enzymatic reactions will enable an improved understanding of current limitations, such as the slow-down of the overall conversion rate, and may point the way for more efficient utilization of the enzymes in order to achieve higher conversion yields. A mechanistically based kinetic model for the enzymatic hydrolysis of cellulose was recently reported in Griggs et al. (2011) (Part I). In this article (Part II), the enzyme system is expanded to include solution-phase kinetics, particularly cellobiose-to-glucose conversion by β-glucosidase (βG), and novel adsorption and product inhibition schemes have been incorporated, based on current structural knowledge of the component enzymes. Model results show cases of cooperative and non-cooperative hydrolysis for an enzyme system consisting of EG(I) and CBH(I). The model is used to explore various potential rate-limiting phenomena, such as substrate accessibility, product inhibition, sterically hindered enzyme adsorption, and the molecular weight of the cellulose substrate.     

A novel organic-inorganic hybrid monolith for trypsin immobilization

In proteomics, attention has focused on various immobilized enzyme reactors (IMERs) for the realization of high throughput digestion. In this report, a novel organic-inorganic hybrid monolith based IMER was prepared in a 100 μm i.d. capillary with 3-glycidoxypropyltrimethoxysilane (GLYMO) as the monomer and tetraethoxysilane (TEOS) as the crosslinker. Trypsin immobilization was achieved via the reaction between vicinal diol groups, which were obtained from hydrolysis of epoxy groups, and the amino groups of trypsin. Bovine serum albumin was digested thoroughly by this IMER in 47 s. After micro-reverse phase liquid chromatography-tandem mass spectrometry (μRPLC-MS/MS) analysis and database searching, beyond 35% sequence coverage was obtained, and the result was comparable to that of 12 h in solution digestion. The present IMER has potential for high throughput digestion.