ConvAn: a convergence analyzing tool for optimization of biochemical networks

Dynamic models of biochemical networks usually are described as a system of nonlinear differential equations. In case of optimization of models for purpose of parameter estimation or design of new properties mainly numerical methods are used. That causes problems of optimization predictability as most of numerical optimization methods have stochastic properties and the convergence of the objective function to the global optimum is hardly predictable. Determination of suitable optimization method and necessary duration of optimization becomes critical in case of evaluation of high number of combinations of adjustable parameters or in case of large dynamic models. This task is complex due to variety of optimization methods, software tools and nonlinearity features of models in different parameter spaces. A software tool ConvAn is developed to analyze statistical properties of convergence dynamics for optimization runs with particular optimization method, model, software tool, set of optimization method parameters and number of adjustable parameters of the model. The convergence curves can be normalized automatically to enable comparison of different methods and models in the same scale. By the help of the biochemistry adapted graphical user interface of ConvAn it is possible to compare different optimization methods in terms of ability to find the global optima or values close to that as well as the necessary computational time to reach them. It is possible to estimate the optimization performance for different number of adjustable parameters. The functionality of ConvAn enables statistical assessment of necessary optimization time depending on the necessary optimization accuracy. Optimization methods, which are not suitable for a particular optimization task, can be rejected if they have poor repeatability or convergence properties. The software ConvAn is freely available on www.biosystems.lv/convan.     

A global optimization method for prediction of muscle forces of human musculoskeletal system

Inverse dynamic optimization is a popular method for predicting muscle and joint reaction forces within human musculoskeletal joints. However, the traditional formulation of the optimization method does not include the joint reaction moment in the moment equilibrium equation, potentially violating the equilibrium conditions of the joint. Consequently, the predicted muscle and joint reaction forces are coordinate system-dependent. This paper presents an improved optimization method for the prediction of muscle forces and joint reaction forces. In this method, the location of the rotation center of the joint is used as an optimization variable, and the moment equilibrium equation is formulated with respect to the joint rotation center to represent an accurate moment constraint condition. The predicted muscle and joint reaction forces are independent of the joint coordinate system. The new optimization method was used to predict muscle forces of an elbow joint. The results demonstrated that the joint rotation center location varied with applied loading conditions. The predicted muscle and joint reaction forces were different from those predicted by using the traditional optimization method. The results further demonstrated that the improved optimization method converged to a minimum for the objective function that is smaller than that reached by using the traditional optimization method. Therefore, the joint rotation center location should be involved as a variable in an inverse dynamic optimization method for predicting muscle and joint reaction forces within human musculoskeletal joints.     

Search-based optimization

The problem of determining the minimum cost hypothetical ancestral sequences for a given cladogram is known to be NP-complete (Wang and Jiang, 1994). Traditionally, point estimations of hypothetical ancestral sequences have been used to gain heuristic, upper bounds on cladogram cost. These include procedures with such diverse approaches as non-additive optimization of multiple sequence alignment, direct optimization (Wheeler, 1996), and fixed-state character optimization (Wheeler, 1999). A method is proposed here which, by extending fixed-state character optimization, replaces the estimation process with a search. This form of optimization examines a diversity of potential state solutions for cost-efficient hypothetical ancestral sequences and can result in greatly more parsimonious cladograms. Additionally, such an approach can be applied to other NP-complete phylogenetic optimization problems such as genomic break-point analysis.     

Set-based particle swarm optimization applied to the multidimensional knapsack problem

Particle swarm optimization algorithms have been successfully applied to discrete/valued optimization problems. However, in many cases the algorithms have been tailored specifically for the problem at hand. This paper proposes a generic set-based particle swarm optimization algorithm for use in discrete-valued optimization problems that can be formulated as set-based problems. A detailed sensitivity analysis of the parameters of the algorithm is conducted. The performance of the proposed algorithm is then compared against three other discrete particle swarm optimization algorithms from literature using the multidimensional knapsack problem and is shown to statistically outperform the existing algorithms.     

Integrated process optimization: lessons from retrovirus and virus-like particle production

The optimization of production and purification processes is usually approached by engineers from a strictly biotechnological point of view. The present paper envisages the definition and application of an optimization model that takes into account the impact of both biological and technological issues upon the optimization protocols and strategies. For this purpose, the optimization of three analogous but different systems comprising animal cell growth and bioparticle production is presented. These systems were: human immunodeficiency 1 (HIV-1) and porcine parvovirus (PPV) virus-like particles (VLPs) produced in insect cells and retrovirus produced in mammalian cells. For the systematization of the optimization process four levels of optimization were defined-product, technology, design and integration. In this paper, the limits of each of the optimization levels defined are discussed by applying the concept to the systems described. This analysis leads to decisions regarding the production of VLPs and retrovirus as well as on the points relevant for further process development. Finally, the definition of the objective function or performance index, the possible strategies and tools for bioprocess optimization are described. Although developed from the three described processes, this approach can, based on the recent literature evidence reviewed here, be applied more universally for the process development of complex biopharmaceuticals.     

Dynamics of granzyme B-induced apoptosis: mathematical modeling

Apoptosis is mediated by an intracellular biochemical system that mainly includes proteins (procaspases, caspases, inhibitors, Bcl-2 protein family as well as substances released from mitochondrial intermembrane space). The dynamics of caspase activation and target cleavage in apoptosis induced by granzyme B in a single K562 cell was studied using a mathematical model of the dynamics of granzyme B-induced apoptosis developed in this work. Also the first application of optimization approach to determination of unknown kinetic constants of biochemical apoptotic reactions was presented. The optimization approach involves solving of two problems: direct and inverse. Solving the direct optimization problem, we obtain the initial (baseline) concentrations of procaspases for known kinetic constants through conditional minimization of a cost function based on the principle of minimum protein consumption by the apoptosis system. The inverse optimization problem is aimed at determination of unknown kinetic constants of apoptotic biochemical reactions proceeding from the condition that the optimal concentrations of procaspases resulting from the solution of the direct optimization problem coincide with the observed ones, that is, those determined by biochemical methods. The Multidimensional Index Method was used to perform numerical solution of the inverse optimization problem.     

改进支持向量机在棉铃虫人工饲料配方优化中的应用

发展新的实验设计与分析方法,通过实施尽可能少的实验而获得满意配方对动植物营养、发酵工程等复杂多因素多水平寻优问题极为重要。本研究结合均匀设计（uniform design, UD）与支持向量回归（support vector regression, SVR）,提出了一种新的配方优化实验设计与分析方法UD-SVR,将其应用于棉铃虫Helicoverpa armigera （Hübner）幼虫人工饲料配方优化。结果表明：在考虑6因素时仅通过2轮22个实验,表征配方优劣的指标--平均蛹重即由初始的0.2436 g高效提升至0.3044 g,明显优于二次多项式偏最小二乘回归等经验风险最小参比模型。UD-SVR预测精度高、指导性强、可解释性好、优化高效,有望在多因素多水平配方优化中得到广泛应用。     

Integration of multi-objective structural optimization into cementless hip prosthesis design: Improved Austin-Moore model

A new strategy of multi-objective structural optimization is integrated into Austin-Moore prosthesis in order to improve its performance. The new resulting model is so-called Improved Austin-Moore. The topology optimization is considered as a conceptual design stage to sketch several kinds of hollow stems according to the daily loading cases. The shape optimization presents the detailed design stage considering several objectives. Here, A new multiplicative formulation is proposed as a performance scale in order to define the best compromise between several requirements. Numerical applications on 2D and 3D problems are carried out to show the advantages of the proposed model.     

Simulated Evolutionary Optimization and Local Search: Introduction and Application to Tree Search

Tree search and its more complicated variant, tree search and simultaneous multiple DNA sequence alignment, are difficult NP-complete optimization problems, which require the application of advanced computational techniques, if large data sets are to be solved within reasonable computation times. Traditionally tree search has been attacked with a search strategy that is best described as multistart hill-climbing; local search by branch swapping has been performed on several different starting trees. Recently a different tree search strategy was tested in the Parsigal parsimony program, which used a combination of evolutionary optimization and local search. Evolutionary optimization algorithms use principles adopted from biological evolution to solve technical optimization tasks. Evolutionary optimization is a stochastic global search method, which means that the method is able to escape local optima, and is in principle able to produce any solution in the search space (although this may take a long time). Local search techniques, such as branch swapping, employ a completely different search strategy; they exploit local information maximally in order to achieve quick improvement in the value of the objective function. However, local search algorithms lack the ability to escape from local optima, which is a fundamental requirement for any search algorithm that aims to be able to discover the global optimum of a multimodal optimization problem. Hence it seems that an optimization strategy combining the good properties of both evolutionary algorithms and local search would be ideal. In this study, aspects of global optimization and local search are discussed, and the method of simulated evolutionary optimization is reviewed in detail. The application of simulated evolutionary optimization to tree search in Parsigal is then reviewed briefly.     

On the probabilistic optimization of spiking neural networks

The construction of a Spiking Neural Network (SNN), i.e. the choice of an appropriate topology and the configuration of its internal parameters, represents a great challenge for SNN based applications. Evolutionary Algorithms (EAs) offer an elegant solution for these challenges and methods capable of exploring both types of search spaces simultaneously appear to be the most promising ones. A variety of such heterogeneous optimization algorithms have emerged recently, in particular in the field of probabilistic optimization. In this paper, a literature review on heterogeneous optimization algorithms is presented and an example of probabilistic optimization of SNN is discussed in detail. The paper provides an experimental analysis of a novel Heterogeneous Multi-Model Estimation of Distribution Algorithm (hMM-EDA). First, practical guidelines for configuring the method are derived and then the performance of hMM-EDA is compared to state-of-the-art optimization algorithms. Results show hMM-EDA as a light-weight, fast and reliable optimization method that requires the configuration of only very few parameters. Its performance on a synthetic heterogeneous benchmark problem is highly competitive and suggests its suitability for the optimization of SNN.     

The role of carbon flux and biometric observations in constraining a terrestrial ecosystem model: a case study in disturbed forests in East Asia

The process of confining unnecessary freedom is a step toward advanced ecosystem modeling. This study demonstrates the importance of carbon flux and biometric observation in constraining a terrestrial ecosystem model with a simple optimization scheme. At the selected sites from AsiaFlux network, a simultaneous optimization scheme for both carbon flux and biomass was compared with carbon flux-oriented and biomass-oriented optimization schemes using the Biome-BGC model. The optimization scheme oriented to either carbon flux or biomass provided simulation results that were consistent with observations, but with reduced performance in unconstrained variables. The simultaneous optimization scheme yielded results that were consistent with observations for both carbon flux and biomass. By comparing long-term projections simulated by three schemes, it was found that the optimization oriented only to carbon flux has limited performance because misrepresented biomass significantly affected a projection of carbon exchange through heterotrophic respiration. From these experiments, we found that (1) a process model like Biome-BGC is capable of reproducing both carbon flux and biomass within acceptable proximity, (2) constraining biomass is importance not just because it is one of carbon cycle components, but also it significantly affects simulations of carbon flux. Thus, it is important to invest more effort to improve simulation of biomass as well as carbon flux.     

Application of multiple response optimization design to quantum dot-encoded microsphere bioconjugates hybridization assay

The optimization of DNA hybridization for genotyping assays is a complex experimental problem that depends on multiple factors such as assay formats, fluorescent probes, target sequence, experimental conditions, and data analysis. Quantum dot-doped particle bioconjugates have been previously described as fluorescent probes to identify single nucleotide polymorphisms even though this advanced fluorescent material has shown structural instability in aqueous environments. To achieve the optimization of DNA hybridization to quantum dot-doped particle bioconjugates in suspension while maximizing the stability of the probe materials, a nonsequential optimization approach was evaluated. The design of experiment with response surface methodology and multiple optimization response was used to maximize the recovery of fluorescent probe at the end of the assay simultaneously with the optimization of target–probe binding. Hybridization efficiency was evaluated by the attachment of fluorescent oligonucleotides to the fluorescent probe through continuous flow cytometry detection. Optimal conditions were predicted with the model and tested for the identification of single nucleotide polymorphisms. The design of experiment has been shown to significantly improve biochemistry and biotechnology optimization processes. Here we demonstrate the potential of this statistical approach to facilitate the optimization of experimental protocol that involves material science and molecular biology.     

Computational study of Wolff's law with trabecular architecture in the human proximal femur using topology optimization

In the field of bone adaptation, it is believed that the morphology of bone is affected by its mechanical loads, and bone has self-optimizing capability; this phenomenon is well known as Wolff's law of the transformation of bone. In this paper, we simulated trabecular bone adaptation in the human proximal femur using topology optimization and quantitatively investigated the validity of Wolff's law. Topology optimization iteratively distributes material in a design domain producing optimal layout or configuration, and it has been widely and successfully used in many engineering fields. We used a two-dimensional micro-FE model with 50 microm pixel resolution to represent the full trabecular architecture in the proximal femur, and performed topology optimization to study the trabecular morphological changes under three loading cases in daily activities. The simulation results were compared to the actual trabecular architecture in previous experimental studies. We discovered that there are strong similarities in trabecular patterns between the computational results and observed data in the literature. The results showed that the strain energy distribution of the trabecular architecture became more uniform during the optimization; from the viewpoint of structural topology optimization, this bone morphology may be considered as an optimal structure. We also showed that the non-orthogonal intersections were constructed to support daily activity loadings in the sense of optimization, as opposed to Wolff's drawing.     

Feedback optimization of fed-batch fermentation

The problem of feedback optimization of the feed rate for fed-batch fermentation processes is formulated in the framework of singular control theory and switching hypersurfaces. Using four differential balance equations that describe a general class of fedbatch processes and a general objective function to be minimized, it is shown that under certain restrictions the feedback optimization of the feed rate can be realized as a nonlinear function of the state variables, such as the concentrations of cell mass, substrate and product, and the fermentor volume. The restrictions on the initial conditions, the fermentation kinetics and the objective function, that are needed for realization of the feedback optimization, are provided. Fed-batch fermentation models of lysine and alcohol are used to construct switching curves and to illustrate the feedback optimization of the feed flow rates.     

Choosing non‐redundant representative subsets of protein sequence data sets using submodular optimization

Selecting a non‐redundant representative subset of sequences is a common step in many bioinformatics workflows, such as the creation of non‐redundant training sets for sequence and structural models or selection of “operational taxonomic units” from metagenomics data. Previous methods for this task, such as CD‐HIT, PISCES, and UCLUST, apply a heuristic threshold‐based algorithm that has no theoretical guarantees. We propose a new approach based on submodular optimization. Submodular optimization, a discrete analogue to continuous convex optimization, has been used with great success for other representative set selection problems. We demonstrate that the submodular optimization approach results in representative protein sequence subsets with greater structural diversity than sets chosen by existing methods, using as a gold standard the SCOPe library of protein domain structures. In this setting, submodular optimization consistently yields protein sequence subsets that include more SCOPe domain families than sets of the same size selected by competing approaches. We also show how the optimization framework allows us to design a mixture objective function that performs well for both large and small representative sets. The framework we describe is the best possible in polynomial time (under some assumptions), and it is flexible and intuitive because it applies a suite of generic methods to optimize one of a variety of objective functions.     

A combined steepest descent and genetic algorithm (SD/GA) approach for the optimization of solvation parameters

An accurate solvation model is essential for computer modeling of protein folding and other biomolecular self-assembly processes. Compared to explicit solvent models, implicit solvent models, such as the Poisson-Boltzmann (PB) with solvent accessible surface area model (PB/SA), offer a much faster speed—the most compelling reason for the popularity of these implicit solvent models. Since these implicit solvent models typically use empirical parameters, such as atomic radii and the surface tensions, an optimal fit of these parameters is crucial for the final accuracy of properties such as solvation free energy and folding free energy. In this paper, we proposed a combined approach, namely SD/GA, which takes the advantage of both local optimization with the steepest descent (SD), and global optimization with the genetic algorithm (GA), for parameters optimization in multi-dimensional space. The SD/GA method is then applied to the optimization of solvation parameters in the non-polar cavity term of the PB/SA model. The results show that the newly optimized parameters from SD/GA not only increase the accuracy in the solvation free energies for ～200 organic molecules, but also significantly improve the free energy landscape of a β-hairpin folding. The current SD/GA method can be readily applied to other multi-dimensional parameter space optimization as well.     

A Bayesian Interpretation of the Particle Swarm Optimization and Its Kernel Extension

Particle swarm optimization is a popular method for solving difficult optimization problems. There have been attempts to formulate the method in formal probabilistic or stochastic terms (e.g. bare bones particle swarm) with the aim to achieve more generality and explain the practical behavior of the method. Here we present a Bayesian interpretation of the particle swarm optimization. This interpretation provides a formal framework for incorporation of prior knowledge about the problem that is being solved. Furthermore, it also allows to extend the particle optimization method through the use of kernel functions that represent the intermediary transformation of the data into a different space where the optimization problem is expected to be easier to be resolved–such transformation can be seen as a form of prior knowledge about the nature of the optimization problem. We derive from the general Bayesian formulation the commonly used particle swarm methods as particular cases.     

人工智能时代发酵优化与放大技术的机遇与挑战

人工智能(artificial intelligence, AI)技术正引发一场新的产业革命,其成功应用正从信息产业迅速渗透到各行各业。传统的发酵工程技术受到巨大挑战的同时更多地迎来了发展变革的机遇。首先,合成生物技术飞速发展使高性能菌株的可获得性及获取效率显著提升,对传统低效的发酵优化放大技术提出很大挑战,亟需对发酵优化放大技术进行升级,以满足高通量菌种性能验证及工艺开发能力的需求;其次,发酵装备技术的持续发展为高效发酵优化技术的进步奠定了良好基础,加之人工智能技术特别是数字孪生与知识图谱等技术的应用,将为传统发酵技术的颠覆性发展带来巨大推动力。本文分别从合成生物时代对发酵优化技术的挑战、发酵优化与放大的核心技术、高通量发酵装备技术、数据可视化技术、数字孪生及知识图谱等智能技术在发酵优化放大中的应用等几个方面进行综述,并对未来工业发酵优化技术的场景以及未来发酵技术对人才培养等提出的新要求进行了展望。