Automated neuron model optimization techniques: a review

The increase in complexity of computational neuron models makes the hand tuning of model parameters more difficult than ever. Fortunately, the parallel increase in computer power allows scientists to automate this tuning. Optimization algorithms need two essential components. The first one is a function that measures the difference between the output of the model with a given set of parameter and the data. This error function or fitness function makes the ranking of different parameter sets possible. The second component is a search algorithm that explores the parameter space to find the best parameter set in a minimal amount of time. In this review we distinguish three types of error functions: feature-based ones, point-by-point comparison of voltage traces and multi-objective functions. We then detail several popular search algorithms, including brute-force methods, simulated annealing, genetic algorithms, evolution strategies, differential evolution and particle-swarm optimization. Last, we shortly describe Neurofitter, a free software package that combines a phase–plane trajectory density fitness function with several search algorithms.     

Searching for structural bias in particle swarm optimization and differential evolution algorithms

During the last two decades, a large number of metaheuristics have been proposed, leading to various studies that call for a deeper insight into the behaviour, efficiency and effectiveness of such methods. Among numerous concerns that are briefly reviewed in this paper, the presence of a structural bias (i.e. the tendency, not justified by the fitness landscape, to visit some regions of the search space more frequently than other regions) has recently been detected in simple versions of the genetic algorithm and particle swarm optimization. As of today, it remains unclear how frequently such a behaviour occurs in population-based swarm intelligence and evolutionary computation methods, and to what extent structural bias affects their performance. The present study focuses on the search for structural bias in various variants of particle swarm optimization and differential evolution algorithms, as well as in the traditional direct search methods proposed by Nelder–Mead and Rosenbrock half a century ago. We found that these historical direct search methods are structurally unbiased. However, most tested new metaheuristics are structurally biased, and at least some presence of structural bias can be observed in almost all their variants. The presence of structural bias seems to be stronger in particle swarm optimization algorithms than in differential evolution algorithms. The relationships between the strength of the structural bias and the dimensionality of the search space, the number of allowed function calls and the population size are complex and hard to generalize. For 14 algorithms tested on the CEC2011 real-world problems and the CEC2014 artificial benchmarks, no clear relationship between the strength of the structural bias and the performance of the algorithm was found. However, at least for artificial benchmarks, such old and structurally unbiased methods like Nelder–Mead algorithm performed relatively well. This is a warning that the presence of structural bias in novel metaheuristics may hamper their search abilities.     

Application of simplex‐based experimental optimization to challenging bioprocess development problems: Case studies in downstream processing

The identification of feasible operating conditions during the early stages of bioprocess development is implemented frequently through High Throughput (HT) studies. These typically employ techniques based on regression analysis, such as Design of Experiments. In this work, an alternative approach, based on a previously developed variant of the Simplex algorithm, is compared to the conventional regression‐based method for three experimental systems involving polishing chromatography and protein refolding. This Simplex algorithm variant was found to be more effective in identifying superior operating conditions, and in fact it reached the global optimum in most cases involving multiple optima. By contrast, the regression‐based method often failed to reach the global optimum, and in many cases reached poor operating conditions. The Simplex‐based method is further shown to be robust in dealing with noisy experimental data, and requires fewer experiments than regression‐based methods to reach favorable operating conditions. The Simplex‐variant also lends itself to the use of HT analytical methods, when they are available, which can assist in avoiding analytical bottlenecks. It is suggested that this Simplex‐variant is ideally suited to rapid optimization in early‐phase process development. © 2016 American Institute of Chemical Engineers Biotechnol. Prog., 32:404–419, 2016     

Robust optimization of total joint replacements incorporating environmental variables.

Direct search techniques for the optimal design of biomechanical devices are computationally intensive requiring many iterations before converging to a global solution. This, along with the incorporation of environmental variables such as multiple loading conditions and bone properties, makes direct search techniques infeasible. In this study, we introduced new methods that are based on the statistical design and analysis of computer experiments to account efficiently for environmental variables. Using data collected at a relatively small set of training sites, the method employs a computationally inexpensive predictor of the structural response that is statistically motivated. By using this predictor in place of the simulator (e.g., finite element model), a sufficient number of iterations can be performed to facilitate the optimization of the complex system. The applicability of these methods was demonstrated through the design of a femoral component for total hip arthroplasty incorporating variations in joint force orientation and cancellous bone properties. Beams on elastic foundation (BOEF) finite element models were developed to simulate the structural response. These simple models were chosen for their short computation time. This allowed us to represent the actual structural response surface by an exhaustive enumeration of the design and environmental variable space, and provided a means by which to validate the statistical predictor. We were able to predict the structural response and the optimal design accurately using only 16 runs of the computer code. The general trends predicted by the BOEF models were in agreement with previous three-dimensional finite element computer simulations, and experimental and clinical results, which demonstrated that the important features of intramedullary fixation systems were captured. These results indicate that the statistically based optimization methods are appropriate for optimization studies using computationally demanding models.     

基于遗传算法的土地利用优化：NSGA-Ⅱ和NSGA-Ⅲ的对比研究

土地利用优化通常要兼顾不同群体的多种要求,理论上是复杂的超多目标(4个及以上)优化问题。但实际操作中却往往被简化为多目标(2—3个)优化问题,通过一种流行的多目标优化算法第Ⅱ代非支配排序遗传算法(NSGA-Ⅱ)求解。究其原因是对超多目标优化算法认知的缺失和与多目标优化算法理论对比的匮乏。对NSGA系列中应用最广泛的多目标优化算法NSGA-Ⅱ和最新提出、面向超多目标优化的算法NSGA-Ⅲ进行探究,从理论和实验两方面对Ⅲ和Ⅱ进行对比,从而探究二者进行土地利用优化时的优劣。在理论上,对比两种算法原理的异同。在实验中,分别设计多目标(3个目标)和超多目标(13个目标)土地利用优化问题,利用两种算法进行求解。对实验结果采用四层架构、六大指标进行全面评价,以对比两种算法的可用性。理论对比发现,两个算法只有种群多样性保护的方法不同,其中NSGA-Ⅲ是基于与固定的参考点的距离,而NSGA-Ⅱ则是基于相邻解间的距离。通过实验对比发现,NSGA-Ⅲ在超多目标优化时运算速度快,且产生的最优方案实用价值更高,NSGA-Ⅱ在算法的有效性方面更有优势。     

A novel numerical optimization algorithm inspired from weed colonization

This paper introduces a novel numerical stochastic optimization algorithm inspired from colonizing weeds. Weeds are plants whose vigorous, invasive habits of growth pose a serious threat to desirable, cultivated plants making them a threat for agriculture. Weeds have shown to be very robust and adaptive to change in environment. Thus, capturing their properties would lead to a powerful optimization algorithm. It is tried to mimic robustness, adaptation and randomness of colonizing weeds in a simple but effective optimizing algorithm designated as Invasive Weed Optimization (IWO). The feasibility, the efficiency and the effectiveness of IWO are tested in details through a set of benchmark multi-dimensional functions, of which global and local minima are known. The reported results are compared with other recent evolutionary-based algorithms: genetic algorithms, memetic algorithms, particle swarm optimization, and shuffled frog leaping. The results are also compared with different versions of simulated annealing — a generic probabilistic meta-algorithm for the global optimization problem — which are simplex simulated annealing, and direct search simulated annealing. Additionally, IWO is employed for finding a solution for an engineering problem, which is optimization and tuning of a robust controller. The experimental results suggest that results from IWO are better than results from other methods. In conclusion, the performance of IWO has a reasonable performance for all the test functions.     

A genetic algorithm approach to optimization of asynchronous automatic assembly systems

This paper presents the application of genetic algorithms to the performance optimization of asynchronous automatic assembly systems (AAS). These stochastic systems are subject to blocking and starvation effects that make complete analytic performance modeling difficult. Therefore, this paper extends genetic algorithms to stochastic systems. The performance of the genetic algorithm is measured through comparison with the results of stochastic quasi-gradient (SQM) methods to the same AAS. The genetic algorithm performs reasonably well in obtaining good solutions (as compared with results of SQM) in this stochastic optimization example, even though genetic algorithms were designed for application to deterministic systems. However, the genetic algorithm's performance does not appear to be superior to SQM.     

Multigradient method for optimization of slow biotechnological processes

A new method (named a "jumping spider") is introduced for the optimization of slow biotechnological processes. The more traditional sequential experimentation (i.e., gradient search, simplex, etc.) is not well suited for slow dynamic processes, e.g., plant cell culture and differentiation. Therefore, a more simultaneous approach is proposed. A large number of initial experiments are performed, on the basis of which several of the initial experiments are selected as starting points. A search is then performed simultaneously from several gradient directions and the optimum is estimated by a quadratic approximation. In simulations, the spider generally climbs up the slopes quickly and the final estimator yields good maximum point estimates even on a complex topography. The spider may even approach more than one local maximum point simultaneously. As a model application, the average xylitol conversion rate of Candida guilliermondii was optimized in relation to cultivation volume (oxygen availability) and the concentration of nitrogen and phosphorus in the medium. A threefold increase in xylitol production was obtained with three experimental steps. (c) 1993 John Wiley & Sons, Inc.     

Simplex‐based optimization of numerical and categorical inputs in early bioprocess development: Case studies in HT chromatography

Bioprocess development studies often involve the investigation of numerical and categorical inputs via the adoption of Design of Experiments (DoE) techniques. An attractive alternative is the deployment of a grid compatible Simplex variant which has been shown to yield optima rapidly and consistently. In this work, the method is combined with dummy variables and it is deployed in three case studies wherein spaces are comprised of both categorical and numerical inputs, a situation intractable by traditional Simplex methods. The first study employs in silico data and lays out the dummy variable methodology. The latter two employ experimental data from chromatography based studies performed with the filter‐plate and miniature column High Throughput (HT) techniques. The solute of interest in the former case study was a monoclonal antibody whereas the latter dealt with the separation of a binary system of model proteins. The implemented approach prevented the stranding of the Simplex method at local optima, due to the arbitrary handling of the categorical inputs, and allowed for the concurrent optimization of numerical and categorical, multilevel and/or dichotomous, inputs. The deployment of the Simplex method, combined with dummy variables, was therefore entirely successful in identifying and characterizing global optima in all three case studies. The Simplex‐based method was further shown to be of equivalent efficiency to a DoE‐based approach, represented here by D‐Optimal designs. Such an approach failed, however, to both capture trends and identify optima, and led to poor operating conditions. It is suggested that the Simplex‐variant is suited to development activities involving numerical and categorical inputs in early bioprocess development.     

Adaptive optimization of fed-batch culture of yeast by using genetic algorithms

A simulation and experimental study has been carried out on the adaptive optimization of fed-batch culture of yeast. In the simulation study, three genetic algorithms based on different optimization strategies were developed. The performance of those three algorithms were compared with one another and with that of a variational calculus approach. The one that showed the best performance was selected to be used in the subsequent experimental study. To confer an adaptability, an online adaptation (or model update) algorithm was developed and incorporated into the selected optimization algorithm. The resulting adaptive algorithm was experimentally applied to fed-batch cultures of a recombinant yeast producing salmon calcitonin, to maximize the cell mass production. It followed the actual process quite well and gave a much higher value of performance index than the simple genetic algorithm with no adaptability.     

Optimization of fermentation medium by a modified method of genetic algorithms

Summary During the study of mevinolin biosynthesis by Aspergillus terreus ATCC 20542, 10 different medium components were selected for medium optimization. A new optimization method based on genetic algorithms and inductive learning was used for experimental design. For better efficiency the method was supported by a model, constructed with machine learning method, to predict the productivity. In four generations of fermentation experiments the productivity increased nearly three times.     

Determining residual reduction algorithm kinematic tracking weights for a sidestep cut via numerical optimization

Musculoskeletal modeling allows for the determination of various parameters during dynamic maneuvers by using in vivo kinematic and ground reaction force (GRF) data as inputs. Differences between experimental and model marker data and inconsistencies in the GRFs applied to these musculoskeletal models may not produce accurate simulations. Therefore, residual forces and moments are applied to these models in order to reduce these differences. Numerical optimization techniques can be used to determine optimal tracking weights of each degree of freedom of a musculoskeletal model in order to reduce differences between the experimental and model marker data as well as residual forces and moments. In this study, the particle swarm optimization (PSO) and simplex simulated annealing (SIMPSA) algorithms were used to determine optimal tracking weights for the simulation of a sidestep cut. The PSO and SIMPSA algorithms were able to produce model kinematics that were within 1.4° of experimental kinematics with residual forces and moments of less than 10 N and 18 Nm, respectively. The PSO algorithm was able to replicate the experimental kinematic data more closely and produce more dynamically consistent kinematic data for a sidestep cut compared to the SIMPSA algorithm. Future studies should use external optimization routines to determine dynamically consistent kinematic data and report the differences between experimental and model data for these musculoskeletal simulations.     

Using a genetic algorithm to drive a microbial ecosystem in a desirable direction

The functioning of natural microbial ecosystems is influenced by various biotic and abiotic conditions. The careful experimental manipulation of environmental conditions can drive microbial ecosystems toward exhibiting desirable types of functionality. Such manipulations can be systematically approached by viewing them as a combinatorial optimization problem, in which the optimal configuration of environmental conditions is sought. Such an effort requires a sound optimization technique. Genetic algorithms are a class of optimization methods that should be suitable for such a task because they can deal with multiple interacting variables and with experimental noise and because they do not require an intricate understanding or modelling of the ecosystem of interest. We propose the use of genetic algorithms to drive undefined microbial ecosystems in desirable directions by combinatorially optimizing sets of environmental conditions. We tested this approach in a model system where the microbial ecosystem of a human saliva sample was manipulated in successive steps to display increasing amounts of azo dye decoloration. The results of our experiments indicated that a genetic algorithm was capable of optimizing ecosystem function by manipulating the presence or absence of a set of 10 chemical supplements. Genetic algorithms hold promise for use as a tool in environmental microbiology for the efficient control of the functioning of natural and undefined microbial ecosystems.     

Algorithms for optimization of the transport system in living and artificial cells

An optimization of the transport system in a cell has been considered from the viewpoint of the operations research. Algorithms for an optimization of the transport system of a cell in terms of both the efficiency and a weak sensitivity of a cell to environmental changes have been proposed. The switching of various systems of transport is considered as the mechanism of weak sensitivity of a cell to changes in environment. The use of the algorithms for an optimization of a cardiac cell has been considered by way of example. We received theoretically for a cell of a cardiac muscle that at the increase of potassium concentration in the environment switching of transport systems for this ion takes place. This conclusion qualitatively coincides with experiments. The problem of synthesizing an optimal system in an artificial cell has been stated.     

单形格子和单形重心设计统计模型的优化分析方法

单形格子和单形重心设计是两种非常实用的配方试验设计方法,但其统计模型的优化分析却很困难．本文通过对单形格子和单形重心设计基本原理的分析,根据数学规划理论,构建了专门对这两种试验设计的统计模型进行优化分析的方法,同时给出了应用实例．     

Constrained global optimization for estimating molecular structure from atomic distances.

Finding optimal three-dimensional molecular configurations based on a limited amount of experimental and/or theoretical data requires efficient nonlinear optimization algorithms. Optimization methods must be able to find atomic configurations that are close to the absolute, or global, minimum error and also satisfy known physical constraints such as minimum separation distances between atoms (based on van der Waals interactions). The most difficult obstacles in these types of problems are that 1) using a limited amount of input data leads to many possible local optima and 2) introducing physical constraints, such as minimum separation distances, helps to limit the search space but often makes convergence to a global minimum more difficult. We introduce a constrained global optimization algorithm that is robust and efficient in yielding near-optimal three-dimensional configurations that are guaranteed to satisfy known separation constraints. The algorithm uses an atom-based approach that reduces the dimensionality and allows for tractable enforcement of constraints while maintaining good global convergence properties. We evaluate the new optimization algorithm using synthetic data from the yeast phenylalanine tRNA and several proteins, all with known crystal structure taken from the Protein Data Bank. We compare the results to commonly applied optimization methods, such as distance geometry, simulated annealing, continuation, and smoothing. We show that compared to other optimization approaches, our algorithm is able combine sparse input data with physical constraints in an efficient manner to yield structures with lower root mean squared deviation.     

On the probabilistic optimization of spiking neural networks

The construction of a Spiking Neural Network (SNN), i.e. the choice of an appropriate topology and the configuration of its internal parameters, represents a great challenge for SNN based applications. Evolutionary Algorithms (EAs) offer an elegant solution for these challenges and methods capable of exploring both types of search spaces simultaneously appear to be the most promising ones. A variety of such heterogeneous optimization algorithms have emerged recently, in particular in the field of probabilistic optimization. In this paper, a literature review on heterogeneous optimization algorithms is presented and an example of probabilistic optimization of SNN is discussed in detail. The paper provides an experimental analysis of a novel Heterogeneous Multi-Model Estimation of Distribution Algorithm (hMM-EDA). First, practical guidelines for configuring the method are derived and then the performance of hMM-EDA is compared to state-of-the-art optimization algorithms. Results show hMM-EDA as a light-weight, fast and reliable optimization method that requires the configuration of only very few parameters. Its performance on a synthetic heterogeneous benchmark problem is highly competitive and suggests its suitability for the optimization of SNN.     

GUIMACS - a Java based front end for GROMACS

Molecular dynamics simulations have gained importance due to their ability to provide valuable insights into understanding structure-function relationships of biological macromolecules. With increasing computational speeds there has been a substantial demand for optimization of simulation algorithms to obtain results even faster. With this on one hand, the need for ease of operation lies on the other. GUI front end programs are important appurtenances to ease the use of command line programs. Effective use of command line based programs requires basic knowledge of the UNIX shell and at least one of the UNIX based text editors, making it difficult for pure biologists to use them efficiently. GROMACS, a widely used suite of molecular dynamics simulation and analysis programs, is no exception to this. As a matter of fact, the increasing dependency of experimental procedures on computational methods for accentuating certain key experimental findings increases the need for interactivity in use of command-line based packages.     

A critical analysis of parameter adaptation in ant colony optimization

Applying parameter adaptation means operating on parameters of an algorithm while it is tackling an instance. For ant colony optimization, several parameter adaptation methods have been proposed. In the literature, these methods have been shown to improve the quality of the results achieved in some particular contexts. In particular, they proved to be successful when applied to novel ant colony optimization algorithms for tackling problems that are not a classical testbed for optimization algorithms. In this paper, we show that the adaptation methods proposed so far do not improve, and often even worsen the performance when applied to high performing ant colony optimization algorithms for some classical combinatorial optimization problems.