Self-adaptive particle swarm optimization: a review and analysis of convergence

Particle swarm optimization (PSO) is a population-based, stochastic search algorithm inspired by the flocking behaviour of birds. The PSO algorithm has been shown to be rather sensitive to its control parameters, and thus, performance may be greatly improved by employing appropriately tuned parameters. However, parameter tuning is typically a time-intensive empirical process. Furthermore, a priori parameter tuning makes the implicit assumption that the optimal parameters of the PSO algorithm are not time-dependent. To address these issues, self-adaptive particle swarm optimization (SAPSO) algorithms adapt their control parameters throughout execution. While there is a wide variety of such SAPSO algorithms in the literature, their behaviours are not well understood. Specifically, it is unknown whether these SAPSO algorithms will even exhibit convergent behaviour. This paper addresses this lack of understanding by investigating the convergence behaviours of 18 SAPSO algorithms both analytically and empirically. This paper also empirically examines whether the adapted parameters reach a stable point and whether the final parameter values adhere to a well-known convergence criterion. The results depict a grim state for SAPSO algorithms; over half of the SAPSO algorithms exhibit divergent behaviour while many others prematurely converge.     

Quantitative measurement of blood glucose influenced by multiple factors via photoacoustic technique combined with optimized wavelet neural networks

In this work, the photoacoustic (PA) quantitative measurement of blood glucose concentration (BGC) influenced by multiple factors was firstly investigated. A set of PA detection system of blood glucose considering the comprehensive influence of five factors was established. The PA signals and peak-to-peak values (PPVs) of 625 rabbit whole blood were obtained under 625 influence combinations. Due to the accurate measurement of BGC limited by the overlap PA signals, wavelet neural network (WNN) was utilized to train the PPVs of blood glucose for 500 rabbit blood. The mean square error (MSE) of BGC for 125 testing blood was approximately 6.5782 mmol/L. To decrease the MSE, the parameters of WNN were optimized by particle swarm optimization (PSO), that is, PSO-WNN algorithm was employed. Under the optimal parameters, MSE of BGC was decreased to approximately 0.48005 mmol/L. To further improve the prediction accuracy of BGC, an improved nonlinear dynamic inertia weight (NDIW) strategy of PSO was proposed, and compared with other two kinds of dynamic inertia weight strategies. Under the optimal parameters, the MSE of BGC was decreased to approximately 0.2635 mmol/L. The comparison of nine algorithms demonstrate that the PA technique combined with PSO-WNN and the improved NDIW strategy is significant in the quantitative measurement of blood glucose influenced by multiple factors.     

A cooperative particle swarm optimizer with migration of heterogeneous probabilistic models

Particle Swarm Optimization (PSO) is a stochastic optimization approach that originated from simulations of bird flocking, and that has been successfully used in many applications as an optimization tool. Estimation of distribution algorithms (EDAs) are a class of evolutionary algorithms which perform a two-step process: building a probabilistic model from which good solutions may be generated and then using this model to generate new individuals. Two distinct research trends that emerged in the past few years are the hybridization of PSO and EDA algorithms and the parallelization of EDAs to exploit the idea of exchanging the probabilistic model information. In this work, we propose the use of a cooperative PSO/EDA algorithm based on the exchange of heterogeneous probabilistic models. The model is heterogeneous because the cooperating PSO/EDA algorithms use different methods to sample the search space. Three different exchange approaches are tested and compared in this work. In all these approaches, the amount of information exchanged is adapted based on the performance of the two cooperating swarms. The performance of the cooperative model is compared to the existing state-of-the-art PSO cooperative approaches using a suite of well-known benchmark optimization functions.     

The PSO family: deduction, stochastic analysis and comparison

The PSO algorithm can be physically interpreted as a stochastic damped mass-spring system: the so-called PSO continuous model. Furthermore, PSO corresponds to a particular discretization of the PSO continuous model. In this paper, we introduce a delayed version of the PSO continuous model, where the center of attraction might be delayed with respect to the particle trajectories. Based on this mechanical analogy, we derive a family of PSO versions. For each member of this family, we deduce the first and second order stability regions and the corresponding spectral radius. As expected, the PSO-family algorithms approach the PSO continuous model (damped-mass-spring system) as the size of the time step goes to zero. All the family members are linearly isomorphic when they are derived using the same delay parameter. If the delay parameter is different, the algorithms have corresponding stability regions of any order, but they differ in their respective force terms. All the PSO versions perform fairly well in a very broad area of the parameter space (inertia weight and local and global accelerations) that is close to the border of the second order stability regions and also to the median lines of the first order stability regions where no temporal correlation between trajectories exists. In addition, these regions are fairly similar for different benchmark functions. When the number of parameters of the cost function increases, these regions move towards higher inertia weight values (w=1) and lower total mean accelerations where the temporal covariance between trajectories is positive. Finally, the comparison between different PSO versions results in the conclusion that the centered version (CC-PSO) and PSO have the best convergence rates. Conversely, the centered-progressive version (CP-PSO) has the greatest exploratory capabilities. These features have to do with the way these algorithms update the velocities and positions of particles in the swarm. Knowledge of their respective dynamics can be used to propose a family of simple and stable algorithms, including hybrid versions.     

Optimal Formation Reconfiguration Control of Multiple UCAVs Using Improved Particle Swarm Optimization

Optimal formation reconfiguration control of multiple Uninhabited Combat Air Vehicles （UCAVs） is a complicated global optimum problem. Particle Swarm Optimization （PSO） is a population based stochastic optimization technique inspired by social behaviour of bird flocking or fish schooling. PSO can achieve better results in a faster, cheaper way compared with other bio-inspired computational methods, and there are few parameters to adjust in PSO. In this paper, we propose an improved PSO model for solving the optimal formation reconfiguration control problem for multiple UCAVs. Firstly, the Control Parameterization and Time Diseretization （CPTD） method is designed in detail. Then, the mutation strategy and a special mutation-escape operator are adopted in the improved PSO model to make particles explore the search space more efficiently. The proposed strategy can produce a large speed value dynamically according to the variation of the speed, which makes the algorithm explore the local and global minima thoroughly at the same time. Series experimental results demonstrate the feasibility and effectiveness of the proposed method in solving the optimal formation reconfiguration control problem for multiple UCAVs.     

Performance evaluation of TRIBES, an adaptive particle swarm optimization algorithm

This paper presents a study of the performance of TRIBES, an adaptive particle swarm optimization algorithm. Particle Swarm Optimization (PSO) is a biologically-inspired optimization method. Recently, researchers have used it effectively in solving various optimization problems. However, like most optimization heuristics, PSO suffers from the drawback of being greatly influenced by the selection of its parameter values. Thus, the common belief is that the performance of a PSO algorithm is directly related to the tuning of such parameters. Usually, such tuning is a lengthy, time consuming and delicate process. A new adaptive PSO algorithm called TRIBES avoids manual tuning by defining adaptation rules which aim at automatically changing the particles’ behaviors as well as the topology of the swarm. In TRIBES, the topology is changed according to the swarm behavior and the strategies of displacement are chosen according to the performances of the particles. A comparative study carried out on a large set of benchmark functions shows that the performance of TRIBES is quite competitive compared to most other similar PSO algorithms that need manual tuning of parameters. The performance evaluation of TRIBES follows the testing procedure introduced during the 2005 IEEE Conference on Evolutionary Computation. The main objective of the present paper is to perform a global study of the behavior of TRIBES under several conditions, in order to determine strengths and drawbacks of this adaptive algorithm.     

Differential-Evolution Control Parameter Optimization for Unmanned Aerial Vehicle Path Planning

The differential evolution algorithm has been widely applied on unmanned aerial vehicle (UAV) path planning. At present, four random tuning parameters exist for differential evolution algorithm, namely, population size, differential weight, crossover, and generation number. These tuning parameters are required, together with user setting on path and computational cost weightage. However, the optimum settings of these tuning parameters vary according to application. Instead of trial and error, this paper presents an optimization method of differential evolution algorithm for tuning the parameters of UAV path planning. The parameters that this research focuses on are population size, differential weight, crossover, and generation number. The developed algorithm enables the user to simply define the weightage desired between the path and computational cost to converge with the minimum generation required based on user requirement. In conclusion, the proposed optimization of tuning parameters in differential evolution algorithm for UAV path planning expedites and improves the final output path and computational cost.     

Parameter optimization by using differential elimination: a general approach for introducing constraints into objective functions

Background

The investigation of network dynamics is a major issue in systems and synthetic biology. One of the essential steps in a dynamics investigation is the parameter estimation in the model that expresses biological phenomena. Indeed, various techniques for parameter optimization have been devised and implemented in both free and commercial software. While the computational time for parameter estimation has been greatly reduced, due to improvements in calculation algorithms and the advent of high performance computers, the accuracy of parameter estimation has not been addressed.

Results

We propose a new approach for parameter optimization by using differential elimination, to estimate kinetic parameter values with a high degree of accuracy. First, we utilize differential elimination, which is an algebraic approach for rewriting a system of differential equations into another equivalent system, to derive the constraints between kinetic parameters from differential equations. Second, we estimate the kinetic parameters introducing these constraints into an objective function, in addition to the error function of the square difference between the measured and estimated data, in the standard parameter optimization method. To evaluate the ability of our method, we performed a simulation study by using the objective function with and without the newly developed constraints: the parameters in two models of linear and non-linear equations, under the assumption that only one molecule in each model can be measured, were estimated by using a genetic algorithm (GA) and particle swarm optimization (PSO). As a result, the introduction of new constraints was dramatically effective: the GA and PSO with new constraints could successfully estimate the kinetic parameters in the simulated models, with a high degree of accuracy, while the conventional GA and PSO methods without them frequently failed.

Conclusions

The introduction of new constraints in an objective function by using differential elimination resulted in the drastic improvement of the estimation accuracy in parameter optimization methods. The performance of our approach was illustrated by simulations of the parameter optimization for two models of linear and non-linear equations, which included unmeasured molecules, by two types of optimization techniques. As a result, our method is a promising development in parameter optimization.

     

Foundations of modeling in cryobiology—I: Concentration,Gibbs energy,and chemical potential relationships

Mathematical modeling plays an enormously important role in understanding the behavior of cells, tissues, and organs undergoing cryopreservation. Uses of these models range from explanation of phenomena, exploration of potential theories of damage or success, development of equipment, and refinement of optimal cryopreservation/cryoablation strategies. Over the last half century there has been a considerable amount of work in bio-heat and mass-transport, and these models and theories have been readily and repeatedly applied to cryobiology with much success. However, there are significant gaps between experimental and theoretical results that suggest missing links in models. One source for these potential gaps is that cryobiology is at the intersection of several very challenging aspects of transport theory: it couples multi-component, moving boundary, multiphase solutions that interact through a semipermeable elastic membrane with multicomponent solutions in a second time-varying domain, during a two-hundred Kelvin temperature change with multi-molar concentration gradients and multi-atmosphere pressure changes. In order to better identify potential sources of error, and to point to future directions in modeling and experimental research, we present a three part series to build from first principles a theory of coupled heat and mass transport in cryobiological systems accounting for all of these effects. The hope of this series is that by presenting and justifying all steps, conclusions may be made about the importance of key assumptions, perhaps pointing to areas of future research or model development, but importantly, lending weight to standard simplification arguments that are often made in heat and mass transport. In this first part, we review concentration variable relationships, their impact on choices for Gibbs energy models, and their impact on chemical potentials.     

Parameter Identification of Robot Manipulators: A Heuristic Particle Swarm Search Approach

Parameter identification of robot manipulators is an indispensable pivotal process of achieving accurate dynamic robot models. Since these kinetic models are highly nonlinear, it is not easy to tackle the matter of identifying their parameters. To solve the difficulty effectively, we herewith present an intelligent approach, namely, a heuristic particle swarm optimization (PSO) algorithm, which we call the elitist learning strategy (ELS) and proportional integral derivative (PID) controller hybridized PSO approach (ELPIDSO). A specified PID controller is designed to improve particles’ local and global positions information together with ELS. Parameter identification of robot manipulators is conducted for performance evaluation of our proposed approach. Experimental results clearly indicate the following findings: Compared with standard PSO (SPSO) algorithm, ELPIDSO has improved a lot. It not only enhances the diversity of the swarm, but also features better search effectiveness and efficiency in solving practical optimization problems. Accordingly, ELPIDSO is superior to least squares (LS) method, genetic algorithm (GA), and SPSO algorithm in estimating the parameters of the kinetic models of robot manipulators.     

A competitive clustering particle swarm optimizer for dynamic optimization problems

Optimization in dynamic optimization problems (DOPs) requires the optimization algorithms not only to locate, but also to continuously track the moving optima. Particle swarm optimization (PSO) is a population-based optimization algorithm, originally developed for static problems. Recently, several researchers have proposed variants of PSO for optimization in DOPs. This paper presents a novel multi-swarm PSO algorithm, namely competitive clustering PSO (CCPSO), designed specially for DOPs. Employing a multi-stage clustering procedure, CCPSO splits the particles of the main swarm over a number of sub-swarms based on the particles positions and on their objective function values. The algorithm automatically adjusts the number of sub-swarms and the corresponding region of each sub-swarm. In addition to the sub-swarms, there is also a group of free particles that explore the environment to locate new emerging optima or exploit the current optima which are not followed by any sub-swarm. The adaptive search strategy adopted by the sub-swarms improves both the exploitation and tracking characteristics of CCPSO. A set of experiments is conducted to study the behavior of the proposed algorithm in different DOPs and to provide guidelines for setting the algorithm’s parameters in different problems. The results of CCPSO on a variety of moving peaks benchmark (MPB) functions are compared with those of several state-of-the-art PSO algorithms, indicating the efficiency of the proposed model.     

Optimized Particle Swarm Optimization (OPSO) and its application to artificial neural network training

Background  

Particle Swarm Optimization (PSO) is an established method for parameter optimization. It represents a population-based adaptive optimization technique that is influenced by several "strategy parameters". Choosing reasonable parameter values for the PSO is crucial for its convergence behavior, and depends on the optimization task. We present a method for parameter meta-optimization based on PSO and its application to neural network training. The concept of the Optimized Particle Swarm Optimization (OPSO) is to optimize the free parameters of the PSO by having swarms within a swarm. We assessed the performance of the OPSO method on a set of five artificial fitness functions and compared it to the performance of two popular PSO implementations.     

A Comprehensive Review of Swarm Optimization Algorithms

Many swarm optimization algorithms have been introduced since the early 60’s, Evolutionary Programming to the most recent, Grey Wolf Optimization. All of these algorithms have demonstrated their potential to solve many optimization problems. This paper provides an in-depth survey of well-known optimization algorithms. Selected algorithms are briefly explained and compared with each other comprehensively through experiments conducted using thirty well-known benchmark functions. Their advantages and disadvantages are also discussed. A number of statistical tests are then carried out to determine the significant performances. The results indicate the overall advantage of Differential Evolution (DE) and is closely followed by Particle Swarm Optimization (PSO), compared with other considered approaches.     

Inference of genetic regulatory networks with recurrent neural network models using particle swarm optimization

Genetic regulatory network inference is critically important for revealing fundamental cellular processes, investigating gene functions, and understanding their relations. The availability of time series gene expression data makes it possible to investigate the gene activities of whole genomes, rather than those of only a pair of genes or among several genes. However, current computational methods do not sufficiently consider the temporal behavior of this type of data and lack the capability to capture the complex nonlinear system dynamics. We propose a recurrent neural network (RNN) and particle swarm optimization (PSO) approach to infer genetic regulatory networks from time series gene expression data. Under this framework, gene interaction is explained through a connection weight matrix. Based on the fact that the measured time points are limited and the assumption that the genetic networks are usually sparsely connected, we present a PSO-based search algorithm to unveil potential genetic network constructions that fit well with the time series data and explore possible gene interactions. Furthermore, PSO is used to train the RNN and determine the network parameters. Our approach has been applied to both synthetic and real data sets. The results demonstrate that the RNN/PSO can provide meaningful insights in understanding the nonlinear dynamics of the gene expression time series and revealing potential regulatory interactions between genes.     

Multiclass cancer classification using semisupervised ellipsoid ARTMAP and particle swarm optimization with gene expression data

It is crucial for cancer diagnosis and treatment to accurately identify the site of origin of a tumor. With the emergence and rapid advancement of DNA microarray technologies, constructing gene expression profiles for different cancer types has already become a promising means for cancer classification. In addition to research on binary classification such as normal versus tumor samples, which attracts numerous efforts from a variety of disciplines, the discrimination of multiple tumor types is also important. Meanwhile, the selection of genes which are relevant to a certain cancer type not only improves the performance of the classifiers, but also provides molecular insights for treatment and drug development. Here, we use semisupervised ellipsoid ARTMAP (ssEAM) for multiclass cancer discrimination and particle swarm optimization for informative gene selection. ssEAM is a neural network architecture rooted in adaptive resonance theory and suitable for classification tasks. ssEAM features fast, stable, and finite learning and creates hyperellipsoidal clusters, inducing complex nonlinear decision boundaries. PSO is an evolutionary algorithm-based technique for global optimization. A discrete binary version of PSO is employed to indicate whether genes are chosen or not. The effectiveness of ssEAM/PSO for multiclass cancer diagnosis is demonstrated by testing it on three publicly available multiple-class cancer data sets. ssEAM/PSO achieves competitive performance on all these data sets, with results comparable to or better than those obtained by other classifiers     

基于改进PSO的洞庭湖水源涵养林空间优化模型

以结构化森林经营思想为理论基础,从与水源林涵养水源、保持水土功能密切相关的林分物种组成(树种混交)、种内及种间竞争、空间分布格局、垂直结构4个方面选择混交度、竞争指数、角尺度、林层指数、空间密度指数、开阔比数作为水源林健康经营和林分空间结构优化的目标函数,建立洞庭湖水源林林分多目标空间优化模型,应用改进的群智能粒子群算法求解林分空间结构优化模型,并针对模型输出的目标树空间结构单元制定周密的经营策略.研究结果表明,优化模型能准确定位林分空间关系的薄弱环节,调控措施能显著改善林分空间结构,有利于促进森林生态系统的正向演替,为恢复洞庭湖水源林生态功能和健康经营提供理论依据和技术支撑.应用优化模型进行水源涵养林健康经营突破了传统森林经营模式,为智能信息技术在森林空间经营中的应用提供了新的思路.     

Determining the optimum model parameters for oligosaccharide production efficiency using response surface integrated particle swarm optimization method: an experimental validation study

Abstract

Glucansucrases (GTFs) catalyzes the synthesis of α-glucans from sucrose and oligosaccharides in the presence of an acceptor sugar by transferring glucosyl units to the acceptor molecule with different linkages. The acceptor reactions can be affected by several parameters and this study aimed to determine the optimal reaction parameters for the production of glucansucrase-based oligosaccharides using sucrose and maltose as the donor and acceptor sugars, respectively via a hybrid technique of Response Surface Method (RSM) and Particle Swarm Optimization (PSO). The experimental design was performed using Central Composite Design and the tested parameters were enzyme concentration, acceptor:donor ratio and the reaction period. The optimization studies showed that enzyme concentration was the most effective parameter for the final oligosaccharides yields. The optimal values of the significant parameters determined for enzyme concentration and acceptor:donor ratio were 3.45?U and 0.62, respectively. Even the response surface plots for input parameters verified the PSO results, an experimental validation study was performed for the reverification. The experimental verification results obtained were also consistent with the PSO results. These findings will help our understanding in the role of different parameters for the production of oligosaccharides in the acceptor reactions of GTFs.     

Effective detection of Parkinson's disease using an adaptive fuzzy k-nearest neighbor approach

In this paper, we present an effective and efficient diagnosis system based on particle swarm optimization (PSO) enhanced fuzzy k-nearest neighbor (FKNN) for Parkinson's disease (PD) diagnosis. In the proposed system, named PSO–FKNN, both the continuous version and binary version of PSO were used to perform the parameter optimization and feature selection simultaneously. On the one hand, the neighborhood size k and the fuzzy strength parameter m in FKNN classifier are adaptively specified by the continuous PSO. On the other hand, binary PSO is utilized to choose the most discriminative subset of features for prediction. The effectiveness of the PSO–FKNN model has been rigorously evaluated against the PD data set in terms of classification accuracy, sensitivity, specificity and the area under the receiver operating characteristic (ROC) curve (AUC). Compared to the existing methods in previous studies, the proposed system has achieved the highest classification accuracy reported so far via 10-fold cross-validation analysis, with the mean accuracy of 97.47%. Promisingly, the proposed diagnosis system might serve as a new candidate of powerful tools for diagnosing PD with excellent performance.     

The relationship between nephron number,kidney size and body weight in two inbred mouse strains

While some reports in humans have shown that nephron number is positively correlated with height, body weight or kidney weight, other studies have not reproduced these findings. To understand the impact of genetic and environmental variation on these relationships, we examined whether nephron number correlates with body weight, kidney planar surface area, or kidney weight in two inbred mouse strains with contrasting kidney sizes but no overt renal pathology: C3H/HeJ and C57BL/6J. C3H/HeJ mice had smaller kidneys at birth and larger kidneys by adulthood, however there was no significant difference in nephron number between the two strains. We did observe a correlation between kidney size and body weight at birth and at adulthood for both strains. However, there was no relationship between nephron number and body weight or between nephron number and kidney size. From other studies, it appears that a greater than 2-fold variation is required in each of these parameters in order to demonstrate these relationships, suggesting they are highly dependent on scale. Our results are therefore not surprising since there was a less than 2-fold variation in each of the parameters examined. In summary, the relationship between nephron number and body or kidney size is most likely to be demonstrated when there is greater phenotypic variation either from genetic and/or environmental factors.