Density estimation using a new dimension adaptive particle swarm optimization algorithm

Current Particle Swarm Optimization (PSO) algorithms do not address problems with unknown dimensions, which arise in many applications that would benefit from the use of PSO. In this paper, we propose a new algorithm, called Dimension Adaptive Particle Swarm Optimization (DA-PSO) that can address problems with any number of dimensions. We also propose and compare three other PSO-based methods with DA-PSO. We apply our algorithms to solve the Weibull mixture model density estimation problem as an illustration. DA-PSO achieves better objective function values than other PSO-based algorithms on four simulated datasets and a real dataset. We also compare DA-PSO with the recursive Expectation-Maximization (EM) estimator, which is a non-PSO-based method, obtaining again very good results.     

A new particle swarm optimization algorithm for noisy optimization problems

We propose a new particle swarm optimization algorithm for problems where objective functions are subject to zero-mean, independent, and identically distributed stochastic noise. While particle swarm optimization has been successfully applied to solve many complex deterministic nonlinear optimization problems, straightforward applications of particle swarm optimization to noisy optimization problems are subject to failure because the noise in objective function values can lead the algorithm to incorrectly identify positions as the global/personal best positions. Instead of having the entire swarm follow a global best position based on the sample average of objective function values, the proposed new algorithm works with a set of statistically global best positions that include one or more positions with objective function values that are statistically equivalent, which is achieved using a combination of statistical subset selection and clustering analysis. The new PSO algorithm can be seamlessly integrated with adaptive resampling procedures to enhance the capability of PSO to cope with noisy objective functions. Numerical experiments demonstrate that the new algorithm is able to consistently find better solutions than the canonical particle swarm optimization algorithm in the presence of stochastic noise in objective function values with different resampling procedures.     

Evaluation of a particle swarm algorithm for biomechanical optimization

Optimization is frequently employed in biomechanics research to solve system identification problems, predict human movement, or estimate muscle or other internal forces that cannot be measured directly. Unfortunately, biomechanical optimization problems often possess multiple local minima, making it difficult to find the best solution. Furthermore, convergence in gradient-based algorithms can be affected by scaling to account for design variables with different length scales or units. In this study we evaluate a recently-developed version of the particle swarm optimization (PSO) algorithm to address these problems. The algorithm's global search capabilities were investigated using a suite of difficult analytical test problems, while its scale-independent nature was proven mathematically and verified using a biomechanical test problem. For comparison, all test problems were also solved with three off-the-shelf optimization algorithms--a global genetic algorithm (GA) and multistart gradient-based sequential quadratic programming (SQP) and quasi-Newton (BFGS) algorithms. For the analytical test problems, only the PSO algorithm was successful on the majority of the problems. When compared to previously published results for the same problems, PSO was more robust than a global simulated annealing algorithm but less robust than a different, more complex genetic algorithm. For the biomechanical test problem, only the PSO algorithm was insensitive to design variable scaling, with the GA algorithm being mildly sensitive and the SQP and BFGS algorithms being highly sensitive. The proposed PSO algorithm provides a new off-the-shelf global optimization option for difficult biomechanical problems, especially those utilizing design variables with different length scales or units.     

A locally convergent rotationally invariant particle swarm optimization algorithm

Several well-studied issues in the particle swarm optimization algorithm are outlined and some earlier methods that address these issues are investigated from the theoretical and experimental points of view. These issues are the: stagnation of particles in some points in the search space, inability to change the value of one or more decision variables, poor performance when the swarm size is small, lack of guarantee to converge even to a local optimum (local optimizer), poor performance when the number of dimensions grows, and sensitivity of the algorithm to the rotation of the search space. The significance of each of these issues is discussed and it is argued that none of the particle swarm optimizers we are aware of can address all of these issues at the same time. To address all of these issues at the same time, a new general form of velocity update rule for the particle swarm optimization algorithm that contains a user-definable function \(f\) is proposed. It is proven that the proposed velocity update rule guarantees to address all of these issues if the function \(f\) satisfies the following two conditions: (i) the function \(f\) is designed in such a way that for any input vector \(\vec {y}\) in the search space, there exists a region \(A\) which contains \(\vec {y}\) and \( f\!\left( {\vec {y}} \right) \) can be located anywhere in \(A\) , and (ii) \(f\) is invariant under any affine transformation. An example of function \(f\) is provided that satisfies these conditions and its performance is examined through some experiments. The experiments confirm that the proposed algorithm (with an appropriate function \(f)\) can effectively address all of these issues at the same time. Also, comparisons with earlier methods show that the overall ability of the proposed method for solving benchmark functions is significantly better.     

Gene selection using hybrid binary black hole algorithm and modified binary particle swarm optimization

In cancer classification, gene selection is an important data preprocessing technique, but it is a difficult task due to the large search space. Accordingly, the objective of this study is to develop a hybrid meta-heuristic Binary Black Hole Algorithm (BBHA) and Binary Particle Swarm Optimization (BPSO) (4-2) model that emphasizes gene selection. In this model, the BBHA is embedded in the BPSO (4-2) algorithm to make the BPSO (4-2) more effective and to facilitate the exploration and exploitation of the BPSO (4-2) algorithm to further improve the performance. This model has been associated with Random Forest Recursive Feature Elimination (RF-RFE) pre-filtering technique. The classifiers which are evaluated in the proposed framework are Sparse Partial Least Squares Discriminant Analysis (SPLSDA); k-nearest neighbor and Naive Bayes. The performance of the proposed method was evaluated on two benchmark and three clinical microarrays. The experimental results and statistical analysis confirm the better performance of the BPSO (4-2)-BBHA compared with the BBHA, the BPSO (4-2) and several state-of-the-art methods in terms of avoiding local minima, convergence rate, accuracy and number of selected genes. The results also show that the BPSO (4-2)-BBHA model can successfully identify known biologically and statistically significant genes from the clinical datasets.     

Robust nonlinear parameter estimation in tracer kinetic analysis using infinity norm regularization and particle swarm optimization

In positron emission tomography (PET) studies, the voxel-wise calculation of individual rate constants describing the tracer kinetics is quite challenging because of the nonlinear relationship between the rate constants and PET data and the high noise level in voxel data. Based on preliminary simulations using a standard two-tissue compartment model, we can hypothesize that it is possible to reduce errors in the rate constant estimates when constraining the overestimation of the larger of two exponents in the model equation. We thus propose a novel approach based on infinity-norm regularization for limiting this exponent. Owing to the non-smooth cost function of this regularization scheme, which prevents the use of conventional Jacobian-based optimization methods, we examined a proximal gradient algorithm and the particle swarm optimization (PSO) through a simulation study. Because it exploits multiple initial values, the PSO method shows much better convergence than the proximal gradient algorithm, which is susceptible to the initial values. In the implementation of PSO, the use of a Gamma distribution to govern random movements was shown to improve the convergence rate and stability compared to a uniform distribution. Consequently, Gamma-based PSO with regularization was shown to outperform all other methods tested, including the conventional basis function method and Levenberg–Marquardt algorithm, in terms of its statistical properties.     

A general method for biological parameter estimation

Summary A general observer-based estimator method is developed and applied for process modelling and monitoring. This parameter estimation technique was successfully applied to a L-lysine fermentation process. It was a useful tool to detect the effect of major culture conditions on cell growth and product synthesis. It can also be used for the development of adaptive optimal control schemes.     

Performance evaluation of TRIBES, an adaptive particle swarm optimization algorithm

This paper presents a study of the performance of TRIBES, an adaptive particle swarm optimization algorithm. Particle Swarm Optimization (PSO) is a biologically-inspired optimization method. Recently, researchers have used it effectively in solving various optimization problems. However, like most optimization heuristics, PSO suffers from the drawback of being greatly influenced by the selection of its parameter values. Thus, the common belief is that the performance of a PSO algorithm is directly related to the tuning of such parameters. Usually, such tuning is a lengthy, time consuming and delicate process. A new adaptive PSO algorithm called TRIBES avoids manual tuning by defining adaptation rules which aim at automatically changing the particles’ behaviors as well as the topology of the swarm. In TRIBES, the topology is changed according to the swarm behavior and the strategies of displacement are chosen according to the performances of the particles. A comparative study carried out on a large set of benchmark functions shows that the performance of TRIBES is quite competitive compared to most other similar PSO algorithms that need manual tuning of parameters. The performance evaluation of TRIBES follows the testing procedure introduced during the 2005 IEEE Conference on Evolutionary Computation. The main objective of the present paper is to perform a global study of the behavior of TRIBES under several conditions, in order to determine strengths and drawbacks of this adaptive algorithm.     

A competitive clustering particle swarm optimizer for dynamic optimization problems

Optimization in dynamic optimization problems (DOPs) requires the optimization algorithms not only to locate, but also to continuously track the moving optima. Particle swarm optimization (PSO) is a population-based optimization algorithm, originally developed for static problems. Recently, several researchers have proposed variants of PSO for optimization in DOPs. This paper presents a novel multi-swarm PSO algorithm, namely competitive clustering PSO (CCPSO), designed specially for DOPs. Employing a multi-stage clustering procedure, CCPSO splits the particles of the main swarm over a number of sub-swarms based on the particles positions and on their objective function values. The algorithm automatically adjusts the number of sub-swarms and the corresponding region of each sub-swarm. In addition to the sub-swarms, there is also a group of free particles that explore the environment to locate new emerging optima or exploit the current optima which are not followed by any sub-swarm. The adaptive search strategy adopted by the sub-swarms improves both the exploitation and tracking characteristics of CCPSO. A set of experiments is conducted to study the behavior of the proposed algorithm in different DOPs and to provide guidelines for setting the algorithm’s parameters in different problems. The results of CCPSO on a variety of moving peaks benchmark (MPB) functions are compared with those of several state-of-the-art PSO algorithms, indicating the efficiency of the proposed model.     

A path planning method based on the particle swarm optimization trained fuzzy neural network algorithm

Cluster Computing - The basic fuzzy neural network algorithm has slow convergence and large amount of calculation, so this paper designed a particle swarm optimization trained fuzzy neural network...     

Inertia weight control strategies for particle swarm optimization

Particle swarm optimization (PSO) is a population-based, stochastic optimization technique inspired by the social dynamics of birds. The PSO algorithm is rather sensitive to the control parameters, and thus, there has been a significant amount of research effort devoted to the dynamic adaptation of these parameters. The focus of the adaptive approaches has largely revolved around adapting the inertia weight as it exhibits the clearest relationship with the exploration/exploitation balance of the PSO algorithm. However, despite the significant amount of research efforts, many inertia weight control strategies have not been thoroughly examined analytically nor empirically. Thus, there are a plethora of choices when selecting an inertia weight control strategy, but no study has been comprehensive enough to definitively guide the selection. This paper addresses these issues by first providing an overview of 18 inertia weight control strategies. Secondly, conditions required for the strategies to exhibit convergent behaviour are derived. Finally, the inertia weight control strategies are empirically examined on a suite of 60 benchmark problems. Results of the empirical investigation show that none of the examined strategies, with the exception of a randomly selected inertia weight, even perform on par with a constant inertia weight.     

A speculative approach to parallelization in particle swarm optimization

Particle swarm optimization (PSO) has previously been parallelized primarily by distributing the computation corresponding to particles across multiple processors. In these approaches, the only benefit of additional processors is an increased swarm size. However, in many cases this is not efficient when scaled to very large swarm sizes (on very large clusters). Current methods cannot answer well the question: “How can 1000 processors be fully utilized when 50 or 100 particles is the most efficient swarm size?” In this paper we attempt to answer that question with a speculative approach to the parallelization of PSO that we refer to as SEPSO.     

A computationally efficient optimization kernel for material parameter estimation procedures

Estimating material parameters is an important part in the study of soft tissue mechanics. Computational time can easily run to days, especially when all available experimental data are taken into account. The material parameter estimation procedure is examplified on a set of homogeneous simple shear experiments to estimate the orthotropic constitutive parameters of myocardium. The modification consists of changing the traditional least-squares approach to a weighted least-squares. This objective function resembles a L(2)-norm type integral which is approximated using Gaussian quadrature. This reduces the computational time of the material parameter estimation by two orders of magnitude.     

Tag SNP selection using particle swarm optimization

Single nucleotide polymorphisms (SNPs) are the most abundant form of genetic variations amongst species. With the genome‐wide SNP discovery, many genome‐wide association studies are likely to identify multiple genetic variants that are associated with complex diseases. However, genotyping all existing SNPs for a large number of samples is still challenging even though SNP arrays have been developed to facilitate the task. Therefore, it is essential to select only informative SNPs representing the original SNP distributions in the genome (tag SNP selection) for genome‐wide association studies. These SNPs are usually chosen from haplotypes and called haplotype tag SNPs (htSNPs). Accordingly, the scale and cost of genotyping are expected to be largely reduced. We introduce binary particle swarm optimization (BPSO) with local search capability to improve the prediction accuracy of STAMPA. The proposed method does not rely on block partitioning of the genomic region, and consistently identified tag SNPs with higher prediction accuracy than either STAMPA or SVM/STSA. We compared the prediction accuracy and time complexity of BPSO to STAMPA and an SVM‐based (SVM/STSA) method using publicly available data sets. For STAMPA and SVM/STSA, BPSO effective improved prediction accuracy for smaller and larger scale data sets. These results demonstrate that the BPSO method selects tag SNP with higher accuracy no matter the scale of data sets is used. © 2009 American Institute of Chemical Engineers Biotechnol. Prog., 2010     

Application of artificial neural network coupling particle swarm optimization algorithm to biocatalytic production of GABA

The biotransformation of L-sodium glutamate (L-MSG) to gamma-aminobutyric acid (GABA) catalyzed by the cells of Lactobacillus brevis with higher glutamate decarboxylase activity was investigated. The results showed that pH, temperature, and FeSO(4) x 7H(2)O concentration had significantly positive effect on GABA yield. The individual and interactive effects of pH, temperature, and FeSO(4) x 7H(2)O concentration were further optimized in terms of GABA yield. In the present work, an artificial neural network (ANN) and response surface methodology (RSM) models were developed, which incorporated pH, temperature, and FeSO(4) x 7H(2)O concentration as input variables, and GABA yield as output variable. The optimized ANN topology included four neurons in the hidden layer and the best network architecture was 3-4-1. The trained ANN gave total root-mean square error (sigma) equal to 1.84 for GABA yield while the RSM gave sigma equal to 2.63. The results demonstrated a slightly higher prediction accuracy of ANN compared to RSM. The modeled maximum GABA yield was identified by applying particle swarm optimization algorithm to the ANN model developed. The modeled maximum GABA yield reached 91 mM under the following optimal conditions: 25 mL Na(2)HPO(4)-citric acid buffer (100 mM, pH 4.23), 120 mM L-MSG, 0.83 g/L FeSO(4) x 7H(2)O, 10 microM PLP, the resting cells obtained from a 60-h culture broth, 2.68 g dry cell weight (DCW)/L, and without agitation at 40 degrees C for 5 h. The previous high value of GABA yield that was observed was 81.8 mM. The optimized conditions allowed GABA yield to be increased from 81.8 to 90.57 mM after verification experiments test.     

Flow simulation-based particle swarm optimization for developing improved hemolysis models

The improvement and development of blood-contacting devices, such as mechanical circulatory support systems, is a life saving endeavor. These devices must be designed in such a way that they ensure the highest hemocompatibility. Therefore, in-silico trials (flow simulations) offer a quick and cost-effective way to analyze and optimize the hemocompatibility and performance of medical devices. In that regard, the prediction of blood trauma, such as hemolysis, is the key element to ensure the hemocompatibility of a device. But, despite decades of research related to numerical hemolysis models, their accuracy and reliability leaves much to be desired. This study proposes a novel optimization path, which is capable of improving existing models and aid in the development of future hemolysis models. First, flow simulations of three, turbulent blood flow test cases (capillary tube, FDA nozzle, FDA pump) were performed and hemolysis was numerically predicted by the widely-applied stress-based hemolysis models. Afterward, a multiple-objective particles swarm optimization (MOPSO) was performed to tie the physiological stresses of the simulated flow field to the measured hemolysis using an equivalent of over one million numerically determined hemolysis predictions. The results show that our optimization is capable of improving upon existing hemolysis models. However, it also unveils some deficiencies and limits of hemolysis prediction with stress-based models, which will need to be addressed in order to improve its reliability.

     

Self-adaptive particle swarm optimization: a review and analysis of convergence

Particle swarm optimization (PSO) is a population-based, stochastic search algorithm inspired by the flocking behaviour of birds. The PSO algorithm has been shown to be rather sensitive to its control parameters, and thus, performance may be greatly improved by employing appropriately tuned parameters. However, parameter tuning is typically a time-intensive empirical process. Furthermore, a priori parameter tuning makes the implicit assumption that the optimal parameters of the PSO algorithm are not time-dependent. To address these issues, self-adaptive particle swarm optimization (SAPSO) algorithms adapt their control parameters throughout execution. While there is a wide variety of such SAPSO algorithms in the literature, their behaviours are not well understood. Specifically, it is unknown whether these SAPSO algorithms will even exhibit convergent behaviour. This paper addresses this lack of understanding by investigating the convergence behaviours of 18 SAPSO algorithms both analytically and empirically. This paper also empirically examines whether the adapted parameters reach a stable point and whether the final parameter values adhere to a well-known convergence criterion. The results depict a grim state for SAPSO algorithms; over half of the SAPSO algorithms exhibit divergent behaviour while many others prematurely converge.     

Distributed scalable multi-robot learning using particle swarm optimization

Designing effective behavioral controllers for mobile robots can be difficult and tedious; this process can be circumvented by using online learning techniques which allow robots to generate their own controllers online in an automated fashion. In multi-robot systems, robots operating in parallel can potentially learn at a much faster rate by sharing information amongst themselves. In this work, we use an adapted version of the Particle Swarm Optimization algorithm in order to accomplish distributed online robotic learning in groups of robots with access to only local information. The effectiveness of the learning technique on a benchmark task (generating high-performance obstacle avoidance behavior) is evaluated for robot groups of various sizes, with the maximum group size allowing each robot to individually contain and manage a single PSO particle. To increase the realism of the technique, different PSO neighborhoods based on limitations of real robotic communication are tested and compared in this scenario. We explore the effect of varying communication power for one of these communication-based PSO neighborhoods. To validate the effectiveness of these learning techniques, fully distributed online learning experiments are run using a group of 10 real robots, generating results which support the findings from our simulations.