Entropic analysis and incremental synthesis of multilayered feedforward neural networks

Neural network architecture optimization is often a critical issue, particularly when VLSI implementation is considered. This paper proposes a new minimization method for multilayered feedforward ANNs and an original approach to their synthesis, both based on the analysis of the information quantity (entropy) flowing through the network. A layer is described as an information filter which selects the relevant characteristics until the complete classification is performed. The basic incremental synthesis method, including the supervised training procedure, is derived to design application-tailored neural paradigms with good generalization capability.     

An Efficacious Multi-Objective Fuzzy Linear Programming Approach for Optimal Power Flow Considering Distributed Generation

This paper proposes a new formulation for the multi-objective optimal power flow (MOOPF) problem for meshed power networks considering distributed generation. An efficacious multi-objective fuzzy linear programming optimization (MFLP) algorithm is proposed to solve the aforementioned problem with and without considering the distributed generation (DG) effect. A variant combination of objectives is considered for simultaneous optimization, including power loss, voltage stability, and shunt capacitors MVAR reserve. Fuzzy membership functions for these objectives are designed with extreme targets, whereas the inequality constraints are treated as hard constraints. The multi-objective fuzzy optimal power flow (OPF) formulation was converted into a crisp OPF in a successive linear programming (SLP) framework and solved using an efficient interior point method (IPM). To test the efficacy of the proposed approach, simulations are performed on the IEEE 30-busand IEEE 118-bus test systems. The MFLP optimization is solved for several optimization cases. The obtained results are compared with those presented in the literature. A unique solution with a high satisfaction for the assigned targets is gained. Results demonstrate the effectiveness of the proposed MFLP technique in terms of solution optimality and rapid convergence. Moreover, the results indicate that using the optimal DG location with the MFLP algorithm provides the solution with the highest quality.     

基于启发式广度优先搜索的SVR参数优化方法研究

支持向量回归机(Support vector regressio,SVR)模型的拟合精度和泛化能力取决于其相关参数的选择,其参数选择实质上是一个优化搜索过程。根据启发式广度优先搜索(Heuristic Breadth first Search,HBFS)算法在求解优化问题上高效的特点,提出了一种以k-fold交叉验证的最小化误差为目标,HBFS为寻优策略的SVR参数选择方法,通过3个基准数据集对该模型进行了仿真实验,结果表明该方法在保证预测精度前提下,大幅度的缩短了训练建模时间,为大样本的SVR参数选择提供了一种新的有效解决方案。     

A selective learning method to improve the generalization of multilayer feedforward neural networks

Multilayer feedforward neural networks with backpropagation algorithm have been used successfully in many applications. However, the level of generalization is heavily dependent on the quality of the training data. That is, some of the training patterns can be redundant or irrelevant. It has been shown that with careful dynamic selection of training patterns, better generalization performance may be obtained. Nevertheless, generalization is carried out independently of the novel patterns to be approximated. In this paper, we present a learning method that automatically selects the training patterns more appropriate to the new sample to be predicted. This training method follows a lazy learning strategy, in the sense that it builds approximations centered around the novel sample. The proposed method has been applied to three different domains: two artificial approximation problems and a real time series prediction problem. Results have been compared to standard backpropagation using the complete training data set and the new method shows better generalization abilities.     

A method for solution of the multi-objective inverse problems under uncertainty

We describe a method to solve multi-objective inverse problems under uncertainty. The method was tested on non-linear models of dynamic series and population dynamics, as well as on the spatiotemporal model of gene expression in terms of non-linear differential equations. We consider how to identify model parameters when experimental data contain additive noise and measurements are performed in discrete time points. We formulate the multi-objective problem of optimization under uncertainty. In addition to a criterion of least squares difference we applied a criterion which is based on the integral of trajectories of the system spatiotemporal dynamics, as well as a heuristic criterion CHAOS based on the decision tree method. The optimization problem is formulated using a fuzzy statement and is constrained by penalty functions based on the normalized membership functions of a fuzzy set of model solutions. This allows us to reconstruct the expression pattern of hairy gene in Drosophila even-skipped mutants that is in good agreement with experimental data. The reproducibility of obtained results is confirmed by solution of inverse problems using different global optimization methods with heuristic strategies.     

基于遗传算法的土地利用优化：NSGA-Ⅱ和NSGA-Ⅲ的对比研究

土地利用优化通常要兼顾不同群体的多种要求,理论上是复杂的超多目标(4个及以上)优化问题。但实际操作中却往往被简化为多目标(2—3个)优化问题,通过一种流行的多目标优化算法第Ⅱ代非支配排序遗传算法(NSGA-Ⅱ)求解。究其原因是对超多目标优化算法认知的缺失和与多目标优化算法理论对比的匮乏。对NSGA系列中应用最广泛的多目标优化算法NSGA-Ⅱ和最新提出、面向超多目标优化的算法NSGA-Ⅲ进行探究,从理论和实验两方面对Ⅲ和Ⅱ进行对比,从而探究二者进行土地利用优化时的优劣。在理论上,对比两种算法原理的异同。在实验中,分别设计多目标(3个目标)和超多目标(13个目标)土地利用优化问题,利用两种算法进行求解。对实验结果采用四层架构、六大指标进行全面评价,以对比两种算法的可用性。理论对比发现,两个算法只有种群多样性保护的方法不同,其中NSGA-Ⅲ是基于与固定的参考点的距离,而NSGA-Ⅱ则是基于相邻解间的距离。通过实验对比发现,NSGA-Ⅲ在超多目标优化时运算速度快,且产生的最优方案实用价值更高,NSGA-Ⅱ在算法的有效性方面更有优势。     

Recognition of gene acceptor site based on multi-objective optimization

A new method for predicting the gene acceptor site based on multi-objective optimization is introduced in this paper. The models for the acceptor, branch and distance between acceptor site and branch site were constructed according to the characteristics of the sequences from the exon-intron database and using common biological knowledge. The acceptor function, branch function and distance function were defined respectively, and the multi-objective optimization model was constructed to recognize the splice site. The test results show that the algorithm used in this study performs better than the SplicePredictor,which is one of the leading acceptor site detectors.     

The assessment of occupational safety and health in Turkey by applying a decision-making method; MULTIMOORA

Abstract

This article presents an application of multi-objective decision-making approach by using multi-objective optimization plus the full multiplicative form (MULTIMOORA) method on the assessment procedure of occupational safety and health based on the counts of occupational accidents and diseases. The method determines the ranks of the years which are decision-making units and designed as time-series analysis. The aim is to determine the occupational safety and health status in Turkey based on the recorded data. The data used in this study indicate an increasing number of worker and occupational safety experts versus a fluctuating trend of the case count of the occupational accidents and diseases. The final ranks obtained from MULTIMOORA show that the last three years assessed in this study have a higher efficiency compared to the previous five years. MULTIMOORA is a simple and quite easy to use method with less computational time.     

Robustness, evolvability, and optimality of evolutionary neural networks

In a typical optimization problem, the main goal is to search for the appropriate values of the variables that provide the optimal solution of the given function. In artificial neural networks (ANN), this translates to the minimization of the error surface during training such that misclassification is minimized during generalization. However, since optimal training performance does not necessarily imply optimal generalization due to the possibility of overfitting or underfitting, we developed SEPA (Structure Evolution and Parameter Adaptation) which addressed these issues by simultaneously evolving ANN structure and weights. Since SEPA primarily relies on the perturbation function to bring variation in its population, this follow-up study aims to find out SEPAs evolvability, optimality, and robustness in other perturbation functions. Our findings indicate that SEPAs optimal generalization performances are stable and robust from the effect of the different perturbation functions. This is due to the feedback loop between its architecture evolution and weight adaptation such that any shortcoming of the former is compensated by the latter, and vice versa. Our results strongly suggest that proper ANN design requires simultaneous adaptation of ANN structure and weights to avoid one-sided or bias convergence to either the weight or architecture space.     

用TOPSIS方法评价黄土丘陵区草场质量的研究

一、自然概况黄土高原位于我国西北部,西起青海日月山,东抵山西太行山,北至晋陕长城,南达甘陕秦岭,构成一个独具特色的、比较完整的自然区域,其地理范围大致是北纬34°—41°,东经103°—113°之间。高原到处有起伏的黄     

The hybrid bacterial foraging algorithm based on many-objective optimizer

A new multi-objective optimized bacterial foraging algorithm - Hybrid Multi-Objective Optimized Bacterial Foraging Algorithm (HMOBFA) is presented in this article. The proposed algorithm combines the crossover-archives strategy and the life-cycle optimization strategy, look for the best method through research area. The crossover-archive strategy with an external archive and internal archive is assigned to different selection principles to focus on diversity and convergence separately. Additionally, according to the local landscape to satisfy population diversity and variability as well as avoiding redundant local searches, individuals can switch their states periodically throughout the colony lifecycle with the life-cycle optimization strategy. all of which may perform significantly well. The performance of the algorithm was examined with several standard criterion functions and compared with other classical multi-objective majorization methods. The examiner results show that the HMOBFA algorithm can achieve a significant enhancement in performance compare with other method and handles many-objective issues with solid complexity, convergence as well as diversity. The HMOBFA algorithm has been proven to be an excellent alternative to past methods for solving the improvement of many-objective problems.     

Genetic algorithm for multi-objective experimental optimization

A new software tool making use of a genetic algorithm for multi-objective experimental optimization (GAME.opt) was developed based on a strength Pareto evolutionary algorithm. The software deals with high dimensional variable spaces and unknown interactions of design variables. This approach was evaluated by means of multi-objective test problems replacing the experimental results. A default parameter setting is proposed enabling users without expert knowledge to minimize the experimental effort (small population sizes and few generations).     

An experimental analysis of design choices of multi-objective ant colony optimization algorithms

There have been several proposals on how to apply the ant colony optimization (ACO) metaheuristic to multi-objective combinatorial optimization problems (MOCOPs). This paper proposes a new formulation of these multi-objective ant colony optimization (MOACO) algorithms. This formulation is based on adding specific algorithm components for tackling multiple objectives to the basic ACO metaheuristic. Examples of these components are how to represent multiple objectives using pheromone and heuristic information, how to select the best solutions for updating the pheromone information, and how to define and use weights to aggregate the different objectives. This formulation reveals more similarities than previously thought in the design choices made in existing MOACO algorithms. The main contribution of this paper is an experimental analysis of how particular design choices affect the quality and the shape of the Pareto front approximations generated by each MOACO algorithm. This study provides general guidelines to understand how MOACO algorithms work, and how to improve their design.     

Collaborative Emission Reduction Model Based on Multi-Objective Optimization for Greenhouse Gases and Air Pollutants

CO₂ emission influences not only global climate change but also international economic and political situations. Thus, reducing the emission of CO₂, a major greenhouse gas, has become a major issue in China and around the world as regards preserving the environmental ecology. Energy consumption from coal, oil, and natural gas is primarily responsible for the production of greenhouse gases and air pollutants such as SO₂ and NO_X, which are the main air pollutants in China. In this study, a mathematical multi-objective optimization method was adopted to analyze the collaborative emission reduction of three kinds of gases on the basis of their common restraints in different ways of energy consumption to develop an economic, clean, and efficient scheme for energy distribution. The first part introduces the background research, the collaborative emission reduction for three kinds of gases, the multi-objective optimization, the main mathematical modeling, and the optimization method. The second part discusses the four mathematical tools utilized in this study, which include the Granger causality test to analyze the causality between air quality and pollutant emission, a function analysis to determine the quantitative relation between energy consumption and pollutant emission, a multi-objective optimization to set up the collaborative optimization model that considers energy consumption, and an optimality condition analysis for the multi-objective optimization model to design the optimal-pole algorithm and obtain an efficient collaborative reduction scheme. In the empirical analysis, the data of pollutant emission and final consumption of energies of Tianjin in 1996–2012 was employed to verify the effectiveness of the model and analyze the efficient solution and the corresponding dominant set. In the last part, several suggestions for collaborative reduction are recommended and the drawn conclusions are stated.     

A new measurement of noise immunity and generalization ability for MLPs

This paper shows a quantitative relation between the regularization techniques, the generalization ability, and the sensitivity of the Multilayer Perceptron (MLP) to input noise. Although many studies about these topics have been presented, in most cases only one of the problems is addressed, and only experimentally obtained evidence is provided to illustrate some kind of correlation between generalization, noise immunity and the use of regularization techniques to obtain a set of weights after training that provides the corresponding MLP with generalization ability and noise immunity. Here, a new measurement of noise immunity for a MLP is presented. This measurement, which is termed Mean Squared Sensitivity (MSS), explicitly evaluates the Mean Squared Error (MSE) degradation of a MLP when it is perturbed by input noise, and can be computed from the statistical sensitivities (previously proposed) of the output neurons. The MSS provides an accurate evaluation of the MLP performance loss when its inputs are perturbed by noise and can also be considered a measurement of the smoothness of the error surface with respect to the inputs. Thus, as the MSS can be used to evaluate the noise immunity or the generalization ability, it gives a criterion to select among different weight configurations that present a similar MSE after training.     

Use of supervised discretization with PCA in wavelet packet transformation-based surface electromyogram classification

This paper describes a preprocessing stage for nonlinear classifier used in wavelet packet transformation (WPT)-based multichannel surface electromyogram (EMG) classification. The preprocessing stage named sdPCA, which consists of supervised discretization coupled with principal component analysis (PCA), was developed for improving surface EMG classifier generalization ability and training speed on overlap segmented signals. The sdPCA outperforms the fast correlation-based filter (FCBF), PCA, supervised discretization, and their combinations in terms of the highest generalization ability, fast training speed, the small feature size, and an ability to reduce the risks of developing oscillation and being trapped in nonlinear classifier training. The experiments were conducted on a data set consisting of 4-channel surface EMG signals measured from 6 hand and wrist gestures of 12 subjects. The experimental results indicate that the classification system using sdPCA has the highest generalization ability along with the second fastest training speed. The classification accuracy in 12 subjects of the system using sdPCA is 93.30 ± 2.42% taking 400 epochs for training by overlap segmented signals within 100 s. This result is very attractive for further development because we can achieve high-classification accuracy for large data sets by means of the proposed sdPCA without the application of additional algorithms such as local discriminant bases (LDB), majority voting (MV), or WPT sub-bands clustering.     

Generalization of motor learning depends on the history of prior action

Generalization of motor learning refers to our ability to apply what has been learned in one context to other contexts. When generalization is beneficial, it is termed transfer, and when it is detrimental, it is termed interference. Insight into the mechanism of generalization may be acquired from understanding why training transfers in some contexts but not others. However, identifying relevant contextual cues has proven surprisingly difficult, perhaps because the search has mainly been for cues that are explicit. We hypothesized instead that a relevant contextual cue is an implicit memory of action with a particular body part. To test this hypothesis we considered a task in which participants learned to control motion of a cursor under visuomotor rotation in two contexts: by moving their hand through motion of their shoulder and elbow, or through motion of their wrist. Use of these contextual cues led to three observations: First, in naive participants, learning in the wrist context was much faster than in the arm context. Second, generalization was asymmetric so that arm training benefited subsequent wrist training, but not vice versa. Third, in people who had prior wrist training, generalization from the arm to the wrist was blocked. That is, prior wrist training appeared to prevent both the interference and transfer that subsequent arm training should have caused. To explain the data, we posited that the learner collected statistics of contextual history: all upper arm movements also move the hand, but occasionally we move our hands without moving the upper arm. In a Bayesian framework, history of limb segment use strongly affects parameter uncertainty, which is a measure of the covariance of the contextual cues. This simple Bayesian prior dictated a generalization pattern that largely reproduced all three findings. For motor learning, generalization depends on context, which is determined by the statistics of how we have previously used the various parts of our limbs.     

微阵列数据的多目标免疫优化双聚类

DNA微阵列技术的发展为基因表达研究提供更有效的工具。分析这些大规模基因数据主要应用聚类方法。最近,提出双聚类技术来发现子矩阵以揭示各种生物模式。多目标优化算法可以同时优化多个相互冲突的目标,因而是求解基因表达矩阵的双聚类的一种很好的方法。本文基于克隆选择原理提出了一个新奇的多目标免疫优化双聚类算法,来挖掘微阵列数据的双聚类。在两个真实数据集上的实验结果表明该方法比其他多目标进化双聚娄算法表现出更优越的性能。     

Integrated framework of system dynamics and meta-heuristic for multi-objective land use planning problem

Land use optimization as a resource allocation problem can be defined as the process of assigning different land uses to a region. Sustainable development also involves the exploitation of environmental resources, investment orientation, technology development, and industrial changes in a coordinated form. This paper studies the multi-objective sustainable land use planning problem and proposes an integrated framework, including simulation, forecasting, and optimization approaches for this problem. Land use optimization, a multifaceted process, requires complex decisions, including selection of land uses, forecasting land use allocation percentage, and assigning locations to land uses. The land use allocation percentage in the selected horizons is simulated and predicted by designing a System Dynamics (SD) model based on socio-economic variables. Furthermore, land use assignment is accomplished with a multi-objective integer programming model that is solved using augmented ε-constraint and non-dominated sorting genetic algorithm II (NSGA-II) methods. According to the results of the SD model, land use changes depend on population growth rate and labor productivity variables. Among the possible scenarios, a scenario focusing more on sustainable planning is chosen and the forecasting results of this scenario are used for optimal land use allocation. The computational results show that the augmented ε-constraint method cannot solve this problem even for medium sizes. The NSGA-II method not only solves the problem at large sizes over a reasonable time, but also generates good-quality solutions. NSGA-II showed better performance in metrics, including number of non-dominated Pareto solutions (NNPS), mean ideal distance (MID), and dispersion metric (DM). Integrated framework is implemented to allocate four types of land uses consisting of residential, commercial, industrial, and agricultural to a given region with 900 cells.

     

Multi-Objective Evolutionary Optimization of Biological Pest Control with Impulsive Dynamics in Soybean Crops

The biological pest control in agriculture, an environment-friendly practice, maintains the density of pests below an economic injury level by releasing a suitable quantity of their natural enemies. This work proposes a multi-objective numerical solution to biological pest control for soybean crops, considering both the cost of application of the control action and the cost of economic damages. The system model is nonlinear with impulsive control dynamics, in order to cope more effectively with the actual control action to be applied, which should be performed in a finite number of discrete time instants. The dynamic optimization problem is solved using the NSGA-II, a fast and trustworthy multi-objective genetic algorithm. The results suggest a dual pest control policy, in which the relative price of control action versus the associated additional harvest yield determines the usage of either a low control action strategy or a higher one.