Pareto-based multi-objective optimization for classification in data mining

This paper explores the possibility of classification based on Pareto multi-objective optimization. The efforts on solving optimization problems using the Pareto-based MOO methodology have gained increasing impetus on comparison of selected constraints. Moreover we have different types of classification problem based on optimization model like single objective optimization, MOO, Pareto optimization and convex optimization. All above techniques fail to generate distinguished class/subclass from existing class based on sensitive data. However, in this regard Pareto-based MOO approach is more powerful and effective in addressing various data mining tasks such as clustering, feature selection, classification, and knowledge extraction. The primary contribution of this paper is to solve such noble classification problem. Our work provides an overview of the existing research on MOO and contribution of Pareto based MOO focusing on classification. Particularly, the entire work deals with association of sub-features for noble classification. Moreover potentially interesting sub-features in MOO for classification are used to strengthen the concept of Pareto based MOO. Experiment has been carried out to validate the theory with different real world data sets which are more sensitive in nature. Finally, experimental results provide effectiveness of the proposed method using sensitive data.     

A divide-and-conquer approach to determine the Pareto frontier for optimization of protein engineering experiments

In developing improved protein variants by site-directed mutagenesis or recombination, there are often competing objectives that must be considered in designing an experiment (selecting mutations or breakpoints): stability versus novelty, affinity versus specificity, activity versus immunogenicity, and so forth. Pareto optimal experimental designs make the best trade-offs between competing objectives. Such designs are not "dominated"; that is, no other design is better than a Pareto optimal design for one objective without being worse for another objective. Our goal is to produce all the Pareto optimal designs (the Pareto frontier), to characterize the trade-offs and suggest designs most worth considering, but to avoid explicitly considering the large number of dominated designs. To do so, we develop a divide-and-conquer algorithm, Protein Engineering Pareto FRontier (PEPFR), that hierarchically subdivides the objective space, using appropriate dynamic programming or integer programming methods to optimize designs in different regions. This divide-and-conquer approach is efficient in that the number of divisions (and thus calls to the optimizer) is directly proportional to the number of Pareto optimal designs. We demonstrate PEPFR with three protein engineering case studies: site-directed recombination for stability and diversity via dynamic programming, site-directed mutagenesis of interacting proteins for affinity and specificity via integer programming, and site-directed mutagenesis of a therapeutic protein for activity and immunogenicity via integer programming. We show that PEPFR is able to effectively produce all the Pareto optimal designs, discovering many more designs than previous methods. The characterization of the Pareto frontier provides additional insights into the local stability of design choices as well as global trends leading to trade-offs between competing criteria.     

Multi-objective optimal control of dynamic bioprocesses using ACADO Toolkit

The optimal design and operation of dynamic bioprocesses gives in practice often rise to optimisation problems with multiple and conflicting objectives. As a result typically not a single optimal solution but a set of Pareto optimal solutions exist. From this set of Pareto optimal solutions, one has to be chosen by the decision maker. Hence, efficient approaches are required for a fast and accurate generation of the Pareto set such that the decision maker can easily and systematically evaluate optimal alternatives. In the current paper the multi-objective optimisation of several dynamic bioprocess examples is performed using the freely available ACADO Multi-Objective Toolkit (http://www.acadotoolkit.org). This toolkit integrates efficient multiple objective scalarisation strategies (e.g., Normal Boundary Intersection and (Enhanced) Normalised Normal Constraint) with fast deterministic approaches for dynamic optimisation (e.g., single and multiple shooting). It has been found that the toolkit is able to efficiently and accurately produce the Pareto sets for all bioprocess examples. The resulting Pareto sets are added as supplementary material to this paper.     

Multi-criterion optimization for heel-toe running

This paper presents an optimization procedure for multi-criterion analysis essential in many biomechanical studies. The optimization is illustrated with a heel-toe running analysis wherein the rate of load and the passive load on support leg are minimized concurrently. The goal of multi-criterion optimization is achieved by incorporating the criterion of Pareto optimality in the genetic algorithm. The proposed procedure can replace the popular weighted-sum approach for problems with multiple objectives. The selection of a final design from the Pareto optimum points (non-dominated designs) can be determined, based on the min-max objective deviation criterion. Nevertheless, a different decision can be made in the final selection without incurring recalculations. The scheme is readily adoptable for parallel computing, which deserves further study to reduce the execution time in a complex biomechanical analysis.     

Pareto Optimization Identifies Diverse Set of Phosphorylation Signatures Predicting Response to Treatment with Dasatinib

Multivariate biomarkers that can predict the effectiveness of targeted therapy in individual patients are highly desired. Previous biomarker discovery studies have largely focused on the identification of single biomarker signatures, aimed at maximizing prediction accuracy. Here, we present a different approach that identifies multiple biomarkers by simultaneously optimizing their predictive power, number of features, and proximity to the drug target in a protein-protein interaction network. To this end, we incorporated NSGA-II, a fast and elitist multi-objective optimization algorithm that is based on the principle of Pareto optimality, into the biomarker discovery workflow. The method was applied to quantitative phosphoproteome data of 19 non-small cell lung cancer (NSCLC) cell lines from a previous biomarker study. The algorithm successfully identified a total of 77 candidate biomarker signatures predicting response to treatment with dasatinib. Through filtering and similarity clustering, this set was trimmed to four final biomarker signatures, which then were validated on an independent set of breast cancer cell lines. All four candidates reached the same good prediction accuracy (83%) as the originally published biomarker. Although the newly discovered signatures were diverse in their composition and in their size, the central protein of the originally published signature — integrin β4 (ITGB4) — was also present in all four Pareto signatures, confirming its pivotal role in predicting dasatinib response in NSCLC cell lines. In summary, the method presented here allows for a robust and simultaneous identification of multiple multivariate biomarkers that are optimized for prediction performance, size, and relevance.     

Bayesian Parameter Estimation and Segmentation in the Multi-Atlas Random Orbit Model

This paper examines the multiple atlas random diffeomorphic orbit model in Computational Anatomy (CA) for parameter estimation and segmentation of subcortical and ventricular neuroanatomy in magnetic resonance imagery. We assume that there exist multiple magnetic resonance image (MRI) atlases, each atlas containing a collection of locally-defined charts in the brain generated via manual delineation of the structures of interest. We focus on maximum a posteriori estimation of high dimensional segmentations of MR within the class of generative models representing the observed MRI as a conditionally Gaussian random field, conditioned on the atlas charts and the diffeomorphic change of coordinates of each chart that generates it. The charts and their diffeomorphic correspondences are unknown and viewed as latent or hidden variables. We demonstrate that the expectation-maximization (EM) algorithm arises naturally, yielding the likelihood-fusion equation which the a posteriori estimator of the segmentation labels maximizes. The likelihoods being fused are modeled as conditionally Gaussian random fields with mean fields a function of each atlas chart under its diffeomorphic change of coordinates onto the target. The conditional-mean in the EM algorithm specifies the convex weights with which the chart-specific likelihoods are fused. The multiple atlases with the associated convex weights imply that the posterior distribution is a multi-modal representation of the measured MRI. Segmentation results for subcortical and ventricular structures of subjects, within populations of demented subjects, are demonstrated, including the use of multiple atlases across multiple diseased groups.     

CNV-CH: A Convex Hull Based Segmentation Approach to Detect Copy Number Variations (CNV) Using Next-Generation Sequencing Data

Copy number variation (CNV) is a form of structural alteration in the mammalian DNA sequence, which are associated with many complex neurological diseases as well as cancer. The development of next generation sequencing (NGS) technology provides us a new dimension towards detection of genomic locations with copy number variations. Here we develop an algorithm for detecting CNVs, which is based on depth of coverage data generated by NGS technology. In this work, we have used a novel way to represent the read count data as a two dimensional geometrical point. A key aspect of detecting the regions with CNVs, is to devise a proper segmentation algorithm that will distinguish the genomic locations having a significant difference in read count data. We have designed a new segmentation approach in this context, using convex hull algorithm on the geometrical representation of read count data. To our knowledge, most algorithms have used a single distribution model of read count data, but here in our approach, we have considered the read count data to follow two different distribution models independently, which adds to the robustness of detection of CNVs. In addition, our algorithm calls CNVs based on the multiple sample analysis approach resulting in a low false discovery rate with high precision.     

Application of Dual Modality Contrast Agent Combined with Multi-Scale Representation in Ultrasound-Magnetic Resonance Imaging Registration Scheme

To achieve the image registration/fusion and perfect the quality of the integration, with dual modality contrast agent (DMCA), a novel multi-scale representation registration method between ultrasound imaging (US) and magnetic resonance imaging (MRI) is presented in the paper, and how DMCA influence on registration accuracy is chiefly discussed. Owing to US’s intense speckle noise, it is a tremendous challenge to register US with any other modality images. How to improve the algorithms for US processing has become the bottleneck, and in the short term it is difficult to have a breakthrough. In that case, DMCA is employed in both US and MRI to enhance the region of interest. Then, because multi-scale representation is a strategy that attempts to diminish or eliminate several possible local minima and lead to convex optimization problems to be solved quickly and more efficiently, a multi-scale representation Gaussian pyramid based affine registration (MRGP-AR) scheme is constructed to complete the US-MRI registration process. In view of the above-mentioned method, the comparison tests indicate that US-MRI registration/fusion may be a remarkable method for gaining high-quality registration image. The experiments also show that it is feasible that novel nano-materials combined with excellent algorithm are used to solve some hard tasks in medical image processing field.     

Pareto optimality,economy–effectiveness trade-offs and ion channel degeneracy: improving population modelling for single neurons

Neurons encounter unavoidable evolutionary trade-offs between multiple tasks. They must consume as little energy as possible while effectively fulfilling their functions. Cells displaying the best performance for such multi-task trade-offs are said to be Pareto optimal, with their ion channel configurations underpinning their functionality. Ion channel degeneracy, however, implies that multiple ion channel configurations can lead to functionally similar behaviour. Therefore, instead of a single model, neuroscientists often use populations of models with distinct combinations of ionic conductances. This approach is called population (database or ensemble) modelling. It remains unclear, which ion channel parameters in the vast population of functional models are more likely to be found in the brain. Here we argue that Pareto optimality can serve as a guiding principle for addressing this issue by helping to identify the subpopulations of conductance-based models that perform best for the trade-off between economy and functionality. In this way, the high-dimensional parameter space of neuronal models might be reduced to geometrically simple low-dimensional manifolds, potentially explaining experimentally observed ion channel correlations. Conversely, Pareto inference might also help deduce neuronal functions from high-dimensional Patch-seq data. In summary, Pareto optimality is a promising framework for improving population modelling of neurons and their circuits.     

Multicriteria optimization of gluconic acid production using net flow

The biochemical process industry is often confronted with the challenge of making decisions in an atmosphere of multiple and conflicting objectives. Recent innovations in the field of operations research and systems science have yielded rigorous multicriteria optimization techniques that can be successfully applied to the field of biochemical engineering. These techniques incorporate the expert's experience into the optimization routine and provide valuable information about the zone of possible solutions. This paper presents a multicriteria optimization strategy that generates a Pareto domain, given a set of conflicting objective criteria, and determines the optimal operating region for the production of gluconic acid using the net flow method (NFM). The objective criteria include maximizing the productivity and concentration of gluconic acid, while minimizing the residual substrate. Three optimization strategies are considered. The first two strategies identify the optimal operating region for the process inputs. The results yielded an acceptable compromise between productivity, gluconic acid concentration and residual substrate concentration. Fixing the process inputs representing the batch time, initial substrate concentration and initial biomass equal to their optimal values, the remaining simulations were used to study the sensitivity of the optimum operating region to changes in the oxygen mass transfer coefficient, K(L) a, by utilizing a multi-level K(L) a strategy. The results show that controlling K(L) a during the reaction reduced the production of biomass, which in turn resulted in increased productivity and concentration of gluconic acid above that of a fixed K(L) a.     

Impact of Stoichiometry Representation on Simulation of Genotype-Phenotype Relationships in Metabolic Networks

Genome-scale metabolic networks provide a comprehensive structural framework for modeling genotype-phenotype relationships through flux simulations. The solution space for the metabolic flux state of the cell is typically very large and optimization-based approaches are often necessary for predicting the active metabolic state under specific environmental conditions. The objective function to be used in such optimization algorithms is directly linked with the biological hypothesis underlying the model and therefore it is one of the most relevant parameters for successful modeling. Although linear combination of selected fluxes is widely used for formulating metabolic objective functions, we show that the resulting optimization problem is sensitive towards stoichiometry representation of the metabolic network. This undesirable sensitivity leads to different simulation results when using numerically different but biochemically equivalent stoichiometry representations and thereby makes biological interpretation intrinsically subjective and ambiguous. We hereby propose a new method, Minimization of Metabolites Balance (MiMBl), which decouples the artifacts of stoichiometry representation from the formulation of the desired objective functions, by casting objective functions using metabolite turnovers rather than fluxes. By simulating perturbed metabolic networks, we demonstrate that the use of stoichiometry representation independent algorithms is fundamental for unambiguously linking modeling results with biological interpretation. For example, MiMBl allowed us to expand the scope of metabolic modeling in elucidating the mechanistic basis of several genetic interactions in Saccharomyces cerevisiae.     

Power analysis for quantitative trait locus mapping in populations derived by multiple backcrosses

 Populations derived by multiple backcrosses are potentially useful for quantitative trait locus (QTL) mapping studies. Comparisons of relative power to detect QTL using populations derived by multiple back-crosses are needed to make decisions when mapping projects are initiated. The objective of this study was to theoretically compare the power to detect QTL in populations derived by multiple backcrosses relative to mapping in a recombinant inbred population of equal size. Backcrossing results in a reduction in genetic variance with each generation and also results in an increasing frequency of the recurrent parent marker genotype. The relevant outcome for QTL mapping is a reduction in genetic variance to partition between marker genotype classes and increasing unbalance of the number of individuals contributing to the mean of the marker genotypes. Both of these factors lead to a decrease in the power to detect a QTL as the number of backcross generations increases. Experimental error was held constant with the populations compared. From a theoretical standpoint, backcross-derived populations offer few advantages for QTL detection. If, however, a backcrossing approach is the most efficient method to achieve a desired breeding objective and if QTL detection is an objective of equal or less importance, backcross-derived populations are a reasonable approach to QTL detection. Received: 4 August 1996 / Accepted: 4 April 1997     

Art pieces that 'move' in our minds--an explanation of illusory motion based on depth reversal

Certain art forms, such as Patrick Hughes's 'reverspectives', Dick Termes's 'Termespheres', intaglios, and hollow masks, appear to move vividly as viewers move in front of them, even though they are stationary. This illusory motion is accompanied by a perceived reversal of depth, where physical convex and concave surfaces are falsely seen as concave and convex, respectively. A geometric explanation is presented that considers this illusory motion as a result of the perceived depth reversal. The main argument is that the visual system constructs a three-dimensional representation of the surfaces, and that this representation is one of the sources that contribute to the illusory motion, together with vestibular signals of self-motion and signals of eye movements. This explanation is extended to stereograms that are also known to appear to move as viewers move in front of them. A quantitative model can be developed around this geometric explanation to examine the extent to which the visual system tolerates large distortions in size and shape and still maintains the illusion.     

Searching for the Pareto frontier in multi-objective protein design

The goal of protein engineering and design is to identify sequences that adopt three-dimensional structures of desired function. Often, this is treated as a single-objective optimization problem, identifying the sequence–structure solution with the lowest computed free energy of folding. However, many design problems are multi-state, multi-specificity, or otherwise require concurrent optimization of multiple objectives. There may be tradeoffs among objectives, where improving one feature requires compromising another. The challenge lies in determining solutions that are part of the Pareto optimal set—designs where no further improvement can be achieved in any of the objectives without degrading one of the others. Pareto optimality problems are found in all areas of study, from economics to engineering to biology, and computational methods have been developed specifically to identify the Pareto frontier. We review progress in multi-objective protein design, the development of Pareto optimization methods, and present a specific case study using multi-objective optimization methods to model the tradeoff between three parameters, stability, specificity, and complexity, of a set of interacting synthetic collagen peptides.     

Two game-based solution concepts for a two-agent scheduling problem

In the current research papers on multi-agent (multi-person) scheduling, a person’s objective function is always considered as a cost function on scheduling, whereas a cooperative profit function is defined to serve as his objective one in this paper. In the two-person scheduling problem addressed in this paper, the two persons jointly order a common operational time interval of a single machine. Each person needs to process a set of his own jobs in that time window. The same objective function of each person still relies on the sequence of all the jobs of both persons since each part of the function is determined by some given parameters except one part assumed to be a given multiple of the total completion time of his own jobs. The two persons have to negotiate a job sequence and determine the (related) final solution on cooperative profit allocation. Such a two-person scheduling problem is essentially a cooperative game. An algorithm is designed to yield the cooperative-profit-based Pareto efficient solution set acting as the first game-based solution concept in this paper. The parallelized version of the algorithm is also developed. The second game-based solution concept is the Shapley value appropriate for the above cooperative-game situation on two-person scheduling. Several instances are presented and analyzed to reveal the necessity to employ the two solution concepts together.     

Neural optimization: Understanding trade-offs with Pareto theory

Nervous systems, like any organismal structure, have been shaped by evolutionary processes to increase fitness. The resulting neural ’bauplan’ has to account for multiple objectives simultaneously, including computational function, as well as additional factors such as robustness to environmental changes and energetic limitations. Oftentimes these objectives compete, and quantification of the relative impact of individual optimization targets is non-trivial. Pareto optimality offers a theoretical framework to decipher objectives and trade-offs between them. We, therefore, highlight Pareto theory as a useful tool for the analysis of neurobiological systems from biophysically detailed cells to large-scale network structures and behavior. The Pareto approach can help to assess optimality, identify relevant objectives and their respective impact, and formulate testable hypotheses.     

The application of a Pareto optimisation method in the design of an integrated bioprocess

A rapid method for designing integrated bioprocesses, using a combination of a windows of operation and a Pareto optimisation approach, is described in this paper. Within bioprocesses, multiple objectives are common, and achieving a satisfactory trade-off amongst the design objectives is crucial. Conventional optimisation results in the identification of the best operating policy for a given desired performance but gives little insight into how the process performance changes in the vicinity of the solution. In this paper, we explore the use of a Pareto optimisation technique to locate the optimal conditions for an integrated bioprocessing sequence and the benefits of first reducing the feasible space by the development of a series of windows of operation to provide a smaller search area for the optimisation. The final results are then presented in performance trade-off graphs and look-up tables, which give the design engineer an easily manageable solution set to work with. In this way, the decision-making procedure for design is made faster and more transparent. Two case studies illustrate the results from this integrated design methodology, some of which are counter-intuitive compared with the general design experience.     

Assessment of uncertainty in functional-structural plant models

Background and Aims

Constructing functional–structural plant models (FSPMs) is a valuable method for examining how physiology and morphology interact in determining plant processes. However, such models always have uncertainty concerned with whether model components have been selected and represented effectively, with the number of model outputs simulated and with the quality of data used in assessment. We provide a procedure for defining uncertainty of an FSPM and how this uncertainty can be reduced.

Methods

An important characteristic of FSPMs is that typically they calculate many variables. These can be variables that the model is designed to predict and also variables that give indications of how the model functions. Together these variables are used as criteria in a method of multi-criteria assessment. Expected ranges are defined and an evolutionary computation algorithm searches for model parameters that achieve criteria within these ranges. Typically, different combinations of model parameter values provide solutions achieving different combinations of variables within their specified ranges. We show how these solutions define a Pareto Frontier that can inform about the functioning of the model.

Key Results

The method of multi-criteria assessment is applied to development of BRANCHPRO, an FSPM for foliage reiteration on old-growth branches of Pseudotsuga menziesii. A geometric model utilizing probabilities for bud growth is developed into a causal explanation for the pattern of reiteration found on these branches and how this pattern may contribute to the longevity of this species.

Conclusions

FSPMs should be assessed by their ability to simulate multiple criteria simultaneously. When different combinations of parameter values achieve different groups of assessment criteria effectively a Pareto Frontier can be calculated and used to define the sources of model uncertainty.