A hybrid neural learning algorithm using evolutionary learning and derivative free local search method

In this paper we investigate a hybrid model based on the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. Also we discuss different variants for hybrid models using the Discrete Gradient method and an evolutionary strategy for determining the weights in a feed forward artificial neural network. The Discrete Gradient method has the advantage of being able to jump over many local minima and find very deep local minima. However, earlier research has shown that a good starting point for the discrete gradient method can improve the quality of the solution point. Evolutionary algorithms are best suited for global optimisation problems. Nevertheless they are cursed with longer training times and often unsuitable for real world application. For optimisation problems such as weight optimisation for ANNs in real world applications the dimensions are large and time complexity is critical. Hence the idea of a hybrid model can be a suitable option. In this paper we propose different fusion strategies for hybrid models combining the evolutionary strategy with the discrete gradient method to obtain an optimal solution much quicker. Three different fusion strategies are discussed: a linear hybrid model, an iterative hybrid model and a restricted local search hybrid model. Comparative results on a range of standard datasets are provided for different fusion hybrid models.     

Evolution of innate behavioral strategies through competitive population dynamics

Many organism behaviors are innate or instinctual and have been “hard-coded” through evolution. Current approaches to understanding these behaviors model evolution as an optimization problem in which the traits of organisms are assumed to optimize an objective function representing evolutionary fitness. Here, we use a mechanistic birth-death dynamics approach to study the evolution of innate behavioral strategies in a simulated population of organisms. In particular, we performed agent-based stochastic simulations and mean-field analyses of organisms exploring random environments and competing with each other to find locations with plentiful resources. We find that when organism density is low, the mean-field model allows us to derive an effective objective function, predicting how the most competitive phenotypes depend on the exploration-exploitation trade-off between the scarcity of high-resource sites and the increase in birth rate those sites offer organisms. However, increasing organism density alters the most competitive behavioral strategies and precludes the derivation of a well-defined objective function. Moreover, there exists a range of densities for which the coexistence of many phenotypes persists for evolutionarily long times.     

Stochastic stability of particle swarm optimisation

Particle swarm optimisation (PSO) is a metaheuristic algorithm used to find good solutions in a wide range of optimisation problems. The success of metaheuristic approaches is often dependent on the tuning of the control parameters. As the algorithm includes stochastic elements that effect the behaviour of the system, it may be studied using the framework of random dynamical systems (RDS). In PSO, the swarm dynamics are quasi-linear, which enables an analytical treatment of their stability. Our analysis shows that the region of stability extends beyond those predicted by earlier approximate approaches. Simulations provide empirical backing for our analysis and show that the best performance is achieved in the asymptotic case where the parameters are selected near the margin of instability predicted by the RDS approach.     

Fragment screening: an introduction

There are clearly many different philosophies associated with adapting fragment screening into mainstream Drug Discovery Lead Generation strategies. Scientists at Astex, for instance, focus entirely on strategies involving use of X-ray crystallography and NMR. However, AstraZeneca uses a number of different fragment screening strategies. One approach is to screen a 2000 compound fragment set (with close to "lead-like" complexity) at 100 microM in parallel with every HTS such that the data are obtained on the entire screening collection at 10 microM plus the extra samples at 100 microM; this provides valuable compound potency data in a concentration range that is usually unexplored. The fragments are then screen-specific "privileged structures" that can be searched for in the rest of the HTS output and other databases as well as having synthesis follow-up. A typical workflow for a fragment screen within AstraZeneca is shown below (Figure 24) and highlights the desirability (particularly when screening >100 microM) for NMR and X-ray information to validate weak hits and give information on how to optimise them. In this chapter, we have provided an introduction to the theoretical and practical issues associated with the use of fragment methods and lead-likeness. Fragment-based approaches are still in an early stage of development and are just one of many interrelated techniques that are now used to identify novel lead compounds for drug development. Fragment based screening has some advantages, but like every other drug hunting strategy will not be universally applicable. There are in particular some practical challenges associated with fragment screening that relate to the generally lower level of potency that such compounds initially possess. Considerable synthetic effort has to be applied for post-fragment screening to build the sort of potency that would be expected to be found from a traditional HTS. However, if there are no low-hanging fruit in a screening collection to be found by HTS then the use of fragment screening can help find novelty that may lead to a target not being discarded as intractable. As such, the approach offers some significant advantages by providing less complex molecules, which may have better potential for novel drug optimisation and by enabling new chemical space to be more effectively explored. Many literature examples that cover examples of fragment screening approaches are still at the "proof of concept" stage and although delivering inhibitors or ligands, may still prove to be unsuitable when further ADMET and toxicity profiling is done. The next few years should see a maturing of the area, and as our understanding of how the concepts can be best applied, there are likely to be many more examples of attractive, small molecule hits, leads and candidate drugs derived from the approaches described.     

The emergence of diversity in plant communities

The diversity of functional forms and strategies in plant communities is essential to the maintenance of the services that ecosystems provide humanity, and ultimately to the homeostasis of the biosphere. This diversity emerges from evolutionary forces operating at lower levels; these exploit the opportunities for specialization presented by exogenous and endogenous spatial and temporal heterogeneity. Two major theoretical approaches have been taken to understand how strategies arise and are maintained: optimization models, which consider the fitnesses of types in isolation, and game-theoretic methods, which take frequency dependence into account. The game-theoretic approach is more powerful, but also more challenging to apply. For some relatively simple problems in the study of biodiversity, we show how the game-theoretic formulation can be translated into an equivalent problem in optimization. More generally, however, new techniques will be needed to explore the dynamics of multiple coexisting types and strategies.     

Multi-objective optimal control of dynamic bioprocesses using ACADO Toolkit

The optimal design and operation of dynamic bioprocesses gives in practice often rise to optimisation problems with multiple and conflicting objectives. As a result typically not a single optimal solution but a set of Pareto optimal solutions exist. From this set of Pareto optimal solutions, one has to be chosen by the decision maker. Hence, efficient approaches are required for a fast and accurate generation of the Pareto set such that the decision maker can easily and systematically evaluate optimal alternatives. In the current paper the multi-objective optimisation of several dynamic bioprocess examples is performed using the freely available ACADO Multi-Objective Toolkit (http://www.acadotoolkit.org). This toolkit integrates efficient multiple objective scalarisation strategies (e.g., Normal Boundary Intersection and (Enhanced) Normalised Normal Constraint) with fast deterministic approaches for dynamic optimisation (e.g., single and multiple shooting). It has been found that the toolkit is able to efficiently and accurately produce the Pareto sets for all bioprocess examples. The resulting Pareto sets are added as supplementary material to this paper.     

The seven deadly sins of comparative analysis

Phylogenetic comparative methods are extremely commonly used in evolutionary biology. In this paper, I highlight some of the problems that are frequently encountered in comparative analyses and review how they can be fixed. In broad terms, the problems boil down to a lack of appreciation of the underlying assumptions of comparative methods, as well as problems with implementing methods in a manner akin to more familiar statistical approaches. I highlight that the advent of more flexible computing environments should improve matters and allow researchers greater scope to explore methods and data.     

A hierarchical method for finding optimal architecture and weights using evolutionary least square based learning

In this paper, we present a novel approach of implementing a combination methodology to find appropriate neural network architecture and weights using an evolutionary least square based algorithm (GALS).1 This paper focuses on aspects such as the heuristics of updating weights using an evolutionary least square based algorithm, finding the number of hidden neurons for a two layer feed forward neural network, the stopping criterion for the algorithm and finally some comparisons of the results with other existing methods for searching optimal or near optimal solution in the multidimensional complex search space comprising the architecture and the weight variables. We explain how the weight updating algorithm using evolutionary least square based approach can be combined with the growing architecture model to find the optimum number of hidden neurons. We also discuss the issues of finding a probabilistic solution space as a starting point for the least square method and address the problems involving fitness breaking. We apply the proposed approach to XOR problem, 10 bit odd parity problem and many real-world benchmark data sets such as handwriting data set from CEDAR, breast cancer and heart disease data sets from UCI ML repository. The comparative results based on classification accuracy and the time complexity are discussed.     

Multi-Objective Community Detection Based on Memetic Algorithm

Community detection has drawn a lot of attention as it can provide invaluable help in understanding the function and visualizing the structure of networks. Since single objective optimization methods have intrinsic drawbacks to identifying multiple significant community structures, some methods formulate the community detection as multi-objective problems and adopt population-based evolutionary algorithms to obtain multiple community structures. Evolutionary algorithms have strong global search ability, but have difficulty in locating local optima efficiently. In this study, in order to identify multiple significant community structures more effectively, a multi-objective memetic algorithm for community detection is proposed by combining multi-objective evolutionary algorithm with a local search procedure. The local search procedure is designed by addressing three issues. Firstly, nondominated solutions generated by evolutionary operations and solutions in dominant population are set as initial individuals for local search procedure. Then, a new direction vector named as pseudonormal vector is proposed to integrate two objective functions together to form a fitness function. Finally, a network specific local search strategy based on label propagation rule is expanded to search the local optimal solutions efficiently. The extensive experiments on both artificial and real-world networks evaluate the proposed method from three aspects. Firstly, experiments on influence of local search procedure demonstrate that the local search procedure can speed up the convergence to better partitions and make the algorithm more stable. Secondly, comparisons with a set of classic community detection methods illustrate the proposed method can find single partitions effectively. Finally, the method is applied to identify hierarchical structures of networks which are beneficial for analyzing networks in multi-resolution levels.     

Natural selection and veridical perceptions

Does natural selection favor veridical perceptions, those that more accurately depict the objective environment? Students of perception often claim that it does. But this claim, though influential, has not been adequately tested. Here we formalize the claim and a few alternatives. To test them, we introduce “interface games,” a class of evolutionary games in which perceptual strategies compete. We explore, in closed-form solutions and Monte Carlo simulations, some simpler games that assume frequency-dependent selection and complete mixing in infinite populations. We find that veridical perceptions can be driven to extinction by non-veridical strategies that are tuned to utility rather than objective reality. This suggests that natural selection need not favor veridical perceptions, and that the effects of selection on sensory perception deserve further study.     

Genomic clocks and evolutionary timescales

For decades, molecular clocks have helped to illuminate the evolutionary timescale of life, but now genomic data pose a challenge for time estimation methods. It is unclear how to integrate data from many genes, each potentially evolving under a different model of substitution and at a different rate. Current methods can be grouped by the way the data are handled (genes considered separately or combined into a 'supergene') and the way gene-specific rate models are applied (global versus local clock). There are advantages and disadvantages to each of these approaches, and the optimal method has not yet emerged. Fortunately, time estimates inferred using many genes or proteins have greater precision and appear to be robust to different approaches.     

Quantum computing: a new paradigm for ecology

A global technology arms race is underway to build evermore powerful and precise quantum computers. Quantum computers have the potential to tackle certain quantitative problems quicker than classical computers. The current focus of quantum computing is on pushing the boundaries of fundamental quantum information and commercial applications in industrial sectors, financial services, and other profit-led sectors, particularly where improvements in optimisation and sampling can improve increased economic return. We believe that ecologists could exploit the computational power of quantum computers because the statistical approaches commonly used in ecology already have proven pathways on quantum computers. Moreover, quantum computing could ultimately leapfrog our understanding of complex ecological systems, if the hardware, opportunity, and creativity of quantitative ecologists all align.     

Towards an ecology of protective coloration

The strategies underlying different forms of protective coloration are well understood but little attention has been paid to the ecological, life-history and behavioural circumstances under which they evolve. While some comparative studies have investigated the ecological correlates of aposematism, and background matching, the latter particularly in mammals, few have examined the ecological correlates of other types of protective coloration. Here, we first outline which types of defensive coloration strategies may be exhibited by the same individual; concluding that many protective coloration mechanisms can be employed simultaneously, particularly in conjunction with background matching. Second, we review the ecological predictions that have been made for each sort of protective coloration mechanism before systematically surveying phylogenetically controlled comparative studies linking ecological and social variables to antipredator defences that involve coloration. We find that some a priori predictions based on small-scale empirical studies and logical arguments are indeed supported by comparative data, especially in relation to how illumination affects both background matching and self-shadow concealment through countershading; how body size is associated with countershading, motion dazzle, flash coloration and aposematism, although only in selected taxa; how immobility may promote background matching in ambush predators; and how mobility may facilitate motion dazzle. Examination of nearly 120 comparative tests reveals that many focus on ecological variables that have little to do with predictions derived from antipredator defence theory, and that broad-scale ecological studies of defence strategies that incorporate phylogenetics are still very much in their infancy. We close by making recommendations for future evolutionary ecological research.     

Pattern and process: evidence for the evolution of photosynthetic traits in natural populations

The patterns of interspecific variation identified by comparative studies provide valuable hypotheses about the role of physiological traits in evolutionary adaptation. This review covers tests of these hypotheses for photosynthetic traits that have used a microevolutionary perspective to characterize physiological variation among and within populations. Studies of physiological differentiation among populations show that evolutionary divergence in photosynthetic traits is common within species, and has a pattern that supports many adaptive hypotheses. These among-population studies imply that selection has influenced photosynthetic traits in some way, but they are not designed to identify the traits targeted by selection or the environmental agents that cause selection. Analyses of genetic and phenotypic variation within populations address these questions. Studies that have quantified genetic variation within populations show that levels of heritable variation can be adequate for evolutionary change in photosynthetic traits. Other studies have measured phenotypic selection for these traits by analyzing how the variation within populations is correlated with fitness. This work has shown that selection for photosynthetic traits may often operate indirectly via correlations with other traits, and emphasizes the importance of viewing the phenotype as an integrated function of growth, morphology, life-history and physiology. We also outline some methodological problems that may be encountered for ecophysiological traits by these types of studies, provide some potential solutions, and discuss future directions for the field of plant evolutionary ecophysiology.     

Optimization over a class of tree shape statistics

Tree shape statistics quantify some aspect of the shape of a phylogenetic tree. They are commonly used to compare reconstructed trees to evolutionary models and to find evidence of tree reconstruction bias. Historically, to find a useful tree shape statistic, formulas have been invented by hand and then evaluated for utility. This paper presents the first method which is capable of optimizing over a class of tree shape statistics, called binary recursive tree shape statistics (BRTSS). After defining the BRTSS class, a set of algebraic expressions is defined which can be used in the recursions. The set of tree shape statistics definable using these expressions in the BRTSS is very general and includes many of the statistics with which phylogenetic researchers are already familiar. We then present a practical genetic algorithm which is capable of performing optimization over BRTSS given any objective function. The chapter concludes with a successful application of the methods to find a new statistic which indicates a significant difference between two distributions on trees which were previously postulated to have similar properties.     

The naturalist in a world of genomics

Functional genomics provides new opportunities to address issues of fundamental interest in evolutionary biology and suggests many new research directions that are ripe for evolutionary investigation. New types of data, and the ability to study biological processes from a whole genome perspective, are likely to have a profound impact on evolutionary biology and ecology. To illustrate, we discuss how genomewide gene expression studies can be used to reformulate questions about trade-offs and pleiotropy. We then touch on some of the new research opportunities that the application of functional genomics affords to evolutionary biologists. We end with some brief notes about how evolutionary biology and comparative approaches will probably have an impact on functional genomics.     

Advantages of Task-Specific Multi-Objective Optimisation in Evolutionary Robotics

The application of multi-objective optimisation to evolutionary robotics is receiving increasing attention. A survey of the literature reveals the different possibilities it offers to improve the automatic design of efficient and adaptive robotic systems, and points to the successful demonstrations available for both task-specific and task-agnostic approaches (i.e., with or without reference to the specific design problem to be tackled). However, the advantages of multi-objective approaches over single-objective ones have not been clearly spelled out and experimentally demonstrated. This paper fills this gap for task-specific approaches: starting from well-known results in multi-objective optimisation, we discuss how to tackle commonly recognised problems in evolutionary robotics. In particular, we show that multi-objective optimisation (i) allows evolving a more varied set of behaviours by exploring multiple trade-offs of the objectives to optimise, (ii) supports the evolution of the desired behaviour through the introduction of objectives as proxies, (iii) avoids the premature convergence to local optima possibly introduced by multi-component fitness functions, and (iv) solves the bootstrap problem exploiting ancillary objectives to guide evolution in the early phases. We present an experimental demonstration of these benefits in three different case studies: maze navigation in a single robot domain, flocking in a swarm robotics context, and a strictly collaborative task in collective robotics.     

Methods and strategies available for the process control and optimization of monoclonal antibody production

The objective of this paper is to explore the range of methods and strategies available for the process control and optimization of monoclonal antibody production by hybridoma cell culture. Emphasis will be placed on the choice of the level of complexity incorporated into the process control and optimisation procedure. It will be shown that the behaviour of hybridomas in culture is influenced by sophisticated cellular metabolic activities and various interactive environmental factors and that the understanding and modelling of the way hybridomas grow in the bioreactor should enable optimisation of bioreactor operating conditions to achieve maximum monoclonal antibody formation. However, due to the lack of on-line instrumentation of important biological variables and the incomplete knowledge of hybridoma cultivation process, there exist many limitations and challenges to the advent of applications of process control and optimisation in this field. To solve the problem, introduction of industrially practical biological measurements and development of new control concepts are inevitable. At the end of this paper, we shall discuss possible schemes for the control of the physsiological state of cells in order that balanced cell growth and maximum monoclonal antibody synthesis may be achieved.     

Integrating Transgenic Vector Manipulation with Clinical Interventions to Manage Vector-Borne Diseases

Many vector-borne diseases lack effective vaccines and medications, and the limitations of traditional vector control have inspired novel approaches based on using genetic engineering to manipulate vector populations and thereby reduce transmission. Yet both the short- and long-term epidemiological effects of these transgenic strategies are highly uncertain. If neither vaccines, medications, nor transgenic strategies can by themselves suffice for managing vector-borne diseases, integrating these approaches becomes key. Here we develop a framework to evaluate how clinical interventions (i.e., vaccination and medication) can be integrated with transgenic vector manipulation strategies to prevent disease invasion and reduce disease incidence. We show that the ability of clinical interventions to accelerate disease suppression can depend on the nature of the transgenic manipulation deployed (e.g., whether vector population reduction or replacement is attempted). We find that making a specific, individual strategy highly effective may not be necessary for attaining public-health objectives, provided suitable combinations can be adopted. However, we show how combining only partially effective antimicrobial drugs or vaccination with transgenic vector manipulations that merely temporarily lower vector competence can amplify disease resurgence following transient suppression. Thus, transgenic vector manipulation that cannot be sustained can have adverse consequences—consequences which ineffective clinical interventions can at best only mitigate, and at worst temporarily exacerbate. This result, which arises from differences between the time scale on which the interventions affect disease dynamics and the time scale of host population dynamics, highlights the importance of accounting for the potential delay in the effects of deploying public health strategies on long-term disease incidence. We find that for systems at the disease-endemic equilibrium, even modest perturbations induced by weak interventions can exhibit strong, albeit transient, epidemiological effects. This, together with our finding that under some conditions combining strategies could have transient adverse epidemiological effects suggests that a relatively long time horizon may be necessary to discern the efficacy of alternative intervention strategies.