Implicit frequency dependence and kin selection in fluctuating environments

Summary I consider a general model of a fluctuating environment in which the environmental state each year is drawn at random from some given distribution. Each year organisms must choose what action to perform before the environmental state for that year is known. There is no interaction with kin. In this scenario, natural selection will tend to produce organisms which maximize their geometric mean fitness. In this paper I introduce the idea of the profile of a strategy. This function quantifies how the strategy peforms for each environmental state. I show that there is a unique profile such that a strategy is optimal if and only if it has this profile. I then give a characterization of the optimal profile which generalizes previous work by others in this area. The characterization of the optimal profile has a game theoretical interpretation. Motivated by this I introduce a game in which individuals play the field in a constant environment. This game may be interpreted as a cooperative game between kin. The key result of this paper shows that a strategy maximizes geometric mean fitness in the original fluctuating environment problem if and only if it is an evolutionarily stable strategy of the deterministic environment game. It is well known that an optimal strategy in a fluctuating environment may be mixed, involving adaptive coin-flipping. Others have previously noted that this may result in some individuals sacrificing individual reproductive success for the good of the genotype. My analysis shows that one may regain the concept of individual optimization if the quantity maximized is suitably defined. Under an optimal strategy every action taken maximizes the expected number of offspring produced, where this expectation is not calculated using the true distribution of environmental states, but a distribution modified to take account of the actions of kin.     

The numerical method for the optimal supporting position and related optimal control for the catalytic reaction system

This paper considers the numerical approximation for the optimal supporting position and related optimal control of a catalytic reaction system with some control and state constraints, which is governed by a nonlinear partial differential equations with given initial and boundary conditions. By the Galerkin finite element method, the original problem is projected into a semi-discrete optimal control problem governed by a system of ordinary differential equations. Then the control parameterization method is applied to approximate the control and reduce the original system to an optimal parameter selection problem, in which both the position and related control are taken as decision variables to be optimized. This problem can be solved as a nonlinear optimization problem by a particle swarm optimization algorithm. The numerical simulations are given to illustrate the effectiveness of the proposed numerical approximation method.     

Sequential Decision Rules for Managing Nematodes with Crop Rotations

A dynamic model of nematode populations under a crop rotation that includes both host and nonhost crops is developed and used to conceptualize the problem of economic control. The steady state of the dynamic system is used to devise an approximately optimal decision policy, which is then applied to cyst nematode (Heterodera schachtii) control in a rotation of sugarbeet with nonhost crops. Long-run economic returns from this approximately optimal decision rule are compared with results from solution of the exact dynamic optimization model. The simple decision rule based on the steady state provides long-run average returns that are similar to the fully optimal solution. For sugarbeet and H. schachtii, the simplified rule can be calculated by maximizing a relatively simple algebraic expression with respect to the number of years in the sequence of nonhost crops. Maximization is easy because only integers are of interest and the number of years in nonhost crops is typically small. Solution of this problem indirectly yields an approximation to the optimal dynamic economic threshold density of nematodes in the soil. The decision rule requires knowledge of annual nematode population change under host and nonhost crops, and the relationship between crop yield and nematode population density.     

Dynamic load control policies for a flexible manufacturing system with stochastic processing rates

Real-time scheduling and load controls of FMSs are complex processes in which the control logic must consider a broad spectrum of instantaneous state variables while taking into account the probabilistic future impact of each decision at each time epoch. These processes are particularly important in the management of modern FMS environment, since they are known to have a significant impact on the FMS productive capacity and economic viability. In this article we outline the approach developed for dynamic load controls within an FMS producing a variety of glass lenses. Two revenue-influencing objective functions are evaluated for this capital-intensive facility. It is shown that by using Semi-Markovian modeling concepts, the FMS states need to be observed only at certain decision epochs. The mean holding time in each state is then obtained using the probability distribution function of the conditional state occupancy times. Several key performance measures are then derived by means of the value equations. In addition, the structure of the optimal policies are exemplified for a variety of operational parameters. It is shown that the optimal policies tend to generate higher buffer stocks of parts in those work centers having the highest revenue-generation rates. These buffer stocks get smaller and smaller as the relative processing capacity of the centers increases. Similar observations lead us to the introduction of several promising heuristics that capture the structural properties of the optimal policies with a significantly smaller computational effort. Results of the empirical evaluation of these heuristics are also analyzed here.     

Mathematical modeling of differentiation in dictyostelium discoideum

Methods for the dynamic analysis of biochemical differentiation are presented. These are demonstrated in the analysis of biochemical differentiation of the carbohydrate system in D. discoideum. Procedures for simplification which are presented are projection and contraction of the system trajectory in state space and the generation of reduced equivalent dynamic metabolic networks. The importance of the hierarchical structure of differentiating systems is discussed and the concept of a dynamic embedding diagram is introduced. It is shown that complex systems must be analyzed on an epoch by epoch basis, each epoch being a period of time characterized by a constant dynamic embedding diagram, and that widely different time scales and state space scales may be necessary in different epochs. In particular there is no a priori lower limit to the time scale which may be necessary during the analysis. Some problems in mathematically defining differentiation are discussed.     

Minimizing the cost of environmental management decisions by optimizing statistical thresholds

Environmental management decisions are prone to expensive mistakes if they are triggered by hypothesis tests using the conventional Type I error rate (α) of 0.05. We derive optimal α‐levels for decision‐making by minimizing a cost function that specifies the overall cost of monitoring and management. When managing an economically valuable koala population, it shows that a decision based on α = 0.05 carries an expected cost over $5 million greater than the optimal decision. For a species of such value, there is never any benefit in guarding against the spurious detection of declines and therefore management should proceed directly to recovery action. This result holds in most circumstances where the species’ value substantially exceeds its recovery costs. For species of lower economic value, we show that the conventional α‐level of 0.05 rarely approximates the optimal decision‐making threshold. This analysis supports calls for reversing the statistical ‘burden of proof’ in environmental decision‐making when the cost of Type II errors is relatively high.     

Estimation of the Optimal Statistical Quality Control Sampling Time Intervals Using a Residual Risk Measure

Background

An open problem in clinical chemistry is the estimation of the optimal sampling time intervals for the application of statistical quality control (QC) procedures that are based on the measurement of control materials. This is a probabilistic risk assessment problem that requires reliability analysis of the analytical system, and the estimation of the risk caused by the measurement error.

Methodology/Principal Findings

Assuming that the states of the analytical system are the reliability state, the maintenance state, the critical-failure modes and their combinations, we can define risk functions based on the mean time of the states, their measurement error and the medically acceptable measurement error. Consequently, a residual risk measure rr can be defined for each sampling time interval. The rr depends on the state probability vectors of the analytical system, the state transition probability matrices before and after each application of the QC procedure and the state mean time matrices. As optimal sampling time intervals can be defined those minimizing a QC related cost measure while the rr is acceptable. I developed an algorithm that estimates the rr for any QC sampling time interval of a QC procedure applied to analytical systems with an arbitrary number of critical-failure modes, assuming any failure time and measurement error probability density function for each mode. Furthermore, given the acceptable rr, it can estimate the optimal QC sampling time intervals.

Conclusions/Significance

It is possible to rationally estimate the optimal QC sampling time intervals of an analytical system to sustain an acceptable residual risk with the minimum QC related cost. For the optimization the reliability analysis of the analytical system and the risk analysis of the measurement error are needed.     

Life histories as adaptive strategies

Optimal control theory is used to produce a general model of life history evolution in a stationary environment. Several disparate trends in current theorizing on life histories are thereby unified. An optimal life history (OLH) is defined as one which maximizes individual fitness (the Malthusian parameter in density-independent populations, the carrying capacity in density-dependent ones). Since the components of fitness depend on the phenotype, the search for an OLH is accomplished in phenotypic space. The optimization is controlled by apportioning the energy obtained at any age between conflicting processes of growth, survival and reproduction. The methods of dynamic optimization which pertain to this problem are reviewed briefly, and its results interpreted biologically. Of these, Pontryagin's method is selected and used to examine some simple models. This method leads one to define a dual variable matched to each phenotypic variable, the prospective value. This provides an indicator of the selective pressures acting at any age on a phenotypic feature to push it towards coincidence with the OLH. This also suggests that at ages in which these dual variables are low (i.e. late ages) there will be greater phenotypic variability around the OLH in any population. The problem of the optimal distribution of reproductive effort over the life history is discussed as well.     

Active inference and agency: optimal control without cost functions

This paper describes a variational free-energy formulation of (partially observable) Markov decision problems in decision making under uncertainty. We show that optimal control can be cast as active inference. In active inference, both action and posterior beliefs about hidden states minimise a free energy bound on the negative log-likelihood of observed states, under a generative model. In this setting, reward or cost functions are absorbed into prior beliefs about state transitions and terminal states. Effectively, this converts optimal control into a pure inference problem, enabling the application of standard Bayesian filtering techniques. We then consider optimal trajectories that rest on posterior beliefs about hidden states in the future. Crucially, this entails modelling control as a hidden state that endows the generative model with a representation of agency. This leads to a distinction between models with and without inference on hidden control states; namely, agency-free and agency-based models, respectively.     

Change and aging senescence as an adaptation

Understanding why we age is a long-lived open problem in evolutionary biology. Aging is prejudicial to the individual, and evolutionary forces should prevent it, but many species show signs of senescence as individuals age. Here, I will propose a model for aging based on assumptions that are compatible with evolutionary theory: i) competition is between individuals; ii) there is some degree of locality, so quite often competition will be between parents and their progeny; iii) optimal conditions are not stationary, and mutation helps each species to keep competitive. When conditions change, a senescent species can drive immortal competitors to extinction. This counter-intuitive result arises from the pruning caused by the death of elder individuals. When there is change and mutation, each generation is slightly better adapted to the new conditions, but some older individuals survive by chance. Senescence can eliminate those from the genetic pool. Even though individual selection forces can sometimes win over group selection ones, it is not exactly the individual that is selected but its lineage. While senescence damages the individuals and has an evolutionary cost, it has a benefit of its own. It allows each lineage to adapt faster to changing conditions. We age because the world changes.     

Muscular synergism--I. On criteria for load sharing between synergistic muscles

The use of optimization techniques to predict individual muscle forces in redundant biomechanical systems implies the formulation of a criterion for load sharing between the muscles. In part I of this paper, the characteristics and performance of several linear and non-linear criteria reported in the literature have been compared for static-isometric knee flexion. The results show that linear criteria inherently predict discrete muscle action (orderly recruitment of muscles) whereas non-linear criteria can predict synergistic action. All criteria predict that relatively more force is allocated to muscles with large moment arms. When muscle stresses (or ratios of muscle force to maximum muscle force) are used as the decision variables in the objective function, then relatively more force is allocated to muscles with large maximum possible force as well. Future formulations of the optimization should consider the differences in fiber type composition among the muscles. Such an approach is presented in part II of the paper.     

Parameter Estimation of Fractional-Order Chaotic Systems by Using Quantum Parallel Particle Swarm Optimization Algorithm

Parameter estimation for fractional-order chaotic systems is an important issue in fractional-order chaotic control and synchronization and could be essentially formulated as a multidimensional optimization problem. A novel algorithm called quantum parallel particle swarm optimization (QPPSO) is proposed to solve the parameter estimation for fractional-order chaotic systems. The parallel characteristic of quantum computing is used in QPPSO. This characteristic increases the calculation of each generation exponentially. The behavior of particles in quantum space is restrained by the quantum evolution equation, which consists of the current rotation angle, individual optimal quantum rotation angle, and global optimal quantum rotation angle. Numerical simulation based on several typical fractional-order systems and comparisons with some typical existing algorithms show the effectiveness and efficiency of the proposed algorithm.     

Human and Machine Learning in Non-Markovian Decision Making

Humans can learn under a wide variety of feedback conditions. Reinforcement learning (RL), where a series of rewarded decisions must be made, is a particularly important type of learning. Computational and behavioral studies of RL have focused mainly on Markovian decision processes, where the next state depends on only the current state and action. Little is known about non-Markovian decision making, where the next state depends on more than the current state and action. Learning is non-Markovian, for example, when there is no unique mapping between actions and feedback. We have produced a model based on spiking neurons that can handle these non-Markovian conditions by performing policy gradient descent [1]. Here, we examine the model’s performance and compare it with human learning and a Bayes optimal reference, which provides an upper-bound on performance. We find that in all cases, our population of spiking neurons model well-describes human performance.     

Optimizing experimental design for comparing models of brain function

This article presents the first attempt to formalize the optimization of experimental design with the aim of comparing models of brain function based on neuroimaging data. We demonstrate our approach in the context of Dynamic Causal Modelling (DCM), which relates experimental manipulations to observed network dynamics (via hidden neuronal states) and provides an inference framework for selecting among candidate models. Here, we show how to optimize the sensitivity of model selection by choosing among experimental designs according to their respective model selection accuracy. Using Bayesian decision theory, we (i) derive the Laplace-Chernoff risk for model selection, (ii) disclose its relationship with classical design optimality criteria and (iii) assess its sensitivity to basic modelling assumptions. We then evaluate the approach when identifying brain networks using DCM. Monte-Carlo simulations and empirical analyses of fMRI data from a simple bimanual motor task in humans serve to demonstrate the relationship between network identification and the optimal experimental design. For example, we show that deciding whether there is a feedback connection requires shorter epoch durations, relative to asking whether there is experimentally induced change in a connection that is known to be present. Finally, we discuss limitations and potential extensions of this work.     

Application of robust optimization in matrix-based LCI for decision making under uncertainty

Purpose

When product systems are optimized to minimize environmental impacts, uncertainty in the process data may impact optimal decisions. The purpose of this article is to propose a mathematical method for life cycle assessment (LCA) optimization that protects decisions against uncertainty at the life cycle inventory (LCI) stage.

Methods

A robust optimization approach is proposed for decision making under uncertainty in the LCI stage. The proposed approach incorporates data uncertainty into an optimization problem in which the matrix-based LCI model appears as a constraint. The level of protection against data uncertainty in the technology and intervention matrices can be controlled to reflect varying degrees of conservatism.

Results and discussion

A simple numerical example on an electricity generation product system is used to illustrate the main features of this methodology. A comparison is made between a robust optimization approach, and decision making using a Monte Carlo analysis. Challenges to implement the robust optimization approach on common uncertainty distributions found in LCA and on large product systems are discussed. Supporting source code is available for download at https://github.com/renwang/Robust_Optimization_LCI_Uncertainty.

Conclusions

A robust optimization approach for matrix-based LCI is proposed. The approach incorporates data uncertainties into an optimization framework for LCI and provides a mechanism to control the level of protection against uncertainty. The tool computes optimal decisions that protects against worst-case realizations of data uncertainty. The robust optimal solution is conservative and is able to avoid the negative consequences of uncertainty in decision making.     

Impact of ramp-up on the optimal capacity-related reconfiguration policy

This paper presents an optimal solution, based on Markov decision theory, for the problem of optimal capacity-related reconfiguration of manufacturing systems, under stochastic market demand. Both capacity expansion and reduction are considered. The solution quantitatively takes into account the effect of the ramp-up phenomenon, following each reconfiguration, on the optimal policy. A closed-form solution is presented for when product demand is independently and generally distributed over time. A real case concerning a flexible manufacturing line in the automotive sector is shown, to prove that ignoring the ramp-up effect in the decision process can lead to significant increases in overall costs.     

Decision making by rats in an unpredictable laboratory environment: final decision rules

A simple laboratory situation was developed in order to test statistical decision making in the white rat. The subject's problem is to obtain rewards (water) which can be achieved by performing one of two possible alternative activities. The profitability of each of these two activities depends on the state of the test apparatus. The subject does not have perfect information about this state but information becomes available as a consequence of doing activity 1 and/or 2. The problem is expressed mathematically and the optimal behaviour calculated. The predictions from the theoretical model are compared with the actual performance of the rat. The conclusion is that the optimal solution predicts the average behaviour of a rat during a test session well. The variation in the responses however cannot be understood from the model. The set up is suitable for pharmacological and physiological behavioural research.     

Optimal feeding strategies by adaptive mesh selection for fed-batch bioprocesses

The objective of this contribution is the design of optimal feeding strategies for fed-batch bioprocesses, where complex dynamic models with input and state constraints are present. For the solution of this dynamic optimization problem a transformation to a finite dimensional optimization problem is made using piecewise linear control profiles. The optimization of these profiles is performed by a sequential approach, that includes an ODE solver for the solution of the model ODE's. Further an adaptive mesh selection algorithm was investigated for an appropriate discretization of the control profiles. The implementation of the resulting optimal feeding profiles is shown for a process example, namely the production of nikkomycin by Streptomyces tendae. This implementation uses a hierarchical process control framework, that consists of components for process monitoring, state estimation, and trajectory control.     

On the optimal policies of cancer screening.

Some problems of optimal screening are considered. A screening strategy is allowed to be nonperiodic. Two approaches to screening optimization are used: the minimum delay time approach and the minimum cost approach. Both approaches are applied to the analysis of an optimization problem when the natural history of the disease is known and when it is unknown (a minimax problem). The structure of optimal screening policies is investigated as well as the benefit they can provide compared to the periodic screening policy. The detection probability is assumed to depend only on the stage of the disease, though it may not be constant throughout each stage. It is shown that periodic screening appears to be optimal when one has no information on the natural history of the disease, the minimum delay time criterion being used for optimization. Some applications to lung cancer screening are presented.     

Productivity improvement in xanthan gum fermentation using multiple substrate optimization

A novel and more comprehensive formulation of the optimal control problem that reflects the operational requirements of a typical industrial fermentation has been proposed in this work. This formulation has been applied to a fed-batch bioreactor with three control variables, i.e., feed rates of carbon source, nitrogen source, and an oxygen source, to result in a 148.7% increase in product formation. Xanthan gum production using Xanthomonas campestris has been used as the model system for this optimization study, and the liquid-phase oxygen supply strategy has been used to supply oxygen to the fermentation. The formulated optimization problem has several constraints associated with it due to the nature of the system. A robust stochastic technique, differential evolution, has been used to solve this challenging optimization problem. The infinite dimensional optimization problem has been approximated to a finite dimensional one by control vector parametrization. The state constraints that are path constraints have been addressed by using penalty functions and by integrating them over the total duration to ensure a feasible solution. End point constraints on final working volume of the reactor and on the final residual concentrations of carbon and nitrogen sources have been included in the problem formulation. Further, the toxicity of the oxygen source, H(2)O(2), has been addressed by imposing a constraint on its maximum usable concentration. In addition, the initial volume of the bioreactor contents and feed concentrations have been handled as decision variables, which has enabled a well-grounded choice for their values from the optimization procedure; adhoc values are normally used in the industry. All results obtained by simulation have been validated experimentally with good agreements between experimental and simulated values.