Spike-based Decision Learning of Nash Equilibria in Two-Player Games

Humans and animals face decision tasks in an uncertain multi-agent environment where an agent''s strategy may change in time due to the co-adaptation of others strategies. The neuronal substrate and the computational algorithms underlying such adaptive decision making, however, is largely unknown. We propose a population coding model of spiking neurons with a policy gradient procedure that successfully acquires optimal strategies for classical game-theoretical tasks. The suggested population reinforcement learning reproduces data from human behavioral experiments for the blackjack and the inspector game. It performs optimally according to a pure (deterministic) and mixed (stochastic) Nash equilibrium, respectively. In contrast, temporal-difference(TD)-learning, covariance-learning, and basic reinforcement learning fail to perform optimally for the stochastic strategy. Spike-based population reinforcement learning, shown to follow the stochastic reward gradient, is therefore a viable candidate to explain automated decision learning of a Nash equilibrium in two-player games.     

Microtubule-Nucleus Interactions in Dictyostelium discoideum Mediated by Central Motor Kinesins

Kinesins are a diverse superfamily of motor proteins that drive organelles and other microtubule-based movements in eukaryotic cells. These motors play important roles in multiple events during both interphase and cell division. Dictyostelium discoideum contains 13 kinesin motors, 12 of which are grouped into nine families, plus one orphan. Functions for 11 of the 13 motors have been previously investigated; we address here the activities of the two remaining kinesins, both isoforms with central motor domains. Kif6 (of the kinesin-13 family) appears to be essential for cell viability. The partial knockdown of Kif6 with RNA interference generates mitotic defects (lagging chromosomes and aberrant spindle assemblies) that are consistent with kinesin-13 disruptions in other organisms. However, the orphan motor Kif9 participates in a completely novel kinesin activity, one that maintains a connection between the microtubule-organizing center (MTOC) and nucleus during interphase. kif9 null cell growth is impaired, and the MTOC appears to disconnect from its normally tight nuclear linkage. Mitotic spindles elongate in a normal fashion in kif9⁻ cells, but we hypothesize that this kinesin is important for positioning the MTOC into the nuclear envelope during prophase. This function would be significant for the early steps of cell division and also may play a role in regulating centrosome replication.Directed cell migration, organelle transport, and cell division involve fundamental motilities that are necessary for eukaryotic cell viability and function. Much of the force required for these motilities is generated through the cyclical interactions of motor proteins with the cell cytoskeleton. Microtubules (MTs) and actin filaments provide structural support and directional guides, and all eukaryotic organisms have diverse, often extensive families of motors that carry out different tasks. Functional studies have revealed that many of the motors work in combination with others, and that the individual deletion of a single motor activity often is insufficient to produce a defect that substantially impairs cell growth or function. The latter phenomenon is particularly evident in some organisms with simple motor families (14, 42). By contrasting homologous motor functions between simple and complex systems, we hope to learn the details of how each motor is custom-tuned for specific tasks.Dictyostelium discoideum is a compact amoeba that exhibits robust forms of motility common to nearly all animal cells, with speeds that frequently exceed corresponding rates in vertebrate cell models (25, 33, 54). Since Dictyostelium possesses a relatively small number of motor proteins (13 kinesin, 1 dynein, and 13 myosin isoforms [23, 24, 26]), it combines advantages of terrific cytology with straightforward molecular genetics and thus represents an excellent model to investigate individual and combined motor protein actions. To date, 11 of the 13 kinesin motors have been analyzed functionally (5, 17, 18, 30, 42, 46, 51, 60). Only 1 of these 11 motors, Kif3, a member of the kinesin-1 family of organelle transporters, appears to be essential for organism viability (51). Individual disruptions of three kinesin genes (kif1, kif4, and kif12) produce distinctive defects in cell growth or organelle transport (30, 42, 46). Analyses of six of the seven other kinesins reveal important phenotypes but only when combined with other motor disruptions or cell stresses. We address here the roles of the remaining two Dictyostelium MT-based motors.kif6 and kif9 encode two central motor kinesins in the Dictyostelium genome (24). The best-studied isoforms of this motor type are represented by the kinesin-13 family, and they largely function to regulate MT length during cell division (13, 16, 40, 41). In some organisms, kinesin-13 motors also have been shown to operate during interphase and to mediate MT and flagellar length control (3, 4, 15) and perhaps even organelle transport (32, 43, 56). kif6 encodes the kinesin-13 family member in Dictyostelium. We demonstrate that Kif6 activity is essential for viability, and that it plays a primary, conserved role in chromosome segregation during cell division.The second of the central motor kinesins, Kif9, does not group with an existing family (24, 38). The gene disruption of this motor reveals a completely novel function for a kinesin in maintaining a connection between the MT-organizing center (MTOC) and the nucleus. By electron microscopy (EM), the MTOC of Dictyostelium appears as a cytoplasmic cube-shaped structure surrounded by amorphous dense material (39, 44). EM, biochemical analyses, antibody labeling, and live-cell imaging studies have demonstrated that during interphase, the cytoplasmic MTOC is firmly and closely attached to the nucleus (28, 29, 44, 48, 49, 63). Upon entry into mitosis, the MTOC duplicates during prophase and is brought to or into a fenestration of the nuclear envelope, and then it establishes an intranuclear bipolar spindle for division (39, 53, 64). While MTOCs can be purified from Dictyostelium, the methods rely heavily on reagents that actively disrupt the attached nuclei (10, 59). A recent study has identified at least one component of this connection, the nuclear envelope protein Sun-1 (67). The perturbation of Sun-1 affects nuclear shape and results in centrosome detachment, hyperamplification, and aneuploidy. We demonstrate in the current work that the disruption of the Kif9 kinesin also perturbs the MTOC-nucleus linkage. Our results suggest that an MT-mediated mechanism plays a significant role in maintaining an MTOC-nucleus connection during interphase, and we discuss how this connection could be important to regulate centrosome replication and ensure proper chromosome segregation during cell division.     

Food Web Effects of Prey Size Refugia: Variable Interactions and Alternative Stable Equilibria

Predators can have highly variable effects on the abundance and composition of food webs, ranging from strong to weak effects of top predators. Typical food web models assume that individual prey are identical in their susceptibility to predators throughout their lives, but many prey species become less vulnerable to predators through ontogeny. A simple set of models is explored where prey must pass through a vulnerable stage prior to achieving a predator-invulnerable size refuge. As productivity of the environment increases, the proportional impact of predators decreases because more individuals become and remain in the invulnerable adult stage. The addition of a competitor prey species that can not achieve size refuge results in contrasting outcomes. At low productivity, the small species wins in competition, and the system is strongly consumer controlled. At high productivity, the large species wins due to the presence of predators, and the system becomes less consumer controlled. At intermediate productivity, either the small or the large species can win depending on initial conditions, and the system can be either strongly or weakly consumer controlled. Such alternative stable equilibria derived from models with prey size refugia may help to explain many natural situations.     

Attractive Protein-Polymer Interactions Markedly Alter the Effect of Macromolecular Crowding on Protein Association Equilibria

The dependence of the fluorescence of catalase upon the concentration of added superoxide dismutase (SOD) indicates that SOD binds to saturable sites on catalase. The affinity of SOD for these sites varies with temperature, and with the concentration of each of three nominally inert polymeric additives—dextran 70, Ficoll 70, and polyethylene glycol 2000. At room temperature (25.0°C) and higher, the addition of high concentrations of polymer is found to significantly enhance the affinity of SOD for catalase, but with decreasing temperature the enhancing effect of polymer addition diminishes, and at 8.0°C, addition of polymer has little or no effect on the affinity of SOD for catalase. The results presented here provide the first experimental evidence for the existence of competition between a repulsive excluded volume interaction between protein and polymer, which tends to enhance association of dilute protein, and an attractive interaction between protein and polymer, which tends to inhibit protein association. The net effect of high concentrations of polymer upon protein associations depends upon the relative strength of these two types of interactions at the temperature of measurement, and may vary significantly between different proteins and/or polymers.     

Root-Shoot Interactions: Functional Equilibria for Nutrient Uptake in Peach (Prunuspersica L. Batsch.)

Peach seedlings were grown in aerated nutrient solution in small(15 cm³) or large (500 cm³) containers. Subsequently, some plantswere transferred from the small containers to the large, orwere stressed by root pruning. Half the plants received a foliarspray of 6-benzylaminopurine (BAP) every 4 days. Analysis of all the results enabled correlations between growthand nutrient uptake to be made. These relationships supportedthe hypothesis that for nutrient uptake a functional equilibriumexisted between the root and the shoot systems. Data for individual nutrients showed that treatment with BAPaltered the functional equilibrium for some nutrients. In particularit was suggested that indirectly BAP depressed calcium uptakewhereas potassium uptake was increased. Prunus persica, peach, growth, nutrient uptake, root-shoot interactions     

Equilibria in structured populations

The existence of a stable positive equilibrium state for the density of a population which is internally structured by means of a single scalar such as age, size, etc. is studied as a bifurcation problem. Using an inherent birth modulus n as a bifurcation parameter it is shown for very general nonlinear model equations, in which vital birth and growth processes depend on population density, that a global unbounded continuum of nontrivial equilibrium pairs (n, ) bifurcates from the unique (normalized) critical point (1, 0). The pairs are locally positive and conditions are given under which the continuum is globally positive. Local stability is shown to depend on the direction of bifurcation. For the important case when density dependence is a nonlinear expression involving a linear functional of density (such as total population size) it is shown how a detailed global bifurcation diagram is easily constructed in applications from the graph of a certain real valued function obtained from an invariant on the continuum. Uniqueness and nonuniqueness of positive equilibrium states are studied. The results are illustrated by several applications to models appearing in the literature.This research was done while the author was on leave at the Lehrstuhl für Biomathematik, Universität Tübingen, Auf der Morgenstelle 10, 7400 Tübingen 1, Federal Republic of Germany     

Molecular Interactions and Residues Involved in Force Generation in the T4 Viral DNA Packaging Motor

Many viruses utilize molecular motors to package their genomes into preformed capsids. A striking feature of these motors is their ability to generate large forces to drive DNA translocation against entropic, electrostatic, and bending forces resisting DNA confinement. A model based on recently resolved structures of the bacteriophage T4 motor protein gp17 suggests that this motor generates large forces by undergoing a conformational change from an extended to a compact state. This transition is proposed to be driven by electrostatic interactions between complementarily charged residues across the interface between the N- and C-terminal domains of gp17. Here we use atomistic molecular dynamics simulations to investigate in detail the molecular interactions and residues involved in such a compaction transition of gp17. We find that although electrostatic interactions between charged residues contribute significantly to the overall free energy change of compaction, interactions mediated by the uncharged residues are equally if not more important. We identify five charged residues and six uncharged residues at the interface that play a dominant role in the compaction transition and also reveal salt bridging, van der Waals, and solvent hydrogen-bonding interactions mediated by these residues in stabilizing the compact form of gp17. The formation of a salt bridge between Glu309 and Arg494 is found to be particularly crucial, consistent with experiments showing complete abrogation in packaging upon Glu309Lys mutation. The computed contributions of several other residues are also found to correlate well with single-molecule measurements of impairments in DNA translocation activity caused by site-directed mutations.     

Reduction in Learning Rates Associated with Anterograde Interference Results from Interactions between Different Timescales in Motor Adaptation

Prior experiences can influence future actions. These experiences can not only drive adaptive changes in motor output, but they can also modulate the rate at which these adaptive changes occur. Here we studied anterograde interference in motor adaptation – the ability of a previously learned motor task (Task A) to reduce the rate of subsequently learning a different (and usually opposite) motor task (Task B). We examined the formation of the motor system''s capacity for anterograde interference in the adaptive control of human reaching-arm movements by determining the amount of interference after varying durations of exposure to Task A (13, 41, 112, 230, and 369 trials). We found that the amount of anterograde interference observed in the learning of Task B increased with the duration of Task A. However, this increase did not continue indefinitely; instead, the interference reached asymptote after 15–40 trials of Task A. Interestingly, we found that a recently proposed multi-rate model of motor adaptation, composed of two distinct but interacting adaptive processes, predicts several key features of the interference patterns we observed. Specifically, this computational model (without any free parameters) predicts the initial growth and leveling off of anterograde interference that we describe, as well as the asymptotic amount of interference that we observe experimentally (R² = 0.91). Understanding the mechanisms underlying anterograde interference in motor adaptation may enable the development of improved training and rehabilitation paradigms that mitigate unwanted interference.     

Equilibria of polyoxometalates in aqueous solution

In most aqueous polyoxometalate systems, numerous, often highly negatively charged species, are formed. To establish the speciation in such complex polyanion systems, many experimental methods and techniques must be used. Moreover, it is of vital importance that the experimental data are of the highest accuracy and that the data collected from different methods are treated simultaneously with an appropriate computer program. In systems containing one or more sensitive NMR nuclei, a combined EMF-NMR method has been shown to be extremely powerful.This article firstly gives some general comments on equilibria in aqueous polyoxometalate systems including ionic medium effects, and then describes an equilibrium EMF-NMR (³¹P and⁵¹V) study of the five component system H⁺-Mo(VI)-V(V)-P(V)-e^–. The study has been focused on so-called Keggin ratio solutions, Mo+V):P=12:1, since these are commonly used in selective oxidation processes. Special attention has been given to Mo₁₀V₂P and Mo₉V₃P solutions, where positional isomers occur. We have been able to identify and characterize all the five possible isomers of -Mo₁₀V₂PO ₄₀ ^5– at 25 and 90 °C. Besides the results from the five component system, some interesting findings from the binary, ternary and quarternary sybsystems are also reported.     

Uniqueness of Nash equilibrium in vaccination games

One crucial condition for the uniqueness of Nash equilibrium set in vaccination games is that the attack ratio monotonically decreases as the vaccine coverage level increasing. We consider several deterministic vaccination models in homogeneous mixing population and in heterogeneous mixing population. Based on the final size relations obtained from the deterministic epidemic models, we prove that the attack ratios can be expressed in terms of the vaccine coverage levels, and also prove that the attack ratios are decreasing functions of vaccine coverage levels. Some thresholds are presented, which depend on the vaccine efficacy. It is proved that for vaccination games in homogeneous mixing population, there is a unique Nash equilibrium for each game.     

Equilibria in systems of interacting structured populations

The existence of a stable positive equilibrium density for a community of k interacting structured species is studied as a bifurcation problem. Under the assumption that a subcommunity of k–1 species has a positive equilibrium and under only very mild restrictions on the density dependent vital growth rates, it is shown that a global continuum of equilibria for the full community bifurcates from the subcommunity equilibrium at a unique critical value of a certain inherent birth modulus for the kth species. Local stability is shown to depend upon the direction of bifurcation. The direction of bifurcation is studied in more detail for the case when vital per unity birth and death rates depend on population density through positive linear functionals of density and for the important case of two interacting species. Some examples involving competition, predation and epidemics are given.     

Cofactor Binding and Oxygen Equilibria in Haemoglobin

THE oxygen affinity of haemoglobin is profoundly affected by the presence of low concentrations of organic phosphates, in particular 2,3-diphosphoglycerate (DPG)^1–5. This phenomenon, which has now been widely recognized as a physiologically vital control mechanism^6–10, can be explained qualitatively by our observations² that deoxyhaemoglobin has a single strong binding site for the cofactor, which in consequence stabilizes it in relation to the oxygenated state. The binding site is associated with the β chains¹¹ and it has been identified as the cavity at the conjunction of the N-termini of these chains^5,12–18, lying therefore on the dyad axis of the tetramer.     

Properties of Equilibria in Multi-Locus Genetic Systems

The classical mathematical theory of population genetics considered, for simplicity, almost exclusively one-locus systems. In the last two decades much work has been done on two-locus and, less frequently, multi-locus systems. This research has usually involved investigating properties of systems with given, and usually rather special, fitness parameters. Real genetic fitness systems are undoubtedly multi-locus and seldom will possess simplifying characteristics. One aim of this paper is to study generalized systems where no special assumptions are made about fitness structure, the number of alleles at each locus, the number of loci involved or the recombination structure between loci. A second aim is to consider marginal properties (often one-locus properties) of complex systems: the fact that many observations involve data from only on locus makes this second aim relevant.     

Cevolutionary instability of mixed Nash solutions

Journal of Mathematical Biology -     

Regulation of Actin by Ion-Linked Equilibria

Actin assembly, filament mechanical properties, and interactions with regulatory proteins depend on the types and concentrations of salts in solution. Salts modulate actin through both nonspecific electrostatic effects and specific binding to discrete sites. Multiple cation-binding site classes spanning a broad range of affinities (nanomolar to millimolar) have been identified on actin monomers and filaments. This review focuses on discrete, low-affinity cation-binding interactions that drive polymerization, regulate filament-bending mechanics, and modulate interactions with regulatory proteins. Cation binding may be perturbed by actin post-translational modifications and linked equilibria. Partial cation occupancy under physiological and commonly used in vitro solution conditions likely contribute to filament mechanical heterogeneity and structural polymorphism. Site-specific cation-binding residues are conserved in Arp2 and Arp3, and may play a role in Arp2/3 complex activation and actin-filament branching activity. Actin-salt interactions demonstrate the relevance of ion-linked equilibria in the operation and regulation of complex biological systems.     

Regulation of Actin by Ion-Linked Equilibria

Actin assembly, filament mechanical properties, and interactions with regulatory proteins depend on the types and concentrations of salts in solution. Salts modulate actin through both nonspecific electrostatic effects and specific binding to discrete sites. Multiple cation-binding site classes spanning a broad range of affinities (nanomolar to millimolar) have been identified on actin monomers and filaments. This review focuses on discrete, low-affinity cation-binding interactions that drive polymerization, regulate filament-bending mechanics, and modulate interactions with regulatory proteins. Cation binding may be perturbed by actin post-translational modifications and linked equilibria. Partial cation occupancy under physiological and commonly used in vitro solution conditions likely contribute to filament mechanical heterogeneity and structural polymorphism. Site-specific cation-binding residues are conserved in Arp2 and Arp3, and may play a role in Arp2/3 complex activation and actin-filament branching activity. Actin-salt interactions demonstrate the relevance of ion-linked equilibria in the operation and regulation of complex biological systems.     

Equilibria and absorption spectra of tryptophanase

Tryptophanase (tryptophan: indole-lyase) from Escherichia coli has been isolated in the holoenzyme form and its absorption spectra and acid-base chemistry have been reevaluated. Apoenzyme has been prepared by dialysis against sodium phosphate and L-alanine and molar absorptivities of the coenzyme bands have been estimated by readdition of pyridoxal 5'-phosphate. The spectrophotometric titration curve, whose midpoint is at pH 7.6 in 0.1 M potassium phosphate buffers, indicates some degree of cooperativity in dissociation of a pair of protons. Resolution of the computed spectra of individual ionic forms of the enzyme with lognormal distribution curves shows that band shapes are similar to those of model Schiff bases and of aspartate aminotransferase. Using molar areas from the latter we estimated amounts of individual tautomeric species. In addition to ketoenamine and enolimine or covalent adduct the high pH form also appears to contain approximately 18% of a species with a dipolar ionic ring (protonated on the ring nitrogen and with phenolate -O-). We suggest that this may be the catalytically active form of the coenzyme in tryptophanase. The equilibrium between tryptophanase and L-alanine has also been reevaluated.