Accuracy and response-time distributions for decision-making: linear perfect integrators versus nonlinear attractor-based neural circuits

Animals choose actions based on imperfect, ambiguous data. “Noise” inherent in neural processing adds further variability to this already-noisy input signal. Mathematical analysis has suggested that the optimal apparatus (in terms of the speed/accuracy trade-off) for reaching decisions about such noisy inputs is perfect accumulation of the inputs by a temporal integrator. Thus, most highly cited models of neural circuitry underlying decision-making have been instantiations of a perfect integrator. Here, in accordance with a growing mathematical and empirical literature, we describe circumstances in which perfect integration is rendered suboptimal. In particular we highlight the impact of three biological constraints: (1) significant noise arising within the decision-making circuitry itself; (2) bounding of integration by maximal neural firing rates; and (3) time limitations on making a decision. Under conditions (1) and (2), an attractor system with stable attractor states can easily best an integrator when accuracy is more important than speed. Moreover, under conditions in which such stable attractor networks do not best the perfect integrator, a system with unstable initial states can do so if readout of the system’s final state is imperfect. Ubiquitously, an attractor system with a nonselective time-dependent input current is both more accurate and more robust to imprecise tuning of parameters than an integrator with such input. Given that neural responses that switch stochastically between discrete states can “masquerade” as integration in single-neuron and trial-averaged data, our results suggest that such networks should be considered as plausible alternatives to the integrator model.     

Dimension reduction-based penalized logistic regression for cancer classification using microarray data

The use of penalized logistic regression for cancer classification using microarray expression data is presented. Two dimension reduction methods are respectively combined with the penalized logistic regression so that both the classification accuracy and computational speed are enhanced. Two other machine-learning methods, support vector machines and least-squares regression, have been chosen for comparison. It is shown that our methods have achieved at least equal or better results. They also have the advantage that the output probability can be explicitly given and the regression coefficients are easier to interpret. Several other aspects, such as the selection of penalty parameters and components, pertinent to the application of our methods for cancer classification are also discussed.     

A Parallel Implementation of the Network Identification by Multiple Regression (NIR) Algorithm to Reverse-Engineer Regulatory Gene Networks

The reverse engineering of gene regulatory networks using gene expression profile data has become crucial to gain novel biological knowledge. Large amounts of data that need to be analyzed are currently being produced due to advances in microarray technologies. Using current reverse engineering algorithms to analyze large data sets can be very computational-intensive. These emerging computational requirements can be met using parallel computing techniques. It has been shown that the Network Identification by multiple Regression (NIR) algorithm performs better than the other ready-to-use reverse engineering software. However it cannot be used with large networks with thousands of nodes - as is the case in biological networks - due to the high time and space complexity. In this work we overcome this limitation by designing and developing a parallel version of the NIR algorithm. The new implementation of the algorithm reaches a very good accuracy even for large gene networks, improving our understanding of the gene regulatory networks that is crucial for a wide range of biomedical applications.     

Evolution of competitive ability: an adaptation speed vs. accuracy tradeoff rooted in gene network size

Ecologists have increasingly come to understand that evolutionary change on short time-scales can alter ecological dynamics (and vice-versa), and this idea is being incorporated into community ecology research programs. Previous research has suggested that the size and topology of the gene network underlying a quantitative trait should constrain or facilitate adaptation and thereby alter population dynamics. Here, I consider a scenario in which two species with different genetic architectures compete and evolve in fluctuating environments. An important trade-off emerges between adaptive accuracy and adaptive speed, driven by the size of the gene network underlying the ecologically-critical trait and the rate of environmental change. Smaller, scale-free networks confer a competitive advantage in rapidly-changing environments, but larger networks permit increased adaptive accuracy when environmental change is sufficiently slow to allow a species time to adapt. As the differences in network characteristics increase, the time-to-resolution of competition decreases. These results augment and refine previous conclusions about the ecological implications of the genetic architecture of quantitative traits, emphasizing a role of adaptive accuracy. Along with previous work, in particular that considering the role of gene network connectivity, these results provide a set of expectations for what we may observe as the field of ecological genomics develops.     

PCA based population generation for genetic network optimization

A gene regulatory network (GRN) represents a set of genes and its regulatory interactions. The inference of the regulatory interactions between genes is usually carried out using an appropriate mathematical model and the available gene expression profile. Among the various models proposed for GRN inference, our recently proposed Michaelis–Menten based ODE model provides a good trade-off between the computational complexity and biological relevance. This model, like other known GRN models, also uses an evolutionary algorithm for parameter estimation. Considering various issues associated with such population based stochastic optimization approaches (e.g. diversity, premature convergence due to local optima, accuracy, etc.), it becomes important to seed the initial population with good individuals which are closer to the optimal solution. In this paper, we exploit the inherent strength of principal component analysis (PCA) in a novel manner to initialize the population for GRN optimization. The benefit of the proposed method is validated by reconstructing in silico and in vivo networks of various sizes. For the same level of accuracy, the approach with PCA based initialization shows improved convergence speed.     

Trajectory formation and speed-accuracy trade-off in aiming movements

The trade-off between speed and accuracy and the patterning of movement kinematics have been central issues for theories of human movement for almost a century. In the present contribution experimentally obtained kinematics of reciprocal aiming movements, performed under different task conditions, are modelled as resulting from a single non-linear dynamical system whose parameters vary so as to respond to the task demands. Providing a unified account of speed-accuracy trade-off and trajectory formation phenomena, the model offers a theoretical framework in which both discrete and continuous movements, performed along one or more dimensions can be understood.     

“Neural” computation of decisions in optimization problems

Highly-interconnected networks of nonlinear analog neurons are shown to be extremely effective in computing. The networks can rapidly provide a collectively-computed solution (a digital output) to a problem on the basis of analog input information. The problems to be solved must be formulated in terms of desired optima, often subject to constraints. The general principles involved in constructing networks to solve specific problems are discussed. Results of computer simulations of a network designed to solve a difficult but well-defined optimization problem-the Traveling-Salesman Problem-are presented and used to illustrate the computational power of the networks. Good solutions to this problem are collectively computed within an elapsed time of only a few neural time constants. The effectiveness of the computation involves both the nonlinear analog response of the neurons and the large connectivity among them. Dedicated networks of biological or microelectronic neurons could provide the computational capabilities described for a wide class of problems having combinatorial complexity. The power and speed naturally displayed by such collective networks may contribute to the effectiveness of biological information processing.     

Tension and robustness in multitasking cellular networks

Cellular networks multitask by exhibiting distinct, context-dependent dynamics. However, network states (parameters) that generate a particular dynamic are often sub-optimal for others, defining a source of "tension" between them. Though multitasking is pervasive, it is not clear where tension arises, what consequences it has, and how it is resolved. We developed a generic computational framework to examine the source and consequences of tension between pairs of dynamics exhibited by the well-studied RB-E2F switch regulating cell cycle entry. We found that tension arose from task-dependent shifts in parameters associated with network modules. Although parameter sets common to distinct dynamics did exist, tension reduced both their accessibility and resilience to perturbation, indicating a trade-off between "one-size-fits-all" solutions and robustness. With high tension, robustness can be preserved by dynamic shifting of modules, enabling the network to toggle between tasks, and by increasing network complexity, in this case by gene duplication. We propose that tension is a general constraint on the architecture and operation of multitasking biological networks. To this end, our work provides a framework to quantify the extent of tension between any network dynamics and how it affects network robustness. Such analysis would suggest new ways to interfere with network elements to elucidate the design principles of cellular networks.     

Influence of prior knowledge in constraint-based learning of gene regulatory networks

Constraint-based structure learning algorithms generally perform well on sparse graphs. Although sparsity is not uncommon, there are some domains where the underlying graph can have some dense regions; one of these domains is gene regulatory networks, which is the main motivation to undertake the study described in this paper. We propose a new constraint-based algorithm that can both increase the quality of output and decrease the computational requirements for learning the structure of gene regulatory networks. The algorithm is based on and extends the PC algorithm. Two different types of information are derived from the prior knowledge; one is the probability of existence of edges, and the other is the nodes that seem to be dependent on a large number of nodes compared to other nodes in the graph. Also a new method based on Gene Ontology for gene regulatory network validation is proposed. We demonstrate the applicability and effectiveness of the proposed algorithms on both synthetic and real data sets.     

Is the whole more than the sum of its parts? Evolutionary trade-offs between burst and sustained locomotion in lacertid lizards

Trade-offs arise when two functional traits impose conflicting demands on the same design trait. Consequently, excellence in one comes at the cost of performance in the other. One of the most widely studied performance trade-offs is the one between sprint speed and endurance. Although biochemical, physiological and (bio)mechanical correlates of either locomotor trait conflict with each other, results at the whole-organism level are mixed. Here, we test whether burst (speed, acceleration) and sustained locomotion (stamina) trade off at both the isolated muscle and whole-organism level among 17 species of lacertid lizards. In addition, we test for a mechanical link between the organismal and muscular (power output, fatigue resistance) performance traits. We find weak evidence for a trade-off between burst and sustained locomotion at the whole-organism level; however, there is a significant trade-off between muscle power output and fatigue resistance in the isolated muscle level. Variation in whole-animal sprint speed can be convincingly explained by variation in muscular power output. The variation in locomotor stamina at the whole-organism level does not relate to the variation in muscle fatigue resistance, suggesting that whole-organism stamina depends not only on muscle contractile performance but probably also on the performance of the circulatory and respiratory systems.     

Optimizing Semantic Pointer Representations for Symbol-Like Processing in Spiking Neural Networks

The Semantic Pointer Architecture (SPA) is a proposal of specifying the computations and architectural elements needed to account for cognitive functions. By means of the Neural Engineering Framework (NEF) this proposal can be realized in a spiking neural network. However, in any such network each SPA transformation will accumulate noise. By increasing the accuracy of common SPA operations, the overall network performance can be increased considerably. As well, the representations in such networks present a trade-off between being able to represent all possible values and being only able to represent the most likely values, but with high accuracy. We derive a heuristic to find the near-optimal point in this trade-off. This allows us to improve the accuracy of common SPA operations by up to 25 times. Ultimately, it allows for a reduction of neuron number and a more efficient use of both traditional and neuromorphic hardware, which we demonstrate here.     

Dynamic properties of network motifs contribute to biological network organization

Biological networks, such as those describing gene regulation, signal transduction, and neural synapses, are representations of large-scale dynamic systems. Discovery of organizing principles of biological networks can be enhanced by embracing the notion that there is a deep interplay between network structure and system dynamics. Recently, many structural characteristics of these non-random networks have been identified, but dynamical implications of the features have not been explored comprehensively. We demonstrate by exhaustive computational analysis that a dynamical property—stability or robustness to small perturbations—is highly correlated with the relative abundance of small subnetworks (network motifs) in several previously determined biological networks. We propose that robust dynamical stability is an influential property that can determine the non-random structure of biological networks.     

The Main Sequence of Saccades Optimizes Speed-accuracy Trade-off

In primates, it is well known that there is a consistent relationship between the duration, peak velocity and amplitude of saccadic eye movements, known as the ‘main sequence’. The reason why such a stereotyped relationship evolved is unknown. We propose that a fundamental constraint on the deployment of foveal vision lies in the motor system that is perturbed by signal-dependent noise (proportional noise) on the motor command. This noise imposes a compromise between the speed and accuracy of an eye movement. We propose that saccade trajectories have evolved to optimize a trade-off between the accuracy and duration of the movement. Taking a semi-analytical approach we use Pontryagin’s minimum principle to show that there is an optimal trajectory for a given amplitude and duration; and that there is an optimal duration for a given amplitude. It follows that the peak velocity is also fixed for a given amplitude. These predictions are in good agreement with observed saccade trajectories and the main sequence. Moreover, this model predicts a small saccadic dead-zone in which it is better to stay eccentric of target than make a saccade onto target. We conclude that the main sequence has evolved as a strategy to optimize the trade-off between accuracy and speed.     

Topology of gene expression networks as revealed by data mining and modeling

MOTIVATION: Interpretation of high-throughput gene expression profiling requires a knowledge of the design principles underlying the networks that sustain cellular machinery. Recently a novel approach based on the study of network topologies has been proposed. This methodology has proven to be useful for the analysis of a variety of biological systems, including metabolic networks, networks of protein-protein interactions, and gene networks that can be derived from gene expression data. In the present paper, we focus on several important issues related to the topology of gene expression networks that have not yet been fully studied. RESULTS: The networks derived from gene expression profiles for both time series experiments in yeast and perturbation experiments in cell lines are studied. We demonstrate that independent from the experimental organism (yeast versus cell lines) and the type of experiment (time courses versus perturbations) the extracted networks have similar topological characteristics suggesting together with the results of other common principles of the structural organization of biological networks. A novel computational model of network growth that reproduces the basic design principles of the observed networks is presented. Advantage of the model is that it provides a general mechanism to generate networks with different types of topology by a variation of a few parameters. We investigate the robustness of the network structure to random damages and to deliberate removal of the most important parts of the system and show a surprising tolerance of gene expression networks to both kinds of disturbance.     

Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments

Systems biology applies quantitative, mechanistic modelling to study genetic networks, signal transduction pathways and metabolic networks. Mathematical models of biochemical networks can look very different. An important reason is that the purpose and application of a model are essential for the selection of the best mathematical framework. Fundamental aspects of selecting an appropriate modelling framework and a strategy for model building are discussed. Concepts and methods from system and control theory provide a sound basis for the further development of improved and dedicated computational tools for systems biology. Identification of the network components and rate constants that are most critical to the output behaviour of the system is one of the major problems raised in systems biology. Current approaches and methods of parameter sensitivity analysis and parameter estimation are reviewed. It is shown how these methods can be applied in the design of model-based experiments which iteratively yield models that are decreasingly wrong and increasingly gain predictive power.     

There is no trade-off between speed and force in a dynamic lever system

Lever systems within a skeleton transmit force with a capacity determined by the mechanical advantage, A. A is the distance from input force to a joint, divided by the distance from the joint to the output force. A lever with a relatively high A in static equilibrium has a great capacity to generate force but moves a load over a small distance. Therefore, the geometry of a skeletal lever presents a trade-off between force and speed under quasi-static conditions. The present study considers skeletal dynamics that do not assume static equilibrium by modelling kicking by a locust leg, which is powered by stored elastic energy. This model predicts that the output force of this lever is proportional to A, but its maximum speed is independent of A. Therefore, no trade-off between force and velocity exists in a lever system with spring-mass dynamics. This demonstrates that the motion of a skeleton depends on the major forces that govern its dynamics and cannot be inferred from skeletal geometry alone.