Boolean Models of Genomic Regulatory Networks: Reduction Mappings,Inference, and External Control

Computational modeling of genomic regulation has become an important focus of systems biology and genomic signal processing for the past several years. It holds the promise to uncover both the structure and dynamical properties of the complex gene, protein or metabolic networks responsible for the cell functioning in various contexts and regimes. This, in turn, will lead to the development of optimal intervention strategies for prevention and control of disease. At the same time, constructing such computational models faces several challenges. High complexity is one of the major impediments for the practical applications of the models. Thus, reducing the size/complexity of a model becomes a critical issue in problems such as model selection, construction of tractable subnetwork models, and control of its dynamical behavior. We focus on the reduction problem in the context of two specific models of genomic regulation: Boolean networks with perturbation (BN_P) and probabilistic Boolean networks (PBN). We also compare and draw a parallel between the reduction problem and two other important problems of computational modeling of genomic networks: the problem of network inference and the problem of designing external control policies for intervention/altering the dynamics of the model.     

A Bridge Role Metric Model for Nodes in Software Networks

A bridge role metric model is put forward in this paper. Compared with previous metric models, our solution of a large-scale object-oriented software system as a complex network is inherently more realistic. To acquire nodes and links in an undirected network, a new model that presents the crucial connectivity of a module or the hub instead of only centrality as in previous metric models is presented. Two previous metric models are described for comparison. In addition, it is obvious that the fitting curve between the results and degrees can well be fitted by a power law. The model represents many realistic characteristics of actual software structures, and a hydropower simulation system is taken as an example. This paper makes additional contributions to an accurate understanding of module design of software systems and is expected to be beneficial to software engineering practices.     

The feasibility of genome-scale biological network inference using Graphics Processing Units

Systems research spanning fields from biology to finance involves the identification of models to represent the underpinnings of complex systems. Formal approaches for data-driven identification of network interactions include statistical inference-based approaches and methods to identify dynamical systems models that are capable of fitting multivariate data. Availability of large data sets and so-called ‘big data’ applications in biology present great opportunities as well as major challenges for systems identification/reverse engineering applications. For example, both inverse identification and forward simulations of genome-scale gene regulatory network models pose compute-intensive problems. This issue is addressed here by combining the processing power of Graphics Processing Units (GPUs) and a parallel reverse engineering algorithm for inference of regulatory networks. It is shown that, given an appropriate data set, information on genome-scale networks (systems of 1000 or more state variables) can be inferred using a reverse-engineering algorithm in a matter of days on a small-scale modern GPU cluster.     

Network Reconstruction Using Nonparametric Additive ODE Models

Network representations of biological systems are widespread and reconstructing unknown networks from data is a focal problem for computational biologists. For example, the series of biochemical reactions in a metabolic pathway can be represented as a network, with nodes corresponding to metabolites and edges linking reactants to products. In a different context, regulatory relationships among genes are commonly represented as directed networks with edges pointing from influential genes to their targets. Reconstructing such networks from data is a challenging problem receiving much attention in the literature. There is a particular need for approaches tailored to time-series data and not reliant on direct intervention experiments, as the former are often more readily available. In this paper, we introduce an approach to reconstructing directed networks based on dynamic systems models. Our approach generalizes commonly used ODE models based on linear or nonlinear dynamics by extending the functional class for the functions involved from parametric to nonparametric models. Concomitantly we limit the complexity by imposing an additive structure on the estimated slope functions. Thus the submodel associated with each node is a sum of univariate functions. These univariate component functions form the basis for a novel coupling metric that we define in order to quantify the strength of proposed relationships and hence rank potential edges. We show the utility of the method by reconstructing networks using simulated data from computational models for the glycolytic pathway of Lactocaccus Lactis and a gene network regulating the pluripotency of mouse embryonic stem cells. For purposes of comparison, we also assess reconstruction performance using gene networks from the DREAM challenges. We compare our method to those that similarly rely on dynamic systems models and use the results to attempt to disentangle the distinct roles of linearity, sparsity, and derivative estimation.     

The challenge of modeling niches and distributions for data‐poor species: a comprehensive approach to model complexity

Models of species ecological niches and geographic distributions now represent a widely used tool in ecology, evolution, and biogeography. However, the very common situation of species with few available occurrence localities presents major challenges for such modeling techniques, in particular regarding model complexity and evaluation. Here, we summarize the state of the field regarding these issues and provide a worked example using the technique Maxent for a small mammal endemic to Madagascar (the nesomyine rodent Eliurus majori). Two relevant model‐selection approaches exist in the literature (information criteria, specifically AICc; and performance predicting withheld data, via a jackknife), but AICc is not strictly applicable to machine‐learning algorithms like Maxent. We compare models chosen under each selection approach with those corresponding to Maxent default settings, both with and without spatial filtering of occurrence records to reduce the effects of sampling bias. Both selection approaches chose simpler models than those made using default settings. Furthermore, the approaches converged on a similar answer when sampling bias was taken into account, but differed markedly with the unfiltered occurrence data. Specifically, for that dataset, the models selected by AICc had substantially fewer parameters than those identified by performance on withheld data. Based on our knowledge of the study species, models chosen under both AICc and withheld‐data‐selection showed higher ecological plausibility when combined with spatial filtering. The results for this species intimate that AICc may consistently select models with fewer parameters and be more robust to sampling bias. To test these hypotheses and reach general conclusions, comprehensive research should be undertaken with a wide variety of real and simulated species. Meanwhile, we recommend that researchers assess the critical yet underappreciated issue of model complexity both via information criteria and performance on withheld data, comparing the results between the two approaches and taking into account ecological plausibility.     

Balance between Noise and Information Flow Maximizes Set Complexity of Network Dynamics

Boolean networks have been used as a discrete model for several biological systems, including metabolic and genetic regulatory networks. Due to their simplicity they offer a firm foundation for generic studies of physical systems. In this work we show, using a measure of context-dependent information, set complexity, that prior to reaching an attractor, random Boolean networks pass through a transient state characterized by high complexity. We justify this finding with a use of another measure of complexity, namely, the statistical complexity. We show that the networks can be tuned to the regime of maximal complexity by adding a suitable amount of noise to the deterministic Boolean dynamics. In fact, we show that for networks with Poisson degree distributions, all networks ranging from subcritical to slightly supercritical can be tuned with noise to reach maximal set complexity in their dynamics. For networks with a fixed number of inputs this is true for near-to-critical networks. This increase in complexity is obtained at the expense of disruption in information flow. For a large ensemble of networks showing maximal complexity, there exists a balance between noise and contracting dynamics in the state space. In networks that are close to critical the intrinsic noise required for the tuning is smaller and thus also has the smallest effect in terms of the information processing in the system. Our results suggest that the maximization of complexity near to the state transition might be a more general phenomenon in physical systems, and that noise present in a system may in fact be useful in retaining the system in a state with high information content.     

Inferring general relations between network characteristics from specific network ensembles

Different network models have been suggested for the topology underlying complex interactions in natural systems. These models are aimed at replicating specific statistical features encountered in real-world networks. However, it is rarely considered to which degree the results obtained for one particular network class can be extrapolated to real-world networks. We address this issue by comparing different classical and more recently developed network models with respect to their ability to generate networks with large structural variability. In particular, we consider the statistical constraints which the respective construction scheme imposes on the generated networks. After having identified the most variable networks, we address the issue of which constraints are common to all network classes and are thus suitable candidates for being generic statistical laws of complex networks. In fact, we find that generic, not model-related dependencies between different network characteristics do exist. This makes it possible to infer global features from local ones using regression models trained on networks with high generalization power. Our results confirm and extend previous findings regarding the synchronization properties of neural networks. Our method seems especially relevant for large networks, which are difficult to map completely, like the neural networks in the brain. The structure of such large networks cannot be fully sampled with the present technology. Our approach provides a method to estimate global properties of under-sampled networks in good approximation. Finally, we demonstrate on three different data sets (C. elegans neuronal network, R. prowazekii metabolic network, and a network of synonyms extracted from Roget's Thesaurus) that real-world networks have statistical relations compatible with those obtained using regression models.     

Robustness in phenotypic plasticity and heterogeneity patterns enabled by EMT networks

Epithelial-mesenchymal plasticity (EMP) is a key arm of cancer metastasis and is observed across many contexts. Cells undergoing EMP can reversibly switch between three classes of phenotypes: epithelial (E), mesenchymal (M), and hybrid E/M. While a large number of multistable regulatory networks have been identified to be driving EMP in various contexts, the exact mechanisms and design principles that enable robustness in driving EMP across contexts are not yet fully understood. Here, we investigated dynamic and structural robustness in EMP networks with regard to phenotypic heterogeneity and plasticity. We use two different approaches to simulate these networks: a computationally inexpensive, parameter-independent continuous state space Boolean model, and an ODE-based parameter-agnostic framework (RACIPE), both of which yielded similar phenotypic distributions. While the latter approach is useful for measurements of plasticity, the former model enabled us to extensively investigate robustness in phenotypic heterogeneity. Using perturbations to network topology and by varying network parameters, we show that multistable EMP networks are structurally and dynamically more robust compared with their randomized counterparts, thereby highlighting their topological hallmarks. These features of robustness are governed by a balance of positive and negative feedback loops embedded in these networks. Using a combination of the number of negative and positive feedback loops weighted by their lengths, we identified a metric that can explain the structural and dynamical robustness of these networks. This metric enabled us to compare networks across multiple sizes, and the network principles thus obtained can be used to identify fragilities in large networks without simulating their dynamics. Our analysis highlights a network topology-based approach to quantify robustness in the phenotypic heterogeneity and plasticity emergent from EMP networks.     

From dissimilarities among species to dissimilarities among communities: a double principal coordinate analysis

This paper proposes a new method to reverse engineer gene regulatory networks from experimental data. The modeling framework used is time-discrete deterministic dynamical systems, with a finite set of states for each of the variables. The simplest examples of such models are Boolean networks, in which variables have only two possible states. The use of a larger number of possible states allows a finer discretization of experimental data and more than one possible mode of action for the variables, depending on threshold values. Furthermore, with a suitable choice of state set, one can employ powerful tools from computational algebra, that underlie the reverse-engineering algorithm, avoiding costly enumeration strategies. To perform well, the algorithm requires wildtype together with perturbation time courses. This makes it suitable for small to meso-scale networks rather than networks on a genome-wide scale. An analysis of the complexity of the algorithm is performed. The algorithm is validated on a recently published Boolean network model of segment polarity development in Drosophila melanogaster.     

Timing in the absence of clocks: encoding time in neural network states

Decisions based on the timing of sensory events are fundamental to sensory processing. However, the mechanisms by which the brain measures time over ranges of milliseconds to seconds remain unclear. The dominant model of temporal processing proposes that an oscillator emits events that are integrated to provide a linear metric of time. We examine an alternate model in which cortical networks are inherently able to tell time as a result of time-dependent changes in network state. Using computer simulations we show that within this framework, there is no linear metric of time, and that a given interval is encoded in the context of preceding events. Human psychophysical studies were used to examine the predictions of the model. Our results provide theoretical and experimental evidence that, for short intervals, there is no linear metric of time, and that time may be encoded in the high-dimensional state of local neural networks.     

Modeling of environmental and genetic interactions with AMBROSIA, an information-theoretic model synthesis method

To develop a model synthesis method for parsimoniously modeling gene-environmental interactions (GEI) associated with clinical outcomes and phenotypes. The AMBROSIA model synthesis approach utilizes the k-way interaction information (KWII), an information-theoretic metric capable of identifying variable combinations associated with GEI. For model synthesis, AMBROSIA considers relevance of combinations to the phenotype, it precludes entry of combinations with redundant information, and penalizes for unjustifiable complexity; each step is KWII based. The performance and power of AMBROSIA were evaluated with simulations and Genetic Association Workshop 15 (GAW15) data sets of rheumatoid arthritis (RA). AMBROSIA identified parsimonious models in data sets containing multiple interactions with linkage disequilibrium present. For the GAW15 data set containing 9187 single-nucleotide polymorphisms, the parsimonious AMBROSIA model identified nine RA-associated combinations with power >90%. AMBROSIA was compared with multifactor dimensionality reduction across several diverse models and had satisfactory power. Software source code is available from http://www.cse.buffalo.edu/DBGROUP/bioinformatics/resources.html. AMBROSIA is a promising method for GEI model synthesis.     

Challenges in developing a computationally efficient plant physiological height-class-structured forest model

Ongoing and future climate change may be of sufficient magnitude to significantly impact global forest ecosystems. In order to anticipate the potential range of changes to forests in the future and to better understand the development and state of forest ecosystems at present, a variety of forest ecosystem models of varying complexity have been developed over the past 40 years. While most of these models focus on representing either forest demographics including age and height structure, or forest biogeochemistry including plant physiology and ecosystem carbon cycling, it is increasingly seen as crucial that forest ecosystem models include equally good representations of both. However, only few models currently include detailed representations of both biogeochemistry and demographics, and those mostly have high computational demands. Here, we present TreeM-LPJ, a first step towards a new, computationally efficient forest dynamics model. We combine the height-class scheme of the forest landscape model TreeMig with the biogeochemistry of the dynamic global vegetation model LPJ-GUESS. The resulting model is able to simulate forest growth by considering vertical spatial variability without stochastic functions, considerably reducing computational demand. Discretization errors are kept small by using a numerical algorithm that extrapolates growth success in height, and thereby dynamically updates the state variables of the trees in the different height classes. We demonstrate TreeM-LPJ in an application on a transect in the central Swiss Alps where we show results from the new model compare favorably with the more complex LPJ-GUESS. TreeM-LPJ provides a combination of biological detail and computational efficiency that can serve as a useful basis for large-scale vegetation modeling.     

Rule extraction from minimal neural networks for credit card screening

While feedforward neural networks have been widely accepted as effective tools for solving classification problems, the issue of finding the best network architecture remains unresolved, particularly so in real-world problem settings. We address this issue in the context of credit card screening, where it is important to not only find a neural network with good predictive performance but also one that facilitates a clear explanation of how it produces its predictions. We show that minimal neural networks with as few as one hidden unit provide good predictive accuracy, while having the added advantage of making it easier to generate concise and comprehensible classification rules for the user. To further reduce model size, a novel approach is suggested in which network connections from the input units to this hidden unit are removed by a very straightaway pruning procedure. In terms of predictive accuracy, both the minimized neural networks and the rule sets generated from them are shown to compare favorably with other neural network based classifiers. The rules generated from the minimized neural networks are concise and thus easier to validate in a real-life setting.     

Link Community Detection Using Generative Model and Nonnegative Matrix Factorization

Discovery of communities in complex networks is a fundamental data analysis problem with applications in various domains. While most of the existing approaches have focused on discovering communities of nodes, recent studies have shown the advantages and uses of link community discovery in networks. Generative models provide a promising class of techniques for the identification of modular structures in networks, but most generative models mainly focus on the detection of node communities rather than link communities. In this work, we propose a generative model, which is based on the importance of each node when forming links in each community, to describe the structure of link communities. We proceed to fit the model parameters by taking it as an optimization problem, and solve it using nonnegative matrix factorization. Thereafter, in order to automatically determine the number of communities, we extend the above method by introducing a strategy of iterative bipartition. This extended method not only finds the number of communities all by itself, but also obtains high efficiency, and thus it is more suitable to deal with large and unexplored real networks. We test this approach on both synthetic benchmarks and real-world networks including an application on a large biological network, and compare it with two highly related methods. Results demonstrate the superior performance of our approach over competing methods for the detection of link communities.     

A metric model of amino acid substitution

MOTIVATION: We address the question of whether there exists an effective evolutionary model of amino-acid substitution that forms a metric-distance function. There is always a trade-off between speed and sensitivity among competing computational methods of determining sequence homology. A metric model of evolution is a prerequisite for the development of an entire class of fast sequence analysis algorithms that are both scalable, O(log n) and sensitive. RESULTS: We have reworked the mathematics of the point accepted mutation model (PAM) by calculating the expected time between accepted mutations in lieu of calculating log-odds probabilities. The resulting substitution matrix (mPAM) forms a metric. We validate the application of the mPAM evolutionary model for sequence homology by executing sequence queries from a controlled yeast protein homology search benchmark. We compare the accuracy of the results of mPAM and PAM similarity matrices as well as three prior metric models. The experiment shows that mPAM significantly outperforms the other three metrics and sufficiently approaches the sensitivity of PAM250 to make it applicable to the management of protein sequence databases.     

EEG classification of driver mental states by deep learning

Driver fatigue is attracting more and more attention, as it is the main cause of traffic accidents, which bring great harm to society and families. This paper proposes to use deep convolutional neural networks, and deep residual learning, to predict the mental states of drivers from electroencephalography (EEG) signals. Accordingly we have developed two mental state classification models called EEG-Conv and EEG-Conv-R. Tested on intra- and inter-subject, our results show that both models outperform the traditional LSTM- and SVM-based classifiers. Our major findings include (1) Both EEG-Conv and EEG-Conv-R yield very good classification performance for mental state prediction; (2) EEG-Conv-R is more suitable for inter-subject mental state prediction; (3) EEG-Conv-R converges more quickly than EEG-Conv. In summary, our proposed classifiers have better predictive power and are promising for application in practical brain-computer interaction .     

Does structural sensitivity alter complexity–stability relationships?

Structural sensitivity, namely the sensitivity of a model dynamics to slight changes in its mathematical formulation, has already been studied in some models with a small number of state variables. The aim of this study is to investigate the impact of structural sensitivity in a food web model. Especially, the importance of structural sensitivity is compared to that of trophic complexity (number of species, connectance), which is known to strongly influence food web dynamics. Food web structures are built using the niche model. Then food web dynamics are modeled using several type II functional responses parameterized to fit the same predation fluxes. Food web persistence was found to be mostly determined by trophic complexity. At the opposite, even if food web connectance promotes equilibrium dynamics, their occurrence is mainly driven by the choice of the functional response. These conclusions are robust to changes in some parameter values, the fitting method and some model assumptions. In a one-prey/one-predator system, it was shown that the possibility that multiple stable states coexist can be highly structural sensitive. Quantifying this type of uncertainty at the scale of ecosystem models will be both a natural extension to this work and a challenging issue.     

Measuring acoustic complexity in continuously varying signals: how complex is a wolf howl?

Communicative complexity is a key behavioural and ecological indicator in the study of animal cognition. Much attention has been given to measures such as repertoire size and syntactic structure in both bird and mammal vocalizations, as large repertoires and complex call combinations may give an indication of the cognitive abilities both of the sender and receiver. However, many animals communicate using a continuous vocal signal that does not easily lend itself to be described by concepts such as ‘repertoire’. For example, dolphin whistles and wolf howls both have complex patterns of frequency modulation, so that no two howls or whistles are quite the same. Is there a sense in which some of these vocalizations may be more ‘complex’ than others? Can we arrive at a quantitative metric for complexity in a continuously varying signal? Such a metric would allow us to extend familiar analyses of communicative complexity to those species where vocal behaviour is not restricted to sequences of stereotyped syllables. We present four measures of complexity in continuous signals (Wiener Entropy, Autocorrelation, Inflection Point Count, and Parsons Entropy), and examine their relevance using example data from members of the genus Canis. We show that each metric can lead to different conclusions regarding which howls could be considered complex or not. Ultimately, complexity is poorly defined and researchers must compare metrics to ensure that they reflect the properties for which the hypothesis is being tested.     

Comparative performance of Bayesian and AIC-based measures of phylogenetic model uncertainty

Reversible-jump Markov chain Monte Carlo (RJ-MCMC) is a technique for simultaneously evaluating multiple related (but not necessarily nested) statistical models that has recently been applied to the problem of phylogenetic model selection. Here we use a simulation approach to assess the performance of this method and compare it to Akaike weights, a measure of model uncertainty that is based on the Akaike information criterion. Under conditions where the assumptions of the candidate models matched the generating conditions, both Bayesian and AIC-based methods perform well. The 95% credible interval contained the generating model close to 95% of the time. However, the size of the credible interval differed with the Bayesian credible set containing approximately 25% to 50% fewer models than an AIC-based credible interval. The posterior probability was a better indicator of the correct model than the Akaike weight when all assumptions were met but both measures performed similarly when some model assumptions were violated. Models in the Bayesian posterior distribution were also more similar to the generating model in their number of parameters and were less biased in their complexity. In contrast, Akaike-weighted models were more distant from the generating model and biased towards slightly greater complexity. The AIC-based credible interval appeared to be more robust to the violation of the rate homogeneity assumption. Both AIC and Bayesian approaches suggest that substantial uncertainty can accompany the choice of model for phylogenetic analyses, suggesting that alternative candidate models should be examined in analysis of phylogenetic data. [AIC; Akaike weights; Bayesian phylogenetics; model averaging; model selection; model uncertainty; posterior probability; reversible jump.].