Reverse Engineering Boolean Networks: From Bernoulli Mixture Models to Rule Based Systems

A Boolean network is a graphical model for representing and analyzing the behavior of gene regulatory networks (GRN). In this context, the accurate and efficient reconstruction of a Boolean network is essential for understanding the gene regulation mechanism and the complex relations that exist therein. In this paper we introduce an elegant and efficient algorithm for the reverse engineering of Boolean networks from a time series of multivariate binary data corresponding to gene expression data. We call our method ReBMM, i.e., reverse engineering based on Bernoulli mixture models. The time complexity of most of the existing reverse engineering techniques is quite high and depends upon the indegree of a node in the network. Due to the high complexity of these methods, they can only be applied to sparsely connected networks of small sizes. ReBMM has a time complexity factor, which is independent of the indegree of a node and is quadratic in the number of nodes in the network, a big improvement over other techniques and yet there is little or no compromise in accuracy. We have tested ReBMM on a number of artificial datasets along with simulated data derived from a plant signaling network. We also used this method to reconstruct a network from real experimental observations of microarray data of the yeast cell cycle. Our method provides a natural framework for generating rules from a probabilistic model. It is simple, intuitive and illustrates excellent empirical results.     

Mapping multivalued onto Boolean dynamics

This paper deals with the generalized logical framework defined by René Thomas in the 70's to qualitatively represent the dynamics of regulatory networks. In this formalism, a regulatory network is represented as a graph, where nodes denote regulatory components (basically genes) and edges denote regulations between these components. Discrete variables are associated to regulatory components accounting for their levels of expression. In most cases, Boolean variables are enough, but some situations may require further values. Despite this fact, the majority of tools dedicated to the analysis of logical models are restricted to the Boolean case. A formal Boolean mapping of multivalued logical models is a natural way of extending the applicability of these tools.Three decades ago, a multivalued to Boolean variable mapping was proposed by P. Van Ham. Since then, all works related to multivalued logical models and using a Boolean representation rely on this particular mapping. We formally show in this paper that this mapping is actually the sole, up to cosmetic changes, that could preserve the regulatory structures of the underlying graphs as well as their dynamical behaviours.     

A Scalable Computational Framework for Establishing Long-Term Behavior of Stochastic Reaction Networks

Reaction networks are systems in which the populations of a finite number of species evolve through predefined interactions. Such networks are found as modeling tools in many biological disciplines such as biochemistry, ecology, epidemiology, immunology, systems biology and synthetic biology. It is now well-established that, for small population sizes, stochastic models for biochemical reaction networks are necessary to capture randomness in the interactions. The tools for analyzing such models, however, still lag far behind their deterministic counterparts. In this paper, we bridge this gap by developing a constructive framework for examining the long-term behavior and stability properties of the reaction dynamics in a stochastic setting. In particular, we address the problems of determining ergodicity of the reaction dynamics, which is analogous to having a globally attracting fixed point for deterministic dynamics. We also examine when the statistical moments of the underlying process remain bounded with time and when they converge to their steady state values. The framework we develop relies on a blend of ideas from probability theory, linear algebra and optimization theory. We demonstrate that the stability properties of a wide class of biological networks can be assessed from our sufficient theoretical conditions that can be recast as efficient and scalable linear programs, well-known for their tractability. It is notably shown that the computational complexity is often linear in the number of species. We illustrate the validity, the efficiency and the wide applicability of our results on several reaction networks arising in biochemistry, systems biology, epidemiology and ecology. The biological implications of the results as well as an example of a non-ergodic biological network are also discussed.     

Formal Methods for Hopfield-Like Networks

Building a meaningful model of biological regulatory network is usually done by specifying the components (e.g. the genes) and their interactions, by guessing the values of parameters, by comparing the predicted behaviors to the observed ones, and by modifying in a trial-error process both architecture and parameters in order to reach an optimal fitness. We propose here a different approach to construct and analyze biological models avoiding the trial-error part, where structure and dynamics are represented as formal constraints. We apply the method to Hopfield-like networks, a formalism often used in both neural and regulatory networks modeling. The aim is to characterize automatically the set of all models consistent with all the available knowledge (about structure and behavior). The available knowledge is formalized into formal constraints. The latter are compiled into Boolean formula in conjunctive normal form and then submitted to a Boolean satisfiability solver. This approach allows to formulate a wide range of queries, expressed in a high level language, and possibly integrating formalized intuitions. In order to explore its potential, we use it to find cycles for 3-nodes networks and to determine the flower morphogenesis regulatory network of Arabidopsis thaliana. Applications of this technique are numerous and concern the building of models from data as well as the design of biological networks possessing specified behaviors.     

The transition from differential equations to Boolean networks: a case study in simplifying a regulatory network model

Methods for modeling cellular regulatory networks as diverse as differential equations and Boolean networks co-exist, however, without much closer correspondence to each other. With the example system of the fission yeast cell cycle control network, we here discuss these two approaches with respect to each other. We find that a Boolean network model can be formulated as a specific coarse-grained limit of the more detailed differential equations model for this system. This demonstrates the mathematical foundation on which Boolean networks can be applied to biological regulatory networks in a controlled way.     

An Efficient Steady-State Analysis Method for Large Boolean Networks with High Maximum Node Connectivity

Boolean networks have been widely used to model biological processes lacking detailed kinetic information. Despite their simplicity, Boolean network dynamics can still capture some important features of biological systems such as stable cell phenotypes represented by steady states. For small models, steady states can be determined through exhaustive enumeration of all state transitions. As the number of nodes increases, however, the state space grows exponentially thus making it difficult to find steady states. Over the last several decades, many studies have addressed how to handle such a state space explosion. Recently, increasing attention has been paid to a satisfiability solving algorithm due to its potential scalability to handle large networks. Meanwhile, there still lies a problem in the case of large models with high maximum node connectivity where the satisfiability solving algorithm is known to be computationally intractable. To address the problem, this paper presents a new partitioning-based method that breaks down a given network into smaller subnetworks. Steady states of each subnetworks are identified by independently applying the satisfiability solving algorithm. Then, they are combined to construct the steady states of the overall network. To efficiently apply the satisfiability solving algorithm to each subnetwork, it is crucial to find the best partition of the network. In this paper, we propose a method that divides each subnetwork to be smallest in size and lowest in maximum node connectivity. This minimizes the total cost of finding all steady states in entire subnetworks. The proposed algorithm is compared with others for steady states identification through a number of simulations on both published small models and randomly generated large models with differing maximum node connectivities. The simulation results show that our method can scale up to several hundreds of nodes even for Boolean networks with high maximum node connectivity. The algorithm is implemented and available at http://cps.kaist.ac.kr/∼ckhong/tools/download/PAD.tar.gz.     

An Evaluation of Methods for Inferring Boolean Networks from Time-Series Data

Regulatory networks play a central role in cellular behavior and decision making. Learning these regulatory networks is a major task in biology, and devising computational methods and mathematical models for this task is a major endeavor in bioinformatics. Boolean networks have been used extensively for modeling regulatory networks. In this model, the state of each gene can be either ‘on’ or ‘off’ and that next-state of a gene is updated, synchronously or asynchronously, according to a Boolean rule that is applied to the current-state of the entire system. Inferring a Boolean network from a set of experimental data entails two main steps: first, the experimental time-series data are discretized into Boolean trajectories, and then, a Boolean network is learned from these Boolean trajectories. In this paper, we consider three methods for data discretization, including a new one we propose, and three methods for learning Boolean networks, and study the performance of all possible nine combinations on four regulatory systems of varying dynamics complexities. We find that employing the right combination of methods for data discretization and network learning results in Boolean networks that capture the dynamics well and provide predictive power. Our findings are in contrast to a recent survey that placed Boolean networks on the low end of the “faithfulness to biological reality” and “ability to model dynamics” spectra. Further, contrary to the common argument in favor of Boolean networks, we find that a relatively large number of time points in the time-series data is required to learn good Boolean networks for certain data sets. Last but not least, while methods have been proposed for inferring Boolean networks, as discussed above, missing still are publicly available implementations thereof. Here, we make our implementation of the methods available publicly in open source at http://bioinfo.cs.rice.edu/.     

Majority Rules with Random Tie-Breaking in Boolean Gene Regulatory Networks

We consider threshold Boolean gene regulatory networks, where the update function of each gene is described as a majority rule evaluated among the regulators of that gene: it is turned ON when the sum of its regulator contributions is positive (activators contribute positively whereas repressors contribute negatively) and turned OFF when this sum is negative. In case of a tie (when contributions cancel each other out), it is often assumed that the gene keeps it current state. This framework has been successfully used to model cell cycle control in yeast. Moreover, several studies consider stochastic extensions to assess the robustness of such a model.Here, we introduce a novel, natural stochastic extension of the majority rule. It consists in randomly choosing the next value of a gene only in case of a tie. Hence, the resulting model includes deterministic and probabilistic updates. We present variants of the majority rule, including alternate treatments of the tie situation. Impact of these variants on the corresponding dynamical behaviours is discussed. After a thorough study of a class of two-node networks, we illustrate the interest of our stochastic extension using a published cell cycle model. In particular, we demonstrate that steady state analysis can be rigorously performed and can lead to effective predictions; these relate for example to the identification of interactions whose addition would ensure that a specific state is absorbing.     

A Dynamic Gene Regulatory Network Model That Recovers the Cyclic Behavior of Arabidopsis thaliana Cell Cycle

Cell cycle control is fundamental in eukaryotic development. Several modeling efforts have been used to integrate the complex network of interacting molecular components involved in cell cycle dynamics. In this paper, we aimed at recovering the regulatory logic upstream of previously known components of cell cycle control, with the aim of understanding the mechanisms underlying the emergence of the cyclic behavior of such components. We focus on Arabidopsis thaliana, but given that many components of cell cycle regulation are conserved among eukaryotes, when experimental data for this system was not available, we considered experimental results from yeast and animal systems. We are proposing a Boolean gene regulatory network (GRN) that converges into only one robust limit cycle attractor that closely resembles the cyclic behavior of the key cell-cycle molecular components and other regulators considered here. We validate the model by comparing our in silico configurations with data from loss- and gain-of-function mutants, where the endocyclic behavior also was recovered. Additionally, we approximate a continuous model and recovered the temporal periodic expression profiles of the cell-cycle molecular components involved, thus suggesting that the single limit cycle attractor recovered with the Boolean model is not an artifact of its discrete and synchronous nature, but rather an emergent consequence of the inherent characteristics of the regulatory logic proposed here. This dynamical model, hence provides a novel theoretical framework to address cell cycle regulation in plants, and it can also be used to propose novel predictions regarding cell cycle regulation in other eukaryotes.     

Network Class Superposition Analyses

Networks are often used to understand a whole system by modeling the interactions among its pieces. Examples include biomolecules in a cell interacting to provide some primary function, or species in an environment forming a stable community. However, these interactions are often unknown; instead, the pieces'' dynamic states are known, and network structure must be inferred. Because observed function may be explained by many different networks (e.g., for the yeast cell cycle process [1]), considering dynamics beyond this primary function means picking a single network or suitable sample: measuring over all networks exhibiting the primary function is computationally infeasible. We circumvent that obstacle by calculating the network class ensemble. We represent the ensemble by a stochastic matrix , which is a transition-by-transition superposition of the system dynamics for each member of the class. We present concrete results for derived from Boolean time series dynamics on networks obeying the Strong Inhibition rule, by applying to several traditional questions about network dynamics. We show that the distribution of the number of point attractors can be accurately estimated with . We show how to generate Derrida plots based on . We show that -based Shannon entropy outperforms other methods at selecting experiments to further narrow the network structure. We also outline an experimental test of predictions based on . We motivate all of these results in terms of a popular molecular biology Boolean network model for the yeast cell cycle, but the methods and analyses we introduce are general. We conclude with open questions for , for example, application to other models, computational considerations when scaling up to larger systems, and other potential analyses.     

Comparative Evolutionary Analysis of Cell Cycle Proteins Networks in Fission and Budding Yeast

Fission yeast and budding yeast are the two distantly related species with common ancestors. Various studies have shown significant differences in metabolic networks and regulatory networks. Cell cycle regulatory proteins in both species have differences in structural as well as in functional organization. Orthologous proteins in cell cycle regulatory protein networks seem to play contemporary role in both species during the evolution but little is known about non-orthologous proteins. Here, we used system biology approach to compare topological parameters of orthologous and non-orthologous proteins to find their contributions during the evolution to make an efficient cell cycle regulation. Observed results have shown a significant role of non-orthologous proteins in fission yeast in maintaining the efficiency of cell cycle regulation with less number of proteins as compared to budding yeast.     

On the dynamics of random Boolean networks subject to noise: Attractors, ergodic sets and cell types

The asymptotic dynamics of random Boolean networks subject to random fluctuations is investigated. Under the influence of noise, the system can escape from the attractors of the deterministic model, and a thorough study of these transitions is presented. We show that the dynamics is more properly described by sets of attractors rather than single ones. We generalize here a previous notion of ergodic sets, and we show that the Threshold Ergodic Sets so defined are robust with respect to noise and, at the same time, that they do not suffer from a major drawback of ergodic sets. The system jumps from one attractor to another of the same Threshold Ergodic Set under the influence of noise, never leaving it. By interpreting random Boolean networks as models of genetic regulatory networks, we also propose to associate cell types to Threshold Ergodic Sets rather than to deterministic attractors or to ergodic sets, as it had been previously suggested. We also propose to associate cell differentiation to the process whereby a Threshold Ergodic Set composed by several attractors gives rise to another one composed by a smaller number of attractors. We show that this approach accounts for several interesting experimental facts about cell differentiation, including the possibility to obtain an induced pluripotent stem cell from a fully differentiated one by overexpressing some of its genes.     

Boolean network model predicts cell cycle sequence of fission yeast

A Boolean network model of the cell-cycle regulatory network of fission yeast (Schizosaccharomyces Pombe) is constructed solely on the basis of the known biochemical interaction topology. Simulating the model in the computer faithfully reproduces the known activity sequence of regulatory proteins along the cell cycle of the living cell. Contrary to existing differential equation models, no parameters enter the model except the structure of the regulatory circuitry. The dynamical properties of the model indicate that the biological dynamical sequence is robustly implemented in the regulatory network, with the biological stationary state G1 corresponding to the dominant attractor in state space, and with the biological regulatory sequence being a strongly attractive trajectory. Comparing the fission yeast cell-cycle model to a similar model of the corresponding network in S. cerevisiae, a remarkable difference in circuitry, as well as dynamics is observed. While the latter operates in a strongly damped mode, driven by external excitation, the S. pombe network represents an auto-excited system with external damping.     

From Genes to Flower Patterns and Evolution: Dynamic Models of Gene Regulatory Networks

Genes and proteins form complex dynamical systems or gene regulatory networks (GRN) that can reach several steady states (attractors). These may be associated with distinct cell types. In plants, the ABC combinatorial model establishes the necessary gene combinations for floral organ cell specification. We have developed dynamic gene regulatory network (GRN) models to understand how the combinatorial selection of gene activity is established during floral organ primordia specification as a result of the concerted action of ABC and non-ABC genes. Our analyses have shown that the floral organ specification GRN reaches six attractors with gene configurations observed in primordial cell types during early stages of flower development and four that correspond to regions of the inflorescence meristem. This suggests that it is the overall GRN dynamics rather than precise signals that underlie the ABC model. Furthermore, our analyses suggest that the steady states of the GRN are robust to random alterations of the logical functions that define the gene interactions. Here we have updated the GRN model and have systematically altered the outputs of all the logical functions and addressed in which cases the original attractors are recovered. We then reduced the original three-state GRN to a two-state (Boolean) GRN and performed the same systematic perturbation analysis. Interestingly, the Boolean GRN reaches the same number and type of attractors as reached by the three-state GRN, and it responds to perturbations in a qualitatively identical manner as the original GRN. These results suggest that a Boolean model is sufficient to capture the dynamical features of the floral network and provide additional support for the robustness of the floral GRN. These findings further support that the GRN model provides a dynamical explanation for the ABC model and that the floral GRN robustness could be behind the widespread conservation of the floral plan among eudicotyledoneous plants. Other aspects of evolution of flower organ arrangement and ABC gene expression patterns are discussed in the context of the approach proposed here. álvaro Chaos, Max Aldana and Elena Alvarez-Buylla contributed equally to this work.     

Mathematical Conditions for Induced Cell Differentiation and Trans-differentiation in Adult Cells

We present a theoretical framework for the analysis of the effect of a fully differentiated cell population on a neighboring stem cell population in Multi-Cellular Organisms (MCOs). Such an organism is constituted by a set of different cell populations, each set of which converges to a different cycle from all possible options, of the same Boolean network. Cells communicate via a subset of the nodes called signals. We show that generic dynamic properties of cycles and nodes in random Boolean networks can induce cell differentiation. Specifically we propose algorithms, conditions and methods to examine if a set of signaling nodes enabling these conversions can be found. Surprisingly we find that robust conversions can be obtained even with a very small number of signals. The proposed conversions can occur in multiple spatial organizations and can be used as a model for regeneration in MCOs, where an islet of cells of one type (representing stem cells) is surrounded by cells of another type (representing differentiated cells). The cells at the outer layer of the islet function like progenitor cells (i.e. dividing asymmetrically and differentiating). To the best of our knowledge, this is the first work showing a tissue-like regeneration in MCO simulations based on random Boolean networks. We show that the probability to obtain a conversion decreases with the log of the node number in the network, showing that the model is relevant for large networks as well. We have further checked that the conversions are not trivial, i.e. conversions do not occur due to irregular structures of the Boolean network, and the converting cycle undergoes a respectable change in its behavior. Finally we show that the model can also be applied to a realistic genetic regulatory network, showing that the basic mathematical insight from regular networks holds in more complex experiment-based networks.     

A Semiquantitative Framework for Gene Regulatory Networks: Increasing the Time and Quantitative Resolution of Boolean Networks

Boolean models have been instrumental in predicting general features of gene networks and more recently also as explorative tools in specific biological applications. In this study we introduce a basic quantitative and a limited time resolution to a discrete (Boolean) framework. Quantitative resolution is improved through the employ of normalized variables in unison with an additive approach. Increased time resolution stems from the introduction of two distinct priority classes. Through the implementation of a previously published chondrocyte network and T helper cell network, we show that this addition of quantitative and time resolution broadens the scope of biological behaviour that can be captured by the models. Specifically, the quantitative resolution readily allows models to discern qualitative differences in dosage response to growth factors. The limited time resolution, in turn, can influence the reachability of attractors, delineating the likely long term system behaviour. Importantly, the information required for implementation of these features, such as the nature of an interaction, is typically obtainable from the literature. Nonetheless, a trade-off is always present between additional computational cost of this approach and the likelihood of extending the model’s scope. Indeed, in some cases the inclusion of these features does not yield additional insight. This framework, incorporating increased and readily available time and semi-quantitative resolution, can help in substantiating the litmus test of dynamics for gene networks, firstly by excluding unlikely dynamics and secondly by refining falsifiable predictions on qualitative behaviour.     

Maximum Number of Fixed Points in Regulatory Boolean Networks

Boolean networks (BNs) have been extensively used as mathematical models of genetic regulatory networks. The number of fixed points of a BN is a key feature of its dynamical behavior. Here, we study the maximum number of fixed points in a particular class of BNs called regulatory Boolean networks, where each interaction between the elements of the network is either an activation or an inhibition. We find relationships between the positive and negative cycles of the interaction graph and the number of fixed points of the network. As our main result, we exhibit an upper bound for the number of fixed points in terms of minimum cardinality of a set of vertices meeting all positive cycles of the network, which can be applied in the design of genetic regulatory networks.     

A Parallel Attractor Finding Algorithm Based on Boolean Satisfiability for Genetic Regulatory Networks

In biological systems, the dynamic analysis method has gained increasing attention in the past decade. The Boolean network is the most common model of a genetic regulatory network. The interactions of activation and inhibition in the genetic regulatory network are modeled as a set of functions of the Boolean network, while the state transitions in the Boolean network reflect the dynamic property of a genetic regulatory network. A difficult problem for state transition analysis is the finding of attractors. In this paper, we modeled the genetic regulatory network as a Boolean network and proposed a solving algorithm to tackle the attractor finding problem. In the proposed algorithm, we partitioned the Boolean network into several blocks consisting of the strongly connected components according to their gradients, and defined the connection between blocks as decision node. Based on the solutions calculated on the decision nodes and using a satisfiability solving algorithm, we identified the attractors in the state transition graph of each block. The proposed algorithm is benchmarked on a variety of genetic regulatory networks. Compared with existing algorithms, it achieved similar performance on small test cases, and outperformed it on larger and more complex ones, which happens to be the trend of the modern genetic regulatory network. Furthermore, while the existing satisfiability-based algorithms cannot be parallelized due to their inherent algorithm design, the proposed algorithm exhibits a good scalability on parallel computing architectures.     

Regulation of the eukaryotic cell cycle: molecular antagonism, hysteresis, and irreversible transitions

In recent years, molecular biologists have uncovered a wealth of information about the proteins controlling cell growth and division in eukaryotes. The regulatory system is so complex that it defies understanding by verbal arguments alone. Quantitative tools are necessary to probe reliably into the details of cell cycle control. To this end, we convert hypothetical molecular mechanisms into sets of nonlinear ordinary differential equations and use standard analytical and numerical methods to study their solutions. First, we present a simple model of the antagonistic interactions between cyclin-dependent kinases and the anaphase promoting complex, which shows how progress through the cell cycle can be thought of as irreversible transitions (Start and Finish) between two stable states (G1 and S-G2-M) of the regulatory system. Then we add new pieces to the "puzzle" until we obtain reasonable models of the control systems in yeast cells, frog eggs, and cultured mammalian cells.