Towards a Rigorous Assessment of Systems Biology Models: The DREAM3 Challenges

Background

Systems biology has embraced computational modeling in response to the quantitative nature and increasing scale of contemporary data sets. The onslaught of data is accelerating as molecular profiling technology evolves. The Dialogue for Reverse Engineering Assessments and Methods (DREAM) is a community effort to catalyze discussion about the design, application, and assessment of systems biology models through annual reverse-engineering challenges.

Methodology and Principal Findings

We describe our assessments of the four challenges associated with the third DREAM conference which came to be known as the DREAM3 challenges: signaling cascade identification, signaling response prediction, gene expression prediction, and the DREAM3 in silico network challenge. The challenges, based on anonymized data sets, tested participants in network inference and prediction of measurements. Forty teams submitted 413 predicted networks and measurement test sets. Overall, a handful of best-performer teams were identified, while a majority of teams made predictions that were equivalent to random. Counterintuitively, combining the predictions of multiple teams (including the weaker teams) can in some cases improve predictive power beyond that of any single method.

Conclusions

DREAM provides valuable feedback to practitioners of systems biology modeling. Lessons learned from the predictions of the community provide much-needed context for interpreting claims of efficacy of algorithms described in the scientific literature.     

Guest Editorial: UNESCO Global Network for Molecular and Cell Biology (MCBN)

World Journal of Microbiology and Biotechnology -     

A Hypothesis Test for Equality of Bayesian Network Models

Bayesian network models are commonly used to model gene expression data. Some applications require a comparison of the network structure of a set of genes between varying phenotypes. In principle, separately fit models can be directly compared, but it is difficult to assign statistical significance to any observed differences. There would therefore be an advantage to the development of a rigorous hypothesis test for homogeneity of network structure. In this paper, a generalized likelihood ratio test based on Bayesian network models is developed, with significance level estimated using permutation replications. In order to be computationally feasible, a number of algorithms are introduced. First, a method for approximating multivariate distributions due to Chow and Liu (1968) is adapted, permitting the polynomial-time calculation of a maximum likelihood Bayesian network with maximum indegree of one. Second, sequential testing principles are applied to the permutation test, allowing significant reduction of computation time while preserving reported error rates used in multiple testing. The method is applied to gene-set analysis, using two sets of experimental data, and some advantage to a pathway modelling approach to this problem is reported.     

Grand Molecular Dynamics: A Method for Open Systems

We present a new molecular dynamics method for studying the dynamics of open systems. The method couples a classical system to a chemical potential reservior. In the formulation, following the extended system dynamics approach, we introduce a variable, v to represent the coupling to the chemical potential reservoir. The new variable governs the dynamics of the variation of number of particles in the system. The number of particles is determined by taking the integer part of v. The fractional part of the new variable is used to scale the potential energy and the kinetic energy of an additional particle: i.e., we introduce a fractional particle. We give the ansatz Lagrangians and equations of motion for both the isothermal and the adiabatic forms of grand molecular dynamics. The averages calculated over the trajectories generated by these equations of motion represent the classical grand canonical ensemble (μVT) and the constant chemical potential adiabatic ensemble (μVL) averages, respectively. The microcanonical phase space densities of the adiabatic and isothermal forms the molecular dynamics method are shown to be equivalent to adiabatic constant chemical potential ensemble, and grand canonical ensemble partition functions. We also discuss the extension to multi-component systems, molecular fluids, ionic solutions and the problems and solutions associated with the implementation of the method. The statistical expressions for thermodynamic functions such as specific heat; adiabatic bulk modulus, Grüneissen parameter and number fluctuations are derived. These expressions are used to analyse trajectories of constant chemical potential systems.     

A Discussion of Molecular Biology Methods for Protein Engineering

A number of molecular biology techniques are available to generate variants from a particular start gene for eventual protein expression. We discuss the basic principles of these methods in a repertoire that may be used to achieve the elemental steps in protein engineering. These include site-directed, deletion and insertion mutagenesis. We provide detailed case studies, drawn from our own experiences, packaged together with conceptual discussions and include an analysis of the techniques presented with regards to their uses in protein engineering.