Statistical evaluation of the Predictive Toxicology Challenge 2000-2001

MOTIVATION: The development of in silico models to predict chemical carcinogenesis from molecular structure would help greatly to prevent environmentally caused cancers. The Predictive Toxicology Challenge (PTC) competition was organized to test the state-of-the-art in applying machine learning to form such predictive models. RESULTS: Fourteen machine learning groups generated 111 models. The use of Receiver Operating Characteristic (ROC) space allowed the models to be uniformly compared regardless of the error cost function. We developed a statistical method to test if a model performs significantly better than random in ROC space. Using this test as criteria five models performed better than random guessing at a significance level p of 0.05 (not corrected for multiple testing). Statistically the best predictor was the Viniti model for female mice, with p value below 0.002. The toxicologically most interesting models were Leuven2 for male mice, and Kwansei for female rats. These models performed well in the statistical analysis and they are in the middle of ROC space, i.e. distant from extreme cost assumptions. These predictive models were also independently judged by domain experts to be among the three most interesting, and are believed to include a small but significant amount of empirically learned toxicological knowledge. AVAILABILITY: PTC details and data can be found at: http://www.predictive-toxicology.org/ptc/.     

A survey of the Predictive Toxicology Challenge 2000-2001

MOTIVATION: The Predictive Toxicology Challenge (PTC) was initiated to stimulate the development of advanced techniques for predictive toxicology models. The goal of this challenge was to compare different approaches for the prediction of rodent carcinogenicity, based on the experimental results of the US National Toxicology Program (NTP). RESULTS: 111 sets of predictions for 185 compounds have been evaluated on quantitative and qualitative scales to select the most predictive models and those with the highest toxicological relevance. The accuracy of the submitted predictions was between 25 and 79 %. An evaluation of the most accurate models by toxicological experts showed, that it is still hard for domain experts to interpret the submitted models and to put them into relation with toxicological knowledge. AVAILABILITY: PTC details and data can be found at: http://www.predictive-toxicology.org/ptc/.     

Predictive ecology: systems approaches

The world is experiencing significant, largely anthropogenically induced, environmental change. This will impact on the biological world and we need to be able to forecast its effects. In order to produce such forecasts, ecology needs to become more predictive--to develop the ability to understand how ecological systems will behave in future, changed, conditions. Further development of process-based models is required to allow such predictions to be made. Critical to the development of such models will be achieving a balance between the brute-force approach that naively attempts to include everything, and over simplification that throws out important heterogeneities at various levels. Central to this will be the recognition that individuals are the elementary particles of all ecological systems. As such it will be necessary to understand the effect of evolution on ecological systems, particularly when exposed to environmental change. However, insights from evolutionary biology will help the development of models even when data may be sparse. Process-based models are more common, and are used for forecasting, in other disciplines, e.g. climatology and molecular systems biology. Tools and techniques developed in these endeavours can be appropriated into ecological modelling, but it will also be necessary to develop the science of ecoinformatics along with approaches specific to ecological problems. The impetus for this effort should come from the demand coming from society to understand the effects of environmental change on the world and what might be performed to mitigate or adapt to them.     

Predictive motifs derived from cytosine methyltransferases.

Thirteen bacterial DNA methyltransferases that catalyze the formation of 5-methylcytosine within specific DNA sequences possess related structures. Similar building blocks (motifs), containing invariant positions, can be found in the same order in all thirteen sequences. Five of these blocks are highly conserved while a further five contain weaker similarities. One block, which has the most invariant residues, contains the proline-cysteine dipeptide of the proposed catalytic site. A region in the second half of each sequence is unusually variable both in length and sequence composition. Those methyltransferases that exhibit significant homology in this region share common specificity in DNA recognition. The five highly conserved motifs can be used to discriminate the known 5-methylcytosine forming methyltransferases from all other methyltransferases of known sequence, and from all other identified proteins in the PIR, GenBank and EMBL databases. These five motifs occur in a mammalian methyltransferase responsible for the formation of 5-methylcytosine within CG dinucleotides. By searching the unidentified open reading frames present in the GenBank and EMBL databases, two potential 5-methylcytosine forming methyltransferases have been found.     

Predictive algorithms for adjuvant therapy: TransATAC

Estrogen receptor (ER) positive primary breast cancers have a wide range of clinical outcomes. Prediction of the likely course of the disease aids treatment decision-making. In the translational arm of the ATAC (anastrozole or tamoxifen alone or combined) trial (TransATAC) we have assessed individual and multiparameter biomarkers for their prediction of overall and distant recurrence. None of the biomarkers identified differential benefit for anastrozole versus tamoxifen. Each of ER, PgR, HER2 and Ki67 was associated with risk of recurrence. A combination of these to create a single predictor IHC4 was as informative as the 21-gene recurrence score (RS). Integration of each of these molecular profiles with classical clinicopathologic variables provided the most accurate prediction of outcome.     

Predictive models in aerobiology: data transformation

This paper attempts to evaluate the effect of mathematical transformations of pollen and meteorogical data used in aerobiological forecasting models. Stepwise multiple regression equations were developed in order to facilitate short term forecasts during the pre-peak period. The daily mean pollen data (x _i) expressed as number of pollen grains per cubic metre of air were used directly and transformed into different scales: log(x _i + 1), ln((x ₁1000/Σp) + 1) and √x _i, where Σp is the sum of the daily mean values throughout the season. Thirteen meteorological parameters and the variable time were used as forecasting variables. The most reliable forecasts were obtained with data transformed by ‘square root’ and with untransformed data. Based on the results obtained, we recommend that the data be transformed by means of the square root if they do not show a normal distribution and that non-linear statistics be used in this kind of study.     

Predictive vs. Empiric Assessment of Schistosomiasis: Implications for Treatment Projections in Ghana

BackgroundMapping the distribution of schistosomiasis is essential to determine where control programs should operate, but because it is impractical to assess infection prevalence in every potentially endemic community, model-based geostatistics (MBG) is increasingly being used to predict prevalence and determine intervention strategies.Conclusions/SignificanceUsing the current predictive map for Ghana as a spatial decision support tool by aggregating prevalence estimates to the district level was clearly not adequate for guiding the national program, but the alternative of assessing each school in potentially endemic areas of Ghana or elsewhere is not at all feasible; modelling must be a tool complementary to empiric assessments. Thus for practical usefulness, predictive risk mapping should not be thought of as a one-time exercise but must, as in the current study, be an iterative process that incorporates empiric testing and model refining to create updated versions that meet the needs of disease control operational managers.     

Photosynthesis: 1900-1930.

During the second half of the 19th century Julius von Sachs established the main principles of the photosynthetic production of sugars. From then, a growing number of biochemists and physiologists attended to the process, that appeared like a "black box", in order to detect what came in and what went out of it. The English group of Frederick Blackman gave a remarkable contribution in individuating the close connection between temperature, light and CO2 concentration. Later, the great importance of light was stressed by Otto Warburg, who evaluated the radiant energy necessary to the process in terms of quantum theory. The biochemical mechanism of photosynthesis was interpreted by the main European schools on the basis of Adolf Baeyer's suggestion which posed formaldehyde as the core of the process. Formaldehyde's theory hold engaged the biochemists for about fifty years although some voices rose up against it. However, nobody could put forward more coherent theories until the 1940s, when Sam Ruben and Martin Kamen individuated the cyclic pattern of the process. Ultimately, the first thirty years of the 20th century must be seen as a preliminary stage studded with light and shade even if, in spite of controversial trends, several findings of remarkable interest became to disclose that "black box" as we know today chlorophyll photosynthesis.