Using Short-Term Tests to Predict Carcinogenic Activity in the Long-Term Bioassay

A method for classifying chemicals with respect to carcinogenic potential based on short-term test results is presented. The method utilizes the logistic regression model to translate results from short-term toxicity assays into predictions of the likelihood that a chemical will be carcinogenic if tested in a long-term bioassay. The proposed method differs from previous approaches in two ways. First, statistical confidence limits on probabilities of cancer rather than central estimates of those probabilities are used for classification. Second, the method does not classify all chemicals in a data base with respect to carcinogenic potential. Instead, it identifies chemicals with highest and lowest likelihood of testing positive for carcinogenicity in the bioassay. A subset of chemicals with intermediate likelihood of being positive remains unclassified, and will require further testing, perhaps in a long-term bioassay. Two data bases of binary short-term and long-term test results from the literature are used to illustrate and evaluate the proposed procedure. A cross-validation analysis of one of the data sets suggests that, for a sufficiently rich data base of chemicals, the development of a robust predictive system to replace the bioassay for some unknown chemicals is a realistic goal.     

Assembly and preliminary analysis of a genotoxicity data base for predicting carcinogens

With a view to developing methodologies for predicting the carcinogenicity of chemicals on the basis of the results of short-term assays and selecting highly predictive batteries of short-term tests, a data base was assembled. The present is a compilation of data extracted from the reports of Gene-Tox working groups, Salmonella mutagenicity data obtained from the U.S. National Toxicology Program and the Environmental Mutagen Information Center and results from BHK21 transformation assays.     

The carcinogenicity prediction and battery selection (CPBS) method: a Bayesian approach

Recently, a large number of relatively inexpensive in vitro short-term tests have been developed to help predict the carcinogenicity of chemicals. The carcinogenicity prediction and battery selection (CPBS) method utilizes the results of such short-term tests to screen for chemicals that are most likely to cause cancer. The method is an integrated approach for analyzing large, often sparsely filled, data bases containing short-term test results, which often have only marginal representation of known non-carcinogens. The CPBS method is developed for the purpose of (i) determining the reliability and predictive capability of individual and batteries of short-term tests, and (ii) developing a strategy for formulating and selecting optimally preferred batteries of short-term tests for screening chemicals for further testing. The term 'optimally preferred' connotes the best acceptable combination of tests in terms of trade-offs among the multiple attributes of each test and resulting battery (e.g., cost, sensitivity, specificity, etc). The CPBS method consists of 5 major tasks: (1) data consolidation, (2) parameter estimation, (3) predictivity calculation, (4) battery selection and (5) risk assessment. Although there is a great need for more research and improvement, the CPBS method at its present stage should add an important method to the maze of the thousands of new chemicals that are introduced into drugs, foods, consumer goods and to the environment every year. This method should also provide an enhanced identification procedure for classifying chemicals more accurately as suspected carcinogens or non-carcinogens.     

Quantification of the predictivity of some short-term assays for carcinogenicity in rodents.

A statistical procedure is described for assessing the predictive performance of short-term tests for carcinogenicity in which the actual number of chemicals tested is taken into consideration. The method is then applied to several widely used short-term assays.     

Deployment of short-term assays for the detection of carcinogens; genetic and molecular considerations

The deployment of short-term assays for the detection of carcinogens inevitably has to be based on the genetic alterations actually involved in carcinogenesis. This paper gives an overview of oncogene activation and other mutagenic events connected with cancer induction. It is emphasized that there are indications of DNA alterations in carcinogenicity, which are not in accordance with "conventional" mutations and mutation frequencies, as measured by short-term assays of point mutations, chromosome aberrations and numerical chromosome changes. This discrepancy between DNA alterations in carcinogenicity and the endpoints of short-term assays in current use include transpositions, insertion mutations, polygene mutations, gene amplifications and DNA methylations. Furthermore, tumourigenicity may imply an induction of a genetic instability, followed by a cascade of genetic alterations. The evaluation of short-term assays for carcinogenesis mostly involves two correlations that is, between mutation and animal cancer data on the one hand and between animal cancer data and human carcinogenicity on the other. It should be stressed that animal bioassays for cancer in general imply tests specifically for the property of chemicals to function as complete carcinogens, which may be a rather poor reflection of the actual situation in human populations. The primary aim of short-term mutagenicity assays is to provide evidence as to whether a compound can be expected to cause mutations in humans, and such evidence has to be considered seriously even against a background of negative cancer data. For the evaluation of data from short-term assays the massive amount of empirical data from different assays should be used and new computer systems in that direction can be expected to provide improved predictions of carcinogenicity.     

Knowledge-based battery design of short-term tests based on dose information

A construction of batteries of short-term tests (STTs) is described which is based on a classification of 73 chemicals in regard to their carcinogenicity. The 73 chemicals were studied within the U.S. National Toxicology Program (Ashby and Tennant, 1988). The batteries are validated using the classification of 35 additional chemicals. They are defined by logically structured combinations of rules. The single rules are defined by the z-scores of the logarithmic values of the limiting doses obtained from the 4 in vitro STTs used in the study by Ashby and Tennant. The limiting dose is defined as the lowest effective dose or the highest ineffective dose (Waters et al., 1987). The batteries are constructed by minimizing the number of disagreements with the classification by Ashby and Tennant. Compared with the results obtained from single STTs, 2 batteries of 3 STTs have higher concordances with the carcinogenicity data, namely 70% for the NTP data and 74-77% for the independent test data. In addition, a theoretical result shows that the proposed battery design, for a large enough learning set of chemicals, leads to results which are replicated with high probability on a large enough validation set. Based on the first results obtained with a limited number of chemicals it is concluded that the knowledge-based battery design is worth further development.     

The structural basis of the mutagenicity of chemicals in Salmonella typhimurium: the Gene-Tox data base

The CASE structure-activity methodology has been applied to a Gene-Tox derived Salmonella mutagenicity data base consisting of 808 chemicals. Based upon qualitative structural features, CASE identified 29 activating and 3 inactivating structural determinants which correctly predicted the probability of carcinogenicity of 93.7% of the known mutagens and non-mutagens in the data base (sensitivity = 0.998, and specificity = 0.704). Additionally, based upon a qualitative structure-activity analysis, CASE's performance was even better, leading to a sensitivity of 0.981 and a specificity of 1.000. Using the structural determinants identified in this data base, CASE gave excellent predictions of the mutagenicity of chemicals not included in the data base. The identified biophores and biophobes can also be used to investigate the structural basis of the mutagenicity of various chemical classes.     

Genetic toxicology: can we design predictive in vivo assays?

The present analysis examines the assumptions in, the perceptions and predictivity of and the need for short-term tests (STTs) for genotoxicity in light of recent findings that most noncarcinogens from the National Toxicology Program are genotoxic (i.e., positive in one or more in vitro STTs). Reasonable assumptions about the prevalence for carcinogens (1-10% of all chemicals), the sensitivity of these STTs (ca. 90% of all carcinogens are genotoxic) and their estimated "false positive" incidence (60-75%) imply that the majority of chemicals elicit genotoxic responses and, consequently, that most in vitro genotoxins are likely to be noncarcinogenic. Thus, either the usual treatment conditions used in these in vitro STTS are producing a large proportion of artifactual and meaningless positive results or else in vitro mutagenicity is too common a property of chemicals to serve as a useful predictor of carcinogenicity or other human risk. In contrast, the limited data base on in vivo STTs suggests that the current versions of these assays may have low sensitivity which appears unlikely to improve without dropping either their 'short-term' aspect or the rodent carcinogenicity benchmark. It is suggested that in vivo genotoxicity protocols be modified to take into consideration both the fundamentals of toxicology as well as the lessons learned from in vitro genetic toxicology. In the meantime, while in vivo assays are undergoing rigorous validation, genetic toxicology, as currently practiced, should not be a formal aspect of chemical or drug development on the grounds that it is incapable of providing realistic and reliable information on human risk. It is urged that data generated in new, unvalidated in vivo genotoxicity assays be exempted from the normal regulatory reporting requirements in order to encourage industry to participate in the laborious and expensive development of this next phase of genetic toxicology.     

Artificial intelligence and Bayesian decision theory in the prediction of chemical carcinogens

Two procedures for predicting the carcinogenicity of chemicals are described. One of these (CASE) is a self-learning artificial intelligence system that automatically recognizes activating and/or deactivating structural subunits of candidate chemicals and uses this to determine the probability that the test chemical is or is not a carcinogen. If the chemical is predicted to be carcinogen, CASE also projects its probable potency.

The second procedure (CPBS) uses Bayesian decision theory to predict the potential carcinogenicity of chemicals based upon the results of batteries of short-term assays. CPBS is useful even if the test results are mixed (i.e. both positive and negative responses are obtained in different genotoxic assays). CPBS can also be used to identify highly predictive as well as cost-effective batteries of assays.

For illustrative purposes the ability of CASE and CPBS to predict the carcinogenicity of a carcinogenic and a non-carcinogenic polycyclic aromatic hydrocarbon is shown. The potential for using the two methods in tandem to increase reliability and decrease cost is presented.     

Application of a cellular test battery in the decision point approach to carcinogen identification

The assessment of the potential carcinogenicity of a chemical requires a systematic approach taking into account various types of data. Important information on the DNA reactivity and other genetic effects of chemicals can be obtained from a battery of cellular tests. A battery is described which includes DNA repair in hepatocytes, mutagenesis in Salmonella typhimurium, mutagenesis, chromosome alterations, and transformation in mammalian cells. The interpretation of findings in this battery for the identification of potential carcinogenicity of chemicals is discussed.     

The unique role of rodents in the detection of possible human carcinogens and mutagens

Some of the probable reasons underlying the observation that not all chemicals shown to be genotoxic in vitro are capable of eliciting tumours in rodents or humans are discussed using appropriate examples. It is suggested that a substantial proportion of the resources currently available for conducting rodent carcinogenicity bioassays should be employed in the short-term evaluation in vivo of some of the many hundreds of chemicals recently defined as genotoxic in vitro, rather than in the protracted evaluation of a few chemicals, often of unknown activity in vitro, for carcinogenicity. A decision tree approach to the evaluation of chemicals for human mutagenic/carcinogenic potential is presented which is at variance with the construction and philosophy of many of the current legislative guidelines. The immediate need for the adoption of one of the available short-term in vivo liver assays, and/or the development of a short-term in vivo rodent assay capable of concomitantly monitoring different genetic end-points in a range of organs or tissues is emphasized.     

Predictive assay for rodent carcinogenicity using in vivo biochemical parameters: operational characteristics and complementarity.

111 chemicals of known rodent carcinogenicity (49 carcinogens, 62 noncarcinogens), including many promoters of carcinogenesis, nongenotoxic carcinogens, hepatocarcinogens, and halogenated hydrocarbons, were selected for study. The chemicals were administered by gavage in two dose levels to female Sprague-Dawley rats. The effects of these 111 chemicals on 4 biochemical assays (hepatic DNA damage by alkaline elution (DD), hepatic ornithine decarboxylase activity (ODC), serum alanine aminotransferase activity (ALT), and hepatic cytochrome P-450 content (P450)) were determined. Composite parameters are defined as follows: CP = [ODC and P450), CT = [ALT and ODC), and TS = [DD or CP or CT]. The operational characteristics of TS for predicting rodent cancer were sensitivity 55%, specificity 87%, positive predictivity 77%, negative predictivity 71%, and concordance 73%. For these chemicals, the 73% concordance of this study was superior to the concordance obtained from published data from other laboratories on the Ames test (53%), structural alerts (SA) (46%), chromosome aberrations in Chinese hamster ovary cells (ABS) (48%), cell mutation in mouse lymphoma 15178Y cells (MOLY) (52%), and sister-chromatid exchange in Chinese hamster ovary cells (SCE) (60%). The 4 in vivo biochemical assays were complementary to each other. The composite parameter TS also shows complementarity to all 5 other predictors of rodent cancer examined in this paper. For example, the Ames test alone has a concordance of only 53%. In combination with TS, the concordance is increased to 62% (Ames or TS) or to 63% (Ames and TS). For the 67 chemicals with data available for SA, the concordance for predicting rodent carcinogenicity was 47% (for SA alone), 54% (for SA or TS), and 66% (for SA and TS). These biochemical assays will be useful: (1) to predict rodent carcinogenicity per se, (2) to 'confirm' the results of short-term mutagenicity tests by the high specificity mode of the biochemical assays (the specificity and positive predictivity are both 100%), and (3) to be a component of future complementary batteries of tests for predicting rodent carcinogenicity.     

Definitive relationships among chemical structure, carcinogenicity and mutagenicity for 301 chemicals tested by the U.S. NTP.

An analysis is presented in which are evaluated correlations among chemical structure, mutagenicity to Salmonella, and carcinogenicity to rats and mice among 301 chemicals tested by the U.S. NTP. Overall, there was a high correlation between structural alerts to DNA reactivity and mutagenicity, but the correlation of either property with carcinogenicity was low. If rodent carcinogenicity is regarded as a singular property of chemicals, then neither structural alerts nor mutagenicity to Salmonella are effective in its prediction. Given this, the database was fragmented and new correlations sought between the derived sub-groups. First, the 301 chemicals were segregated into six broad chemical groupings. Second, the rodent cancer data were partially segregated by target tissue. Using the previously assigned structural alerts to DNA reactivity (electrophilicity), the chemicals were split into 154 alerting chemicals and 147 non-alerting chemicals. The alerting chemicals were split into three chemical groups; aromatic amino/nitro-types, alkylating agents and miscellaneous structurally-alerting groups. The non-alerting chemicals were subjectively split into three broad categories; non-alerting, non-alerting containing a non-reactive halogen group, and non-alerting chemical with minor concerns about a possible structural alert. The tumor data for all 301 chemicals are re-presented according to these six chemical groupings. The most significant findings to emerge from comparisons among these six groups of chemicals were as follows: (a) Most of the rodent carcinogens, including most of the 2-species and/or multiple site carcinogens, were among the structurally alerting chemicals. (b) Most of the structurally alerting chemicals were mutagenic; 84% of the carcinogens and 66% of the non-carcinogens. 100% of the 33 aromatic amino/nitro-type 2-species carcinogens were mutagenic. Thus, for structurally alerting chemicals, the Salmonella assay showed high sensitivity and low specificity (0.84 and 0.33, respectively). (c) Among the 147 non-alerting chemicals less than 5% were mutagenic, whether they were carcinogens or non-carcinogens (sensitivity 0.04).     

Invited contribution: An objective approach to the development of short-term tests predictive of carcinogenicity

The Carcinogenicity Prediction and Battery Selection procedure was developed to address two problems: (1) the identification of highly predictive, yet cost-effective, batteries of short-term tests and (2) the objective prediction of the potential carcinogenicity of chemicals based upon the results of short-term tests even when a mixture of positive and negative results is obtained. In the present report the usefulness of the Carcinogenicity Prediction and Battery Selection procedure is demonstrated using benzo[a]pyrene, benzoin and diethylstilbestrol as examples. In addition, its applicability in the analysis of all the possible outcomes of a battery is illustrated together with an analysis of the worth of additional testing.Abbreviations B[a]P benzo[a]pyrene - CASE Computer-Automated Structure Evaluation - CPBS Carcinogenicity Prediction and Battery Selection - DEHP diethylhexylphthalate - DES diethylstilbestrol - NTA nitrilotriacetate - TCDD 2,3,7,8-tetrachlorodibenzo-p-dioxin     

The SOS Chromotest, a colorimetric bacterial assay for genotoxins: validation study with 83 compounds

The SOS Chromotest is a simple bacterial colorimetric assay for genotoxicity. It is based on the measure of the induction of sfiA, a gene controlled by the general repressor of the SOS system in E. coli. Expression of sfiA is monitored by means of a gene fusion with lacZ, the structural gene for beta-galactosidase. We have examined 83 compounds of various chemical classes with the SOS Chromotest using a standard procedure. Comparison of the results with those obtained in the Mutatest (the Ames test) showed that most (90%) of the mutagenic compounds were also SOS inducers. For these compounds a quantitative correlation was observed between the mutagenic potency and the SOS-inducing potency (SOSIP). The case of the 10% remaining compounds giving conflicting results in the two tests is discussed. Sensitivity, specificity and accuracy for carcinogenicity prediction have been evaluated for the SOS Chromotest and the Mutatest using 73 chemicals for which carcinogenicity data were available. In spite of some differences, similar results were obtained in the two tests. The present data indicate that the SOS Chromotest has many practical advantages and may be used as a primary screening tool or as part of a battery of short-term tests for carcinogens.     

Computer-aided rodent carcinogenicity prediction

The potential of the computer program PASS (Prediction Activity Spectra for Substances) to predict rodent carcinogenicity for chemical compounds was studied. PASS predicts carcinogenicity of chemical compounds on the basis of their structural formula and of structure-activity relationship analysis of known carcinogens and non-carcinogens. The data on structures and experimental results of 2-year carcinogenicity assays for 412 chemicals from the NTP (National Toxicological Program) and 1190 chemicals from the CPDB (Carcinogenic Potency Database) were used in our study. The predictions take into consideration information about species and sex of animals. For evaluation of the predictive accuracy we used two procedures: leave-one-out cross-validation (LOO CV) and leave-20%-out cross-validation. In the last case we randomly divided the studied data set 20 times into two subsets. The data from the first subset, containing 80% of the compounds, were added to the PASS training set (which includes about 46,000 compounds with about 1500 biological activity types collected during the last 20 years to predict biological activity spectra), the second subset with 20% of the compounds was used as an evaluation set. The mean accuracy of prediction calculated by LOO CV is about 73% for NTP compounds in the 'equivocal' category of carcinogenic activity and 80% for NTP compounds in the 'evidence' category of carcinogenicity. The mean accuracy of prediction for the CPDB database is 89.9% calculated by LOO CV and 63.4% calculated by leave-20%-out cross-validation. Influence of incorporation of species and sex data on the accuracy of carcinogenicity prediction was also investigated. It was shown that the accuracy was increased only for data on male animals.     

Evaluation of the ability of a battery of three in vitro genotoxicity tests to discriminate rodent carcinogens and non-carcinogens II. Further analysis of mammalian cell results, relative predictivity and tumour profiles

One of the consequences of the low specificity of the in vitro mammalian cell genotoxicity assays reported in our previous paper [D. Kirkland, M. Aardema, L. Henderson, L. Muller, Evaluation of the ability of a battery of three in vitro genotoxicity tests to discriminate rodent carcinogens and non-carcinogens. I. Sensitivity, specificity and relative predictivity, Mutat. Res. 584 (2005) 1-256] is industry and regulatory agencies dealing with a large number of false-positive results during the safety assessment of new chemicals and drugs. Addressing positive results from in vitro genotoxicity assays to determine which are "false" requires extensive resources, including the conduct of additional animal studies. In order to reduce animal usage, and to conserve industry and regulatory agency resources, we thought it was important to raise the question as to whether the protocol requirements for a valid in vitro assay or the criteria for a positive result could be changed in order to increase specificity without a significant loss in sensitivity of these tests. We therefore analysed some results of the mouse lymphoma assay (MLA) and the chromosomal aberration (CA) test obtained for rodent carcinogens and non-carcinogens in more detail. For a number of chemicals that are positive only in either of these mammalian cell tests (i.e. negative in the Ames test) there was no correlation between rodent carcinogenicity and level of toxicity (we could not analyse this for the CA test as insufficient data were available in publications), magnitude of response or lowest effective positive concentration. On the basis of very limited in vitro and in vivo data, we could also find no correlation between the above parameters and formation of DNA adducts. Therefore, a change to the current criteria for required level of toxicity in the MLA, to limit positive calls to certain magnitudes of response, or to certain concentration ranges would not improve the specificity of the tests without significantly reducing the sensitivity. We also investigated a possible correlation between tumour profile (trans-species, trans-sex and multi-site versus single-species, single-sex and single-site) and pattern of genotoxicity results. Carcinogens showing the combination of trans-species, trans-sex and multi-site tumour profile were much more prevalent (70% more) in the group of chemicals giving positive results in all three in vitro assays than amongst those giving all negative results. However, single-species, single-sex, single-site carcinogens were not very prevalent even amongst those chemicals giving three negative results in vitro. Surprisingly, when mixed positive and negative results were compared, multi-site carcinogens were highly prevalent amongst chemicals giving only a single positive result in the battery of three in vitro tests. Finally we extended our relative predictivity (RP) calculations to combinations of positive and negative results in the genotoxicity battery. For two out of three tests positive, the RP for carcinogenicity was no higher than 1.0 and for 2/3 tests negative the RP for non-carcinogenicity was either zero (for Ames+MLA+MN) or 1.7 (for Ames+MLA+CA). Thus, all values were less than a meaningful RP of two, and indicate that it is not possible to predict outcome of the rodent carcinogenicity study when only 2/3 genotoxicity results are in agreement.     

The influence of the proportion of carcinogens on the cost-effectiveness of short-term tests

The cost-effectiveness of using short-term genotoxicity tests to screen unknown chemicals for carcinogenicity depends upon the inherent reliability of the tests (sensitivity, or fraction of carcinogens giving positive results, and specificity, or fraction of non-carcinogens giving negative results) and also upon the proportion of carcinogens in the population of chemicals to be screened. Individual tests may be combined into batteries to improve reliability; however, this requires decision rules to declare the overall result positive or negative. A framework for developing such rules based upon minimizing costs of false-positives and false-negatives was presented in a seminal paper by Lave and Omenn (1986, Nature (London), 324, 29-34). We have extended their work, which is based on logit analysis, to consider, using Bayes' theorem, the influence of the proportion of carcinogens upon the decision rules for declaring a battery result positive or negative. If the proportion of carcinogens is high (20% or greater), then the most effective tests are those with high sensitivity, and if the proportion of carcinogens is low, then the most effective tests are those with high specificity.     

An international validation study of a Bhas 42 cell transformation assay for the prediction of chemical carcinogenicity

The Bhas 42 cell transformation assay is a sensitive short-term system for predicting chemical carcinogenicity. Bhas 42 cells were established from BALB/c 3T3 cells by the transfection of v-Ha-ras gene and postulated to have acquired an initiated state in the two-stage carcinogenesis theory. The Bhas 42 cell transformation assay is capable of detecting both tumor-initiating and tumor-promoting activities of chemical carcinogens. The full assay protocol consists of two components, the initiation assay and the promotion assay, to detect the initiating activity and the promoting activity, respectively. An international study was carried out to validate this cell transformation assay in which six laboratories from three countries participated. Twelve coded chemicals were examined in total and each chemical was tested by three laboratories. In the initiation assay, concordant results were obtained by three laboratories for eight out of ten chemicals and in the promotion assay, concordant results were achieved for ten of twelve chemicals. The positive results were obtained in all three laboratories with the following chemicals: 2-acetylaminofluorene was positive in both initiation and promotion assays; dibenz[a,h]anthracene was positive in the initiation assay; sodium arsenite, lithocholic acid, cadmium chloride, mezerein and methapyrilene hydrochloride were positive in the promotion assay. o-Toluidin hydrochloride was positive in the both assays in two of the three laboratories. d-Mannitol, caffeine and l-ascorbic acid were negative in both assays in all the laboratories, and anthracene was negative in both assays in two of the three laboratories except one laboratory obtaining positive result in the promotion assay. Consequently, the Bhas 42 cell transformation assay correctly discriminated all six carcinogens and two tumor promoters from four non-carcinogens. Thus, the present study demonstrated that the Bhas 42 cell transformation assay is transferable and reproducible between laboratories and applicable to the prediction of chemical carcinogenicity. In addition, by comparison of the present results with intra-laboratory data previously published, within-laboratory reproducibility using the Bhas 42 cell transformation assay was also confirmed.