Estimating an oncogenetic tree when false negatives and positives are present

Human solid tumors are believed to be caused by a sequence of genetic abnormalities arising in the tumor cells. The understanding of these sequences is extremely important for improving cancer treatment. Models for the occurrence of the abnormalities include linear structure and a recently proposed tree-based structure. In this paper we extend the pure oncogenetic tree model by introducing false positive and false negative observations. We state conditions sufficient for the reconstruction of the generating tree. As an example we analyze a comparative genomic hybridization data set and show that addition of the error model significantly improves the ability of the model to describe the data.     

Screening for possible human carcinogens and mutagens. False positives, false negatives: statistical implications

A screening method aimed at identifying potential human carcinogens using either animal cancer bioassays or short-term genotoxic assays has 4 possible results: true positive, true negative, false positive and false negative. Such a categorisation is superficially similar to the results of hypothesis testing in a statistical analysis. In this latter case the false positive rate is determined by the significance level of the test and the false negative rate by the statistical power of the test. Although the two types of categorisation appear somewhat similar, different statistical issues are involved in their interpretation. Statistical methods appropriate for the analysis of the results of a series of assays include the use of Bayes' theorem and multivariate methods such as clustering techniques for the selection of batteries of short-term test capable of a better prediction of potential carcinogens. The conclusions drawn from such studies are dependent upon the estimates of values of sensitivity and specificity used, the choice of statistical method and the nature of the data set. The statistical issues resulting from the analysis of specific genotoxicity experiments involve the choice of suitable experimental designs and appropriate analyses together with the relationship of statistical significance to biological importance. The purpose of statistical analysis should increasingly be to estimate and explore effects rather than for formal hypothesis testing.     

Approaches to reduce false positives and false negatives in the analysis of microarray data: applications in type 1 diabetes research

Background

As studies of molecular biology system attempt to achieve a comprehensive understanding of a particular system, Type 1 errors may be a significant problem. However, few investigators are inclined to accept the increase in Type 2 errors (false positives) that may result when less stringent statistical cut-off values are used. To address this dilemma, we developed an analysis strategy that used a stringent statistical analysis to create a list of differentially expressed genes that served as "bait" to "fish out" other genes with similar patterns of expression.

Results

Comparing two strains of mice (NOD and C57Bl/6), we identified 93 genes with statistically significant differences in their patterns of expression. Hierarchical clustering identified an additional 39 genes with similar patterns of expression differences between the two strains. Pathway analysis was then employed: 1) identify the central genes and define biological processes that may be regulated by the genes identified, and 2) identify genes on the lists that could not be connected to each other in pathways (potential false positives). For networks created by both gene lists, the most connected (central) genes were interferon gamma (IFN-γ) and tumor necrosis factor alpha (TNF-α). These two cytokines are relevant to the biological differences between the two strains of mice. Furthermore, the network created by the list of 39 genes also suggested other biological differences between the strains.

Conclusion

Taken together, these data demonstrate how stringent statistical analysis, combined with hierarchical clustering and pathway analysis may offer deeper insight into the biological processes reflected from a set of expression array data. This approach allows us to 'recapture" false negative genes that otherwise would have been missed by the statistical analysis.

     

Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values

MOTIVATION: The occurrence of false positives and false negatives in a microarray analysis could be easily estimated if the distribution of p-values were approximated and then expressed as a mixture of null and alternative densities. Essentially any distribution of p-values can be expressed as such a mixture by extracting a uniform density from it. RESULTS: The occurrence of false positives and false negatives in a microarray analysis could be easily estimated if the distribution of p-values were approximated and then expressed as a mixture of null and alternative densities. Essentially any distribution of p-values can be expressed as such a mixture by extracting a uniform density from it. AVAILABILITY: An S-plus function library is available from http://www.stjuderesearch.org/statistics.     

On minimizing assignment errors and the trade‐off between false positives and negatives in parentage analysis

Genetic parentage analyses provide a practical means with which to identify parent–offspring relationships in the wild. In Harrison et al.'s study (2013a), we compare three methods of parentage analysis and showed that the number and diversity of microsatellite loci were the most important factors defining the accuracy of assignments. Our simulations revealed that an exclusion‐Bayes theorem method was more susceptible to false‐positive and false‐negative assignments than other methods tested. Here, we analyse and discuss the trade‐off between type I and type II errors in parentage analyses. We show that controlling for false‐positive assignments, without reporting type II errors, can be misleading. Our findings illustrate the need to estimate and report both the rate of false‐positive and false‐negative assignments in parentage analyses.     

A new method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays

The z-score method and its variants for testing mean difference are commonly used for hit selection in high-throughput screening (HTS) assays. Strictly standardized mean difference (SSMD) offers a way to measure and classify the short interfering RNA (siRNA) effects. In this article, based on SSMD, the authors propose a new testing method for hit selection in RNA interference (RNAi) HTS assays. This SSMD-based method allows the differentiation between siRNAs with large and small effects on the assay output and maintains flexible and balanced control of both the false-negative rate, in which the siRNAs with strong effects are not selected as hits, and the restricted false-positive rate, in which the siRNAs with weak or no effects are selected as hits. This method directly addresses the size of siRNA effects represented by the strength of difference between an siRNA and a negative reference, whereas the classic z-score method and t-test of testing no mean difference address whether the mean of an siRNA is exactly the same as the mean of a negative reference. This method can readily control the false-negative rate, whereas it is nontrivial for the classic z-score method and t-test to control the false-negative rate. Therefore, theoretically, the SSMD-based method offers better control of the sizes of siRNA effects and the associated false-positive and false-negative rates than the commonly used z-score method and t-test for hit selection in HTS assays. The SSMD-based method should generally be applicable to any assay in which the end point is a difference in signal compared to a reference sample, including those for RNAi, receptor, enzyme, and cellular function.     

A method with flexible and balanced control of false negatives and false positives for hit selection in RNA interference high-throughput screening assays: a statistical terminology

Zhang suggests a new method that is flexible and controls the balance between false negatives and false positives for hit selection in RNA high-throughput screening assays. The author shows that the same decision rules and balances can be expressed by familiar statistical terms such as type I error and power and hence connects the new method to known statistical tools. (Journal of Biomolecular Screening 2008:309-311).     

Tentacle probes: eliminating false positives without sacrificing sensitivity

The majority of efforts to increase specificity or sensitivity in biosensors result in trade-offs with little to no gain in overall accuracy. This is because a biosensor cannot be more accurate than the affinity interaction it is based on. Accordingly, we have developed a new class of reagents based on mathematical principles of cooperativity to enhance the accuracy of the affinity interaction. Tentacle probes (TPs) have a hairpin structure similar to molecular beacons (MBs) for enhanced specificity, but are modified by the addition of a capture probe for increased kinetics and affinity. They produce kinetic rate constants up to 200-fold faster than MB with corresponding stem strengths. Concentration-independent specificity was observed with no false positives at up to 1 mM concentrations of variant analyte. In contrast, MBs were concentration dependent and experienced false positives above 3.88 muM of variant analyte. The fast kinetics of this label-free reagent may prove important for extraction efficiency, hence sensitivity and detection time, in microfluidic assays. The concentration-independent specificity of TPs may prove extremely useful in assays where starting concentrations and purities are unknown as would be the case in bioterror or clinical point of care diagnostics.     

Minimizing false positives in kinase virtual screens

In spite of recent improvements in docking and scoring methods, high false-positive rates remain a common issue in structure-based virtual screening. In this study, the distinctive features of false positives in kinase virtual screens were investigated. A series of retrospective virtual screens on kinase targets was performed on specifically designed test sets, each combining true ligands and experimentally confirmed inactive compounds. A systematic analysis of the docking poses generated for the top-ranking compounds highlighted key aspects differentiating true hits from false positives. The most recurring feature in the poses of false positives was the absence of certain key interactions known to be required for kinase binding. A systematic analysis of 444 crystal structures of ligand-bound kinases showed that at least two hydrogen bonds between the ligand and the backbone protein atoms in the kinase hinge region are present in 90% of the complexes, with very little variability across targets. Closer inspection showed that when the two hydrogen bonds are present, one of three preferred hinge-binding motifs is involved in 96.5% of the cases. Less than 10% of the false positives satisfied these two criteria in the minimized docking poses generated by our standard protocol. Ligand conformational artifacts were also shown to contribute to the occurrence of false positives in a number of cases. Application of this knowledge in the form of docking constraints and post-processing filters provided consistent improvements in virtual screening performance on all systems. The false-positive rates were significantly reduced and the enrichment factors increased by an average of twofold. On the basis of these results, a generalized two-step protocol for virtual screening on kinase targets is suggested.     

Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives

In a wide range of contexts, including predator avoidance, medical decision-making and security screening, decision accuracy is fundamentally constrained by the trade-off between true and false positives. Increased true positives are possible only at the cost of increased false positives; conversely, decreased false positives are associated with decreased true positives. We use an integrated theoretical and experimental approach to show that a group of decision-makers can overcome this basic limitation. Using a mathematical model, we show that a simple quorum decision rule enables individuals in groups to simultaneously increase true positives and decrease false positives. The results from a predator-detection experiment that we performed with humans are in line with these predictions: (i) after observing the choices of the other group members, individuals both increase true positives and decrease false positives, (ii) this effect gets stronger as group size increases, (iii) individuals use a quorum threshold set between the average true- and false-positive rates of the other group members, and (iv) individuals adjust their quorum adaptively to the performance of the group. Our results have broad implications for our understanding of the ecology and evolution of group-living animals and lend themselves for applications in the human domain such as the design of improved screening methods in medical, forensic, security and business applications.     

Approaches to detecting false positives in yeast two-hybrid systems

While many novel associations predicted by two-hybrid library screens reflect actual biological associations of two proteins in vivo, at times the functional co-relevance of two proteins scored as interacting in the two-hybrid system is unlikely. The reason for this positive score remains obscure, which leads to designating such clones as false positives. After investigating the effect of over-expressing a series of putative false positives in yeast, we determined that expression of some of these clones induces an array of biological effects in yeast, including altered growth rate and cell permeability, that bias perceived activity of LacZ reporters. Based on these observations, we identify four simple strategies that can assist in determining whether a protein is likely to have been selected in a two-hybrid screen because of indirect metabolic effects.     

"Patchy-tachy" leads to false positives for recombination

Indirect tests have detected recombination in mitochondrial DNA (mtDNA) from many animal lineages, including mammals. However, it is possible that features of the molecular evolutionary process without recombination could be incorrectly inferred by indirect tests as being due to recombination. We have identified one such example, which we call "patchy-tachy" (PT), where different partitions of sequences evolve at different rates, that leads to an excess of false positives for recombination inferred by indirect tests. To explore this phenomena, we characterized the false positive rates of six widely used indirect tests for recombination using simulations of general models for mtDNA evolution with PT but without recombination. All tests produced 30-99% false positives for recombination, although the conditions that produced the maximal level of false positives differed between the tests. To evaluate the degree to which conditions that exacerbate false positives are found in published sequence data, we turned to 20 animal mtDNA data sets in which recombination is suggested by indirect tests. Using a model where different regions of the sequences were free to evolve at different rates in different lineages, we demonstrated that PT is prevalent in many data sets in which recombination was previously inferred using indirect tests. Taken together, our results argue that PT without recombination is a viable alternative explanation for detection of widespread recombination in animal mtDNA using indirect tests.     

A note on controlling the number of false positives

Summary :   Lehmann and Romano (2005, Annals of Statistics 33, 1138–1154) discuss a Bonferroni-type procedure that bounds the probability that the number of false positives is larger than a specified number. We note that this procedure will have poor power as compared to a multivariate permutation test type procedure when the experimental design accommodates a permutation test. An example is given involving gene expression microarray data of breast cancer tumors.