Theoretical and empirical power of regression and maximum-likelihood methods to map quantitative trait loci in general pedigrees

Both theoretical calculations and simulation studies have been used to compare and contrast the statistical power of methods for mapping quantitative trait loci (QTLs) in simple and complex pedigrees. A widely used approach in such studies is to derive or simulate the expected mean test statistic under the alternative hypothesis of a segregating QTL and to equate a larger mean test statistic with larger power. In the present study, we show that, even when the test statistic under the null hypothesis of no linkage follows a known asymptotic distribution (the standard being chi(2)), it cannot be assumed that the distribution under the alternative hypothesis is noncentral chi(2). Hence, mean test statistics cannot be used to indicate power differences, and a comparison between methods that are based on simulated average test statistics may lead to the wrong conclusion. We illustrate this important finding, through simulations and analytical derivations, for a recently proposed new regression method for the analysis of general pedigrees to map quantitative trait loci. We show that this regression method is not necessarily more powerful nor computationally more efficient than a maximum-likelihood variance-component approach. We advocate the use of empirical power to compare trait-mapping methods.     

Testing against Ordered Alternatives in One-way Layout

In this paper we investigate a simple method of modifying well known nonparametric tests for the several samples location problem to yield a class of test statistics suitable for ordered alternatives. Optimum member of the class is identified in each case and its efficacy obtained. The method is applied to five statistics.     

Multipoint linkage detection in the presence of heterogeneity

Linkage heterogeneity is common for complex diseases. It is well known that loss of statistical power for detecting linkage will result if one assumes complete homogeneity in the presence of linkage heterogeneity. To this end, Smith (1963, Annals of Human Genetics 27, 175-182) proposed an admixture model to account for linkage heterogeneity. It is well known that for this model, the conventional chi-squared approximation to the likelihood ratio test for no linkage does not apply even when the sample size is large. By dealing with nuclear families and one marker at a time for genetic diseases with simple modes of inheritance, score-based test statistics (Liang and Rathouz, 1999, Biometrics 55, 65-74) and likelihood-ratio-based test statistics (Lemdani and Pons, 1995, Biometrics 51, 1033-1041) have been proposed which have a simple large-sample distribution under the null hypothesis of linkage. In this paper, we extend their work to more practical situations that include information from multiple markers and multi-generational pedigrees while allowing for a class of general genetic models. Three different approaches are proposed to eliminate the nuisance parameters in these test statistics. We show that all three approaches lead to the same asymptotic distribution under the null hypothesis of no linkage. Simulation results show that the proposed test statistics have adequate power to detect linkage and that the performances of these two classes of test statistics are quite comparable. We have applied the proposed method to a family study of asthma (Barnes et al., 1996), in which the score-based test shows evidence of linkage with p-value <0.0001 in the region of interest on chromosome 12. Additionally, we have implemented this score-based test within the frequently used computer package GENEHUNTER.     

Beta diversity reconsidered

A simple analytical procedure to test for differences in beta diversity among sets of plots has been recently presented. Here, we describe an improved randomization procedure that replaces the one previously proposed. This procedure consists of shuffling within-group dissimilarities among groups and disregarding between-group dissimilarities. By repeating this operation many times, a distribution of the test statistics under the null hypothesis of no differences in the mean plot-to-plot dissimilarities within groups is obtained. This procedure ensures that the correct null model is selected. To describe this new procedure, we used plant and water beetle (Coleoptera) data collected from 45 permanent ponds. Beta diversity was compared between plant and water beetle (Coleoptera) assemblages.     

Test of Independence Against a Class of Ordered Alternatives in an R × C Contingency Table

Test procedures are developed for testing the hypotheses of independence in a two-way contingency table against a class of ordered alternatives defined in terms of pooled crossproduct ratios. The alternatives hypotheses of “positive dependence” defines new notions of dependence. The distribution of the test statistics is obtained. Admissibility of the test is deduced and power comparison with some known test procedures are given. A numerical example is presented to illustrate the techniques developed.     

A statistically valid alternative to the TDT

OBJECTIVE: To present an alternative linkage test to the transmission/disequilibrium test (TDT) which is conservative under the null hypothesis and generally more powerful under alternatives. METHODS: The exact distribution of the TDT is examined under both the null hypothesis and relevant alternatives. The TDT is rewritten in an alternate form based on the contributions from each of the three relevant parental mating types. This makes it possible to show that a particular term in the estimate is an exact tie and thus to rewrite the estimate without this term and to replace the multinomial 'variance estimate' of Spielman et al. [Am J Hum Genet 1993;52:506-516] by the binomial variance. RESULTS: The resulting test is shown to be a stratified McNemar test (SMN). The significance level attained by the SMN is shown to be conservative when compared to the asymptotic chi(2) distribution, while the TDT often exceeds the nominal level alpha. Under alternatives, the proposed test is shown to be typically more powerful than the TDT. CONCLUSION: The properties of the TDT as a statistical test have never been fully investigated. The proposed test replaces the heuristically motivated TDT by a formally derived test, which is also computationally simple.     

Weighted Logrank Testing Procedures for the Simple Tree Alternatives

Weighted logrank testing procedures for comparing r treatments with a control when some of the data are randomly censored are discussed. Four kinds of test statistics for the simple tree alternatives are considered. The weighted logrank statistics based on pairwise ranking scheme is proposed and the covariances of the test statistics are explicitly obtained. This class of test statistics can be viewed as the general statistics of constructing the test procedures for various order restricted alternatives by modifying weights. Four kinds of weighted logrank tests are illustrated with an example. Simulation studies are performed to compare the sizes and the powers of the considered tests with the other.     

Assessing the adequacy of variance function in heteroscedastic regression models

Heteroscedastic data arise in many applications. In heteroscedastic regression analysis, the variance is often modeled as a parametric function of the covariates or the regression mean. We propose a kernel-smoothing type nonparametric test for checking the adequacy of a given parametric variance structure. The test does not need to specify a parametric distribution for the random errors. It is shown that the test statistic has an asymptotical normal distribution under the null hypothesis and is powerful against a large class of alternatives. We suggest a simple bootstrap algorithm to approximate the distribution of the test statistic in finite sample size. Numerical simulations demonstrate the satisfactory performance of the proposed test. We also illustrate the application by the analysis of a radioimmunoassay data set.     

A mixture model-based strategy for selecting sets of genes in multiclass response microarray experiments

MOTIVATION: Multiclass response (MCR) experiments are those in which there are more than two classes to be compared. In these experiments, though the null hypothesis is simple, there are typically many patterns of gene expression changes across the different classes that led to complex alternatives. In this paper, we propose a new strategy for selecting genes in MCR that is based on a flexible mixture model for the marginal distribution of a modified F-statistic. Using this model, false positive and negative discovery rates can be estimated and combined to produce a rule for selecting a subset of genes. Moreover, the method proposed allows calculation of these rates for any predefined subset of genes. RESULTS: We illustrate the performance our approach using simulated datasets and a real breast cancer microarray dataset. In this latter study, we investigate predefined subset of genes and point out interesting differences between three distinct biological pathways. AVAILABILITY: http://www.bgx.org.uk/software.html     

A linear rank test for use when the main interest is in differences in cure rates

For diseases with a positive probability of being cured, a family of alternatives to the null hypothesis of equality of survival distributions is introduced, which is designed to focus power against alternatives with differences in cure rates. The optimal linear rank test for this alternative is derived, and found to be substantially more efficient than the log-rank test for this alternative when cure rates are less than 50%, while there is little difference between the tests if the cure rates are 50% or greater. The simple test based on the difference of Kaplan-Meier estimates of the proportion cured is also examined, and found to be fully efficient for this alternative with no censoring, while its efficiency rapidly drops as censoring is increased. The new test is not a pure test of equality of cure rates when the data are censored, but rather is a test of equality of survival distributions that focuses power against late differences in the survival curves.     

An efficient Monte Carlo approach to assessing statistical significance in genomic studies

MOTIVATION: Multiple hypothesis testing is a common problem in genome research, particularly in microarray experiments and genomewide association studies. Failure to account for the effects of multiple comparisons would result in an abundance of false positive results. The Bonferroni correction and Holm's step-down procedure are overly conservative, whereas the permutation test is time-consuming and is restricted to simple problems. RESULTS: We developed an efficient Monte Carlo approach to approximating the joint distribution of the test statistics along the genome. We then used the Monte Carlo distribution to evaluate the commonly used criteria for error control, such as familywise error rates and positive false discovery rates. This approach is applicable to any data structures and test statistics. Applications to simulated and real data demonstrate that the proposed approach provides accurate error control, and can be substantially more powerful than the Bonferroni and Holm methods, especially when the test statistics are highly correlated.     

Statistical hypothesis testing in intraspecific phylogeography: nested clade phylogeographical analysis vs. approximate Bayesian computation

Nested clade phylogeographical analysis (NCPA) and approximate Bayesian computation (ABC) have been used to test phylogeographical hypotheses. Multilocus NCPA tests null hypotheses, whereas ABC discriminates among a finite set of alternatives. The interpretive criteria of NCPA are explicit and allow complex models to be built from simple components. The interpretive criteria of ABC are ad hoc and require the specification of a complete phylogeographical model. The conclusions from ABC are often influenced by implicit assumptions arising from the many parameters needed to specify a complex model. These complex models confound many assumptions so that biological interpretations are difficult. Sampling error is accounted for in NCPA, but ABC ignores important sources of sampling error that creates pseudo-statistical power. NCPA generates the full sampling distribution of its statistics, but ABC only yields local probabilities, which in turn make it impossible to distinguish between a good fitting model, a non-informative model, and an over-determined model. Both NCPA and ABC use approximations, but convergences of the approximations used in NCPA are well defined whereas those in ABC are not. NCPA can analyse a large number of locations, but ABC cannot. Finally, the dimensionality of tested hypothesis is known in NCPA, but not for ABC. As a consequence, the 'probabilities' generated by ABC are not true probabilities and are statistically non-interpretable. Accordingly, ABC should not be used for hypothesis testing, but simulation approaches are valuable when used in conjunction with NCPA or other methods that do not rely on highly parameterized models.     

Lattice models, packing density, and Boltzmann-like distribution of cavities in proteins

A model reproducing the experimental Boltzmann-like distribution of empty cavity sizes in proteins is introduced. Proteins are represented by lattices of different dimensionalities, corresponding to different numbers of nearest neighbor contacts. Small cavities emerge and join into larger ones in a random process that can be related to random mutations. Simulations of cavity creation are performed under the constraint of a limiting total packing density. Cavities sufficiently large (20 A(3) or more), that they might accommodate at least one additional methyl group produced by a mutation, are counted and compared to the distribution of cavities according to their sizes from protein statistics. The distributions calculated with this very simple model within a realistic range of packing densities are in good agreement with the empirical cavity distribution. The results suggest that the Boltzmann-like distribution of cavities in proteins might be affected by a mechanism controlled by limiting packing density and maximum allowed protein destabilization. This supports an earlier suggestion that the agreement between the free energies of cavity formation from the mutational experiments and from the statistics of the empty cavity distribution in X-ray protein structures is nonfortuitous. A possible relation of the suggested model to the Boltzmann hypothesis is discussed.     

Two-sample Kolmogorov-Smirnov test for truncated data

A nonparametric statistical test to compare two cumulative frequency distribution functions is presented that can be used even when both samples include censored data, as is often the case when comparing the survival of two groups of laboratory animals under conditions in which the experiment is terminated before all the animals die. (Such a design can produce considerable savings and is to be recommended.) The program calculates exact probabilities for both the one-sided and two-sided alternatives to the null hypothesis, applicable to the case of equal group size, as well as the corresponding general asymptotic values; a continuity correction is employed that markedly improves the asymptotic approximation. Expressions are stated in terms of two different but related statistics, and the one that utilizes more information in any particular set of data is selected for the probability calculations. All basic equations and definitions are provided.     

Distribution of the number of false discoveries in large-scale family-based association testing with application to the association between <Emphasis Type="Italic">PTPN1</Emphasis> and hypertension and obesity

We present a model-free approach to the study of the number of false discoveries for large-scale simultaneous family-based association tests (FBATs) in which the set of discoveries is decided by applying a threshold to the test statistics. When the association between a set of markers in a candidate gene and a group of phenotypes is studied by a class of FBATs, we indicate that a joint null hypothesis distribution for these statistics can be obtained by the fundamental statistical method of conditioning on sufficient statistics for the null hypothesis. Based on the joint null distribution of these statistics, we can obtain the distribution of the number of false discoveries for the set of discoveries defined by a threshold; the size of this set is referred to as its tail count. Simulation studies are presented to demonstrate that the conditional, not the unconditional, distribution of the tail count is appropriate for the study of false discoveries. The usefulness of this approach is illustrated by re-examining the association between PTPN1 and a group of blood-pressure-related phenotypes reported by Olivier et al. (Hum Mol Genet 13:1885–1892, 2004); our results refine and reinforce this association.     

Tail posterior probability for inference in pairwise and multiclass gene expression data

We consider the problem of identifying differentially expressed genes in microarray data in a Bayesian framework with a noninformative prior distribution on the parameter quantifying differential expression. We introduce a new rule, tail posterior probability, based on the posterior distribution of the standardized difference, to identify genes differentially expressed between two conditions, and we derive a frequentist estimator of the false discovery rate associated with this rule. We compare it to other Bayesian rules in the considered settings. We show how the tail posterior probability can be extended to testing a compound null hypothesis against a class of specific alternatives in multiclass data.     

Generalized maximally selected statistics

SUMMARY: Maximally selected statistics for the estimation of simple cutpoint models are embedded into a generalized conceptual framework based on conditional inference procedures. This powerful framework contains most of the published procedures in this area as special cases, such as maximally selected chi(2) and rank statistics, but also allows for direct construction of new test procedures for less standard test problems. As an application, a novel maximally selected rank statistic is derived from this framework for a censored response partitioned with respect to two ordered categorical covariates and potential interactions. This new test is employed to search for a high-risk group of rectal cancer patients treated with a neo-adjuvant chemoradiotherapy. Moreover, a new efficient algorithm for the evaluation of the asymptotic distribution for a large class of maximally selected statistics is given enabling the fast evaluation of a large number of cutpoints.     

Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate

Clinical trials are often concerned with the comparison of two treatment groups with multiple endpoints. As alternatives to the commonly used methods, the T2 test and the Bonferroni method, O'Brien (1984, Biometrics 40, 1079-1087) proposes tests based on statistics that are simple or weighted sums of the single endpoints. This approach turns out to be powerful if all treatment differences are in the same direction [compare Pocock, Geller, and Tsiatis (1987, Biometrics 43, 487-498)]. The disadvantage of these multivariate methods is that they are suitable only for demonstrating a global difference, whereas the clinician is further interested in which specific endpoints or sets of endpoints actually caused this difference. It is shown here that all tests are suitable for the construction of a closed multiple test procedure where, after the rejection of the global hypothesis, all lower-dimensional marginal hypotheses and finally the single hypotheses are tested step by step. This procedure controls the experimentwise error rate. It is just as powerful as the multivariate test and, in addition, it is possible to detect significant differences between the endpoints or sets of endpoints.     

Permutation tests for random effects in linear mixed models

Inference regarding the inclusion or exclusion of random effects in linear mixed models is challenging because the variance components are located on the boundary of their parameter space under the usual null hypothesis. As a result, the asymptotic null distribution of the Wald, score, and likelihood ratio tests will not have the typical χ(2) distribution. Although it has been proved that the correct asymptotic distribution is a mixture of χ(2) distributions, the appropriate mixture distribution is rather cumbersome and nonintuitive when the null and alternative hypotheses differ by more than one random effect. As alternatives, we present two permutation tests, one that is based on the best linear unbiased predictors and one that is based on the restricted likelihood ratio test statistic. Both methods involve weighted residuals, with the weights determined by the among- and within-subject variance components. The null permutation distributions of our statistics are computed by permuting the residuals both within and among subjects and are valid both asymptotically and in small samples. We examine the size and power of our tests via simulation under a variety of settings and apply our test to a published data set of chronic myelogenous leukemia patients.     

Significance levels for studies with correlated test statistics

When testing large numbers of null hypotheses, one needs to assess the evidence against the global null hypothesis that none of the hypotheses is false. Such evidence typically is based on the test statistic of the largest magnitude, whose statistical significance is evaluated by permuting the sample units to simulate its null distribution. Efron (2007) has noted that correlation among the test statistics can induce substantial interstudy variation in the shapes of their histograms, which may cause misleading tail counts. Here, we show that permutation-based estimates of the overall significance level also can be misleading when the test statistics are correlated. We propose that such estimates be conditioned on a simple measure of the spread of the observed histogram, and we provide a method for obtaining conditional significance levels. We justify this conditioning using the conditionality principle described by Cox and Hinkley (1974). Application of the method to gene expression data illustrates the circumstances when conditional significance levels are needed.