Testing the Intraclass Version of Kappa Coeffcient of Agreement with Binary Scale and Sample Size Determination

The intraclass version of kappa coefficient has been commonly applied as a measure of agreement for two ratings per subject with binary outcome in reliability studies. We present an efficient statistic for testing the strength of kappa agreement using likelihood scores, and derive asymptotic power and sample size formula. Exact evaluation shows that the score test is generally conservative and more powerful than a method based on a chi‐square goodness‐of‐fit statistic (Donner and Eliasziw , 1992, Statistics in Medicine 11 , 1511–1519). In particular, when the research question is one directional, the one‐sided score test is substantially more powerful and the reduction in sample size is appreciable.     

UMP(U)‐Tests for a Binomial Parameter: A Paradox

We consider uniformly most powerful (UMP) as well as uniformly most powerful unbiased (UMPU) tests and their non‐randomized versions for certain hypotheses concerning a binomial parameter. It will be shown that the power function of a UMP(U)‐test based on sample size n can coincide on the entire parameter space with the power function of the corresponding test based on sample size n + 1. A complete characterization of this paradox will be derived. Apart some exceptional cases for two‐sided tests and equivalence tests the paradox appears if and only if a test based on sample size n is non‐randomized.     

Exact Tests for Comparing Two Paired Proportions with Incomplete Data

Various asymptotic test procedures have been developed previously for testing the equality of two binomial proportions with partially incomplete paired data. Test procedures that discard incomplete observations have been shown to be less powerful than those procedures that utilize all available observations. On the other hand, asymptotic test procedures that utilize all available observations may not be reliable in small‐sample problems or sparse data structures. In this article, unconditional exact test procedures are proposed for testing the equality of two paired binomial proportions with partially incomplete paired data under a random mechanism. The proposed unconditional exact test methods are illustrated with real data from a neurological study. Empirical studies are conducted to investigate the performance of these and other test procedures with respect to size and power. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Stratified Fisher's exact test and its sample size calculation

Chi‐squared test has been a popular approach to the analysis of a 2 × 2 table when the sample sizes for the four cells are large. When the large sample assumption does not hold, however, we need an exact testing method such as Fisher's test. When the study population is heterogeneous, we often partition the subjects into multiple strata, so that each stratum consists of homogeneous subjects and hence the stratified analysis has an improved testing power. While Mantel–Haenszel test has been widely used as an extension of the chi‐squared test to test on stratified 2 × 2 tables with a large‐sample approximation, we have been lacking an extension of Fisher's test for stratified exact testing. In this paper, we discuss an exact testing method for stratified 2 × 2 tables that is simplified to the standard Fisher's test in single 2 × 2 table cases, and propose its sample size calculation method that can be useful for designing a study with rare cell frequencies.     

Improved power of familywise error rate procedures for discrete data under dependency

In many applications where it is necessary to test multiple hypotheses simultaneously, the data encountered are discrete. In such cases, it is important for multiplicity adjustment to take into account the discreteness of the distributions of the p‐values, to assure that the procedure is not overly conservative. In this paper, we review some known multiple testing procedures for discrete data that control the familywise error rate, the probability of making any false rejection. Taking advantage of the fact that the exact permutation or exact pairwise permutation distributions of the p‐values can often be determined when the sample size is small, we investigate procedures that incorporate the dependence structure through the exact permutation distribution and propose two new procedures that incorporate the exact pairwise permutation distributions. A step‐up procedure is also proposed that accounts for the discreteness of the data. The performance of the proposed procedures is investigated through simulation studies and two applications. The results show that by incorporating both discreteness and dependency of p‐value distributions, gains in power can be achieved.     

Incorporating Correlation for Multivariate Failure Time Data When Cluster Size Is Large

Summary We propose a new estimation method for multivariate failure time data using the quadratic inference function (QIF) approach. The proposed method efficiently incorporates within‐cluster correlations. Therefore, it is more efficient than those that ignore within‐cluster correlation. Furthermore, the proposed method is easy to implement. Unlike the weighted estimating equations in Cai and Prentice (1995, Biometrika 82 , 151–164), it is not necessary to explicitly estimate the correlation parameters. This simplification is particularly useful in analyzing data with large cluster size where it is difficult to estimate intracluster correlation. Under certain regularity conditions, we show the consistency and asymptotic normality of the proposed QIF estimators. A chi‐squared test is also developed for hypothesis testing. We conduct extensive Monte Carlo simulation studies to assess the finite sample performance of the proposed methods. We also illustrate the proposed methods by analyzing primary biliary cirrhosis (PBC) data.     

Spatially autocorrelated sampling falsely inflates measures of accuracy for presence‐only niche models

Aim Environmental niche models that utilize presence‐only data have been increasingly employed to model species distributions and test ecological and evolutionary predictions. The ideal method for evaluating the accuracy of a niche model is to train a model with one dataset and then test model predictions against an independent dataset. However, a truly independent dataset is often not available, and instead random subsets of the total data are used for ‘training’ and ‘testing’ purposes. The goal of this study was to determine how spatially autocorrelated sampling affects measures of niche model accuracy when using subsets of a larger dataset for accuracy evaluation. Location The distribution of Centaurea maculosa (spotted knapweed; Asteraceae) was modelled in six states in the western United States: California, Oregon, Washington, Idaho, Wyoming and Montana. Methods Two types of niche modelling algorithms – the genetic algorithm for rule‐set prediction (GARP) and maximum entropy modelling (as implemented with Maxent) – were used to model the potential distribution of C. maculosa across the region. The effect of spatially autocorrelated sampling was examined by applying a spatial filter to the presence‐only data (to reduce autocorrelation) and then comparing predictions made using the spatial filter with those using a random subset of the data, equal in sample size to the filtered data. Results The accuracy of predictions from both algorithms was sensitive to the spatial autocorrelation of sampling effort in the occurrence data. Spatial filtering led to lower values of the area under the receiver operating characteristic curve plot but higher similarity statistic (I) values when compared with predictions from models built with random subsets of the total data, meaning that spatial autocorrelation of sampling effort between training and test data led to inflated measures of accuracy. Main conclusions The findings indicate that care should be taken when interpreting the results from presence‐only niche models when training and test data have been randomly partitioned but occurrence data were non‐randomly sampled (in a spatially autocorrelated manner). The higher accuracies obtained without the spatial filter are a result of spatial autocorrelation of sampling effort between training and test data inflating measures of prediction accuracy. If independently surveyed data for testing predictions are unavailable, then it may be necessary to explicitly account for the spatial autocorrelation of sampling effort between randomly partitioned training and test subsets when evaluating niche model predictions.     

Sample Size Determination for Designing a Strata-Matched Case-Control Study to Detect Multiple Risk Factors

Investigations of sample size for planning case-control studies have usually been limited to detecting a single factor. In this paper, we investigate sample size for multiple risk factors in strata-matched case-control studies. We construct an omnibus statistic for testing M different risk factors based on the jointly sufficient statistics of parameters associated with the risk factors. The statistic is non-iterative, and it reduces to the Cochran statistic when M = 1. The asymptotic power function of the test is a non-central chi-square with M degrees of freedom and the sample size required for a specific power can be obtained by the inverse relationship. We find that the equal sample allocation is optimum. A Monte Carlo experiment demonstrates that an approximate formula for calculating sample size is satisfactory in typical epidemiologic studies. An approximate sample size obtained using Bonferroni's method for multiple comparisons is much larger than that obtained using the omnibus test. Approximate sample size formulas investigated in this paper using the omnibus test, as well as the individual tests, can be useful in designing case-control studies for detecting multiple risk factors.     

Accounting for missing data in the estimation of contemporary genetic effective population size (Ne)

Theoretical models are often applied to population genetic data sets without fully considering the effect of missing data. Researchers can deal with missing data by removing individuals that have failed to yield genotypes and/or by removing loci that have failed to yield allelic determinations, but despite their best efforts, most data sets still contain some missing data. As a consequence, realized sample size differs among loci, and this poses a problem for unbiased methods that must explicitly account for random sampling error. One commonly used solution for the calculation of contemporary effective population size (N_e) is to calculate the effective sample size as an unweighted mean or harmonic mean across loci. This is not ideal because it fails to account for the fact that loci with different numbers of alleles have different information content. Here we consider this problem for genetic estimators of contemporary effective population size (N_e). To evaluate bias and precision of several statistical approaches for dealing with missing data, we simulated populations with known N_e and various degrees of missing data. Across all scenarios, one method of correcting for missing data (fixed‐inverse variance‐weighted harmonic mean) consistently performed the best for both single‐sample and two‐sample (temporal) methods of estimating N_e and outperformed some methods currently in widespread use. The approach adopted here may be a starting point to adjust other population genetics methods that include per‐locus sample size components.     

K‐Sample Test and Sample Size Calculation for Comparing Slopes in Data with Repeated Measurements

Sample size calculations based on two‐sample comparisons of slopes in repeated measurements have been reported by many investigators. In contrast, the literature has paid relatively little attention to the design and analysis of K‐sample trials in repeated measurements studies where K is 3 or greater. Jung and Ahn (2003) derived a closed sample size formula for two‐sample comparisons of slopes by taking into account the impact of missing data. We extend their method to compare K‐sample slopes in repeated measurement studies using the generalized estimating equation (GEE) approach based on independent working correlation structure. We investigate the performance of the sample size formula since the sample size formula is based on asymptotic theory. The proposed sample size formula is illustrated using a clinical trial example. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Cost analysis in choosing group size when group testing for Potato virus Y in the presence of classification errors

In many areas of the world, Potato virus Y (PVY) is one of the most economically important disease problems in seed potatoes. In Taiwan, generation 2 (G2) class certified seed potatoes are required by law to be free of detectable levels of PVY. To meet this standard, it is necessary to perform accurate tests at a reasonable cost. We used a two‐stage testing design involving group testing which was performed in Taiwan's Seed Improvement and Propagation Station to identify plants infected with PVY. At the first stage of this two‐stage testing design, plants are tested in groups. The second stage involves no retesting for negative test groups and exhaustive testing of all constituent individual samples from positive test groups. In order to minimise costs while meeting government standards, it is imperative to estimate optimal group size. However, because of limited test accuracy, classification errors for diagnostic tests are inevitable; to get a more accurate estimate, it is necessary to adjust for these errors. Therefore, this paper describes an analysis of diagnostic test data in which specimens are grouped for batched testing to offset costs. The optimal batch size is determined by various cost parameters as well as test sensitivity, specificity and disease prevalence. Here, the Bayesian method is employed to deal with uncertainty in these parameters. Moreover, we developed a computer program to determine optimal group size for PVY tests such that the expected cost is minimised even when using imperfect diagnostic tests of pooled samples. Results from this research show that, compared with error free testing, when the presence of diagnostic testing errors is taken into account, the optimal group size becomes smaller. Higher diagnostic testing costs, lower costs of false negatives or smaller prevalence can all lead to a larger optimal group size. Regarding the effects of sensitivity and specificity, optimal group size increases as sensitivity increases; however, specificity has little effect on determining optimal group size. From our simulated study, it is apparent that the Bayesian method can truly update the prior information to more closely approximate the intrinsic characteristics of the parameters of interest. We believe that the results of this study will be useful in the implementation of seed potato certification programmes, particularly those which require zero tolerance for quarantine diseases in certified tubers.     

Using between-model comparisons to fine-tune linear models of species ranges

Aim To offer an objective approach to some of the problems associated with the development of logistic regression models: how to compare different models, determination of sample size adequacy, the influence of the ratio of positive to negative cells on model accuracy, and the appropriate scale at which the hypothesis of a non‐random distribution should be tested. Location Test data were taken from Southern Africa. Methods The approach relies mainly on the use of the AUC (Area under the Curve) statistic, based on ROC (threshold Receiver Operating Characteristic) plots, for between‐model comparisons. Data for the distribution of the bont tick Amblyomma hebraeum Koch (Acari: Ixodidae) are used to illustrate the methods. Results Methods for the estimation of minimum sample sizes and more accurate hypothesis‐testing are outlined. Logistic regression is robust to the assumption that uncollected cells can be scored as negative, provided that the sample size of cells scored as positive is adequate. The variation in temperature and rainfall at localities where A. hebraeum has been collected is significantly lower than expected from a random sample of points across the data set, suggesting that within‐site variation may be an important determinant of its distribution. Main conclusions Between‐model comparisons relying on AUCs can be used to enhance objectivity in the development and refinement of logistic regression models. Both between‐site and within‐site variability should be considered as potentially important factors determining species distributions.     

Confidence Interval for Rate Ratio in a 2 × 2 Table with Structural Zero: An Application in Assessing False-Negative Rate Ratio When Combining Two Diagnostic Tests

Summary . In this article, we consider problems with correlated data that can be summarized in a 2 × 2 table with structural zero in one of the off‐diagonal cells. Data of this kind sometimes appear in infectious disease studies and two‐step procedure studies. Lui (1998, Biometrics 54, 706–711) considered confidence interval estimation of rate ratio based on Fieller‐type, Wald‐type, and logarithmic transformation statistics. We reexamine the same problem under the context of confidence interval construction on false‐negative rate ratio in diagnostic performance when combining two diagnostic tests. We propose a score statistic for testing the null hypothesis of nonunity false‐negative rate ratio. Score test–based confidence interval construction for false‐negative rate ratio will also be discussed. Simulation studies are conducted to compare the performance of the new derived score test statistic and existing statistics for small to moderate sample sizes. In terms of confidence interval construction, our asymptotic score test–based confidence interval estimator possesses significantly shorter expected width with coverage probability being close to the anticipated confidence level. In terms of hypothesis testing, our asymptotic score test procedure has actual type I error rate close to the pre‐assigned nominal level. We illustrate our methodologies with real examples from a clinical laboratory study and a cancer study.     

On Exact Unconditional Test for Linear Trend in Dose‐Response Studies

The one‐degree‐of‐freedom Cochran‐Armitage (C‐A) test statistic for linear trend has been widely applied in various dose‐response studies (e.g., anti‐ulcer medications and short‐term antibiotics, animal carcinogenicity bioassays and occupational toxicant studies). This approximate statistic relies, however, on asymptotic theory that is reliable only when the sample sizes are reasonably large and well balanced across dose levels. For small, sparse, or skewed data, the asymptotic theory is suspect and exact conditional method (based on the C‐A statistic) seems to provide a dependable alternative. Unfortunately, the exact conditional method is only practical for the linear logistic model from which the sufficient statistics for the regression coefficients can be obtained explicitly. In this article, a simple and efficient recursive polynomial multiplication algorithm for exact unconditional test (based on the C‐A statistic) for detecting a linear trend in proportions is derived. The method is applicable for all choices of the model with monotone trend including logistic, probit, arcsine, extreme value and one hit. We also show that this algorithm can be easily extended to exact unconditional power calculation for studies with up to a moderately large sample size. A real example is given to illustrate the applicability of the proposed method.     

Bootstrap and Second‐Order Tests of Risk Difference

Summary Clinical trials data often come in the form of low‐dimensional tables of small counts. Standard approximate tests such as score and likelihood ratio tests are imperfect in several respects. First, they can give quite different answers from the same data. Second, the actual type‐1 error can differ significantly from nominal, even for quite large sample sizes. Third, exact inferences based on these can be strongly nonmonotonic functions of the null parameter and lead to confidence sets that are discontiguous. There are two modern approaches to small sample inference. One is to use so‐called higher order asymptotics ( Reid, 2003 , Annal of Statistics 31 , 1695–1731) to provide an explicit adjustment to the likelihood ratio statistic. The theory for this is complex but the statistic is quick to compute. The second approach is to perform an exact calculation of significance assuming the nuisance parameters equal their null estimate ( Lee and Young, 2005 , Statistic and Probability Letters 71 , 143–153), which is a kind of parametric bootstrap. The purpose of this article is to explain and evaluate these two methods, for testing whether a difference in probabilities p₂? p₁ exceeds a prechosen noninferiority margin δ₀ . On the basis of an extensive numerical study, we recommend bootstrap P‐values as superior to all other alternatives. First, they produce practically identical answers regardless of the basic test statistic chosen. Second, they have excellent size accuracy and higher power. Third, they vary much less erratically with the null parameter value δ₀ .     

Detecting Genomic Aberrations Using Products in a Multiscale Analysis

Summary Genomic instability, such as copy‐number losses and gains, occurs in many genetic diseases. Recent technology developments enable researchers to measure copy numbers at tens of thousands of markers simultaneously. In this article, we propose a nonparametric approach for detecting the locations of copy‐number changes and provide a measure of significance for each change point. The proposed test is based on seeking scale‐based changes in the sequence of copy numbers, which is ordered by the marker locations along the chromosome. The method leads to a natural way to estimate the null distribution for the test of a change point and adjusted p‐values for the significance of a change point using a step‐down maxT permutation algorithm to control the family‐wise error rate. A simulation study investigates the finite sample performance of the proposed method and compares it with a more standard sequential testing method. The method is illustrated using two real data sets.     

k‐FWER Control without p ‐value Adjustment,with Application to Detection of Genetic Determinants of Multiple Sclerosis in Italian Twins

Summary We show a novel approach for k‐FWER control which does not involve any correction, but only testing the hypotheses along a (possibly data‐driven) order until a suitable number of p‐values are found above the uncorrected α level. p‐values can arise from any linear model in a parametric or nonparametric setting. The approach is not only very simple and computationally undemanding, but also the data‐driven order enhances power when the sample size is small (and also when k and/or the number of tests is large). We illustrate the method on an original study about gene discovery in multiple sclerosis, in which were involved a small number of couples of twins, discordant by disease. The methods are implemented in an R package (someKfwer ), freely available on CRAN.     

Extension of Peto's Test by Attribution of Tumor Lethality in the Absence of Cause‐of‐Death Information

A new statistical testing approach is developed for rodent tumorigenicity assays that have a single terminal sacrifice or occasionally interim sacrifices but not cause‐of‐death data. For experiments that lack cause‐of‐death data, statistically imputed numbers of fatal tumors and incidental tumors are used to modify Peto's cause‐of‐death test which is usually implemented using pathologist‐assigned cause‐of‐death information. The numbers of fatal tumors are estimated using a constrained nonparametric maximum likelihood estimation method. A new Newton‐based approach under inequality constraints is proposed for finding the global maximum likelihood estimates. In this study, the proposed method is concentrated on data with a single sacrifice experiment without implementing further assumptions. The new testing approach may be more reliable than Peto's test because of the potential for a misclassification of cause‐of‐death by pathologists. A Monte Carlo simulation study for the proposed test is conducted to assess size and power of the test. Asymptotic normality for the statistic of the proposed test is also investigated. The proposed testing approach is illustrated using a real data set.     

Effect of sampling strategy on estimation of fine‐scale spatial genetic structure in Androsace tapete (Primulaceae), an alpine plant endemic to Qinghai‐Tibetan Plateau

Abstract The fine‐scale spatial genetic structure (SGS) of alpine plants is receiving increasing attention, from which seed and pollen dispersal can be inferred. However, estimation of SGS may depend strongly on the sampling strategy, including the sample size and spatial sampling scheme. Here, we examined the effects of sample size and three spatial schemes, simple‐random, line‐transect, and random‐cluster sampling, on the estimation of SGS in Androsace tapete, an alpine cushion plant endemic to Qinghai‐Tibetan Plateau. Using both real data and simulated data of dominant molecular markers, we show that: (i) SGS is highly sensitive to sample strategy especially when the sample size is small (e.g., below 100); (ii) the commonly used SGS parameter (the intercept of the autocorrelogram) is more susceptible to sample error than a newly developed Sp statistic; and (iii) the random‐cluster scheme is susceptible to obvious bias in parameter estimation even when the sample size is relatively large (e.g., above 200). Overall, the line‐transect scheme is recommendable, in that it performs slightly better than the simple‐random scheme in parameter estimation and is more efficient to encompass broad spatial scales. The consistency between simulated data and real data implies that these findings might hold true in other alpine plants and more species should be examined in future work.