Nonparametric confidence regions for the symmetry point-based optimal cutpoint and associated sensitivity of a continuous-scale diagnostic test

In medical research, diagnostic tests with continuous values are widely employed to attempt to distinguish between diseased and non-diseased subjects. The diagnostic accuracy of a test (or a biomarker) can be assessed by using the receiver operating characteristic (ROC) curve of the test. To summarize the ROC curve and primarily to determine an “optimal” threshold for test results to use in practice, several approaches may be considered, such as those based on the Youden index, on the so-called close-to-(0,1) point, on the concordance probability and on the symmetry point. In this paper, we focus on the symmetry point-based approach, that simultaneously controls the probabilities of the two types of correct classifications (healthy as healthy and diseased as diseased), and show how to get joint nonparametric confidence regions for the corresponding optimal cutpoint and the associated sensitivity (= specificity) value. Extensive simulation experiments are conducted to evaluate the finite sample performances of the proposed method. Real datasets are also used to illustrate its application.     

Distribution-free ROC analysis using binary regression techniques

Receiver operating characteristic (ROC) regression methodology is used to identify factors that affect the accuracy of medical diagnostic tests. In this paper, we consider a ROC model for which the ROC curve is a parametric function of covariates but distributions of the diagnostic test results are not specified. Covariates can be either common to all subjects or specific to those with disease. We propose a new estimation procedure based on binary indicators defined by the test result for a diseased subject exceeding various specified quantiles of the distribution of test results from non-diseased subjects with the same covariate values. This procedure is conceptually and computationally simplified relative to existing procedures. Simulation study results indicate that the approach has fairly high statistical efficiency. The new ROC regression methodology is used to evaluate childhood measurements of body mass index as a predictive marker of adult obesity.     

The Clinical Profiles of Individuals With and Without Root Surface Caries1

As part of a collaborative multidisciplinary investigation of root surface caries (RSC), 273 subjects (median age = 57) were clinically evaluated for coronal caries, debris, calculus, gingivitis, periodontal pocketing, recession, abrasion, as well as RSC. The entire study group was divided into three subgroups: (1) subjects whose exposed root surface were without lesions or restorations (non-diseased individuals, n= 43); (2) subjects whose exposed root surfaces had one or more lesions and may or may not have had restorations (diseased individuals, n= 110); (3) subjects with one or more restorations but no lesions, since it was not known whether these restorations were preceded by RSC or abrasion the disease status of these individuals was unclear (n= 120). A comparison of Subgroup 1 (unequivocally non-diseased) and Subgroup 2 (unequivocally diseased) revealed that subjects without RSC had more teeth, less coronal caries, less recession, less debris, less calculus, less gingivitis, and more abrasion. We infer that the common factor underlying most if not all of the subgroup differences is oral hygiene which must therefore be a major disease determinant.     

Simple, defensible sample sizes based on cost efficiency

Summary .   The conventional approach of choosing sample size to provide 80% or greater power ignores the cost implications of different sample size choices. Costs, however, are often impossible for investigators and funders to ignore in actual practice. Here, we propose and justify a new approach for choosing sample size based on cost efficiency, the ratio of a study's projected scientific and/or practical value to its total cost. By showing that a study's projected value exhibits diminishing marginal returns as a function of increasing sample size for a wide variety of definitions of study value, we are able to develop two simple choices that can be defended as more cost efficient than any larger sample size. The first is to choose the sample size that minimizes the average cost per subject. The second is to choose sample size to minimize total cost divided by the square root of sample size. This latter method is theoretically more justifiable for innovative studies, but also performs reasonably well and has some justification in other cases. For example, if projected study value is assumed to be proportional to power at a specific alternative and total cost is a linear function of sample size, then this approach is guaranteed either to produce more than 90% power or to be more cost efficient than any sample size that does. These methods are easy to implement, based on reliable inputs, and well justified, so they should be regarded as acceptable alternatives to current conventional approaches.     

Cost-effective designs for linkage disequilibrium mapping of complex traits

The current development of densely spaced collections of single nucleotide polymorphisms (SNPs) will lead to genomewide association studies for a wide range of diseases in many different populations. Determinations of the appropriate number of SNPs to genotype involve a balancing of power and cost. Several variables are important in these determinations. We show that there are different combinations of sample size and marker density that can be expected to achieve the same power. Within certain bounds, investigators can choose between designs with more subjects and fewer markers or those with more markers and fewer subjects. Which designs are more cost-effective depends on the cost of phenotyping versus the cost of genotyping. We show that, under the assumption of a set cost for genotyping, one can calculate a "threshold cost" for phenotyping; when phenotyping costs per subject are less than this threshold, designs with more subjects will be more cost-effective than designs with more markers. This framework for determining a cost-effective study will aid in the planning of studies, especially if there are choices to be made with respect to phenotyping methods or study populations.     

An efficient sample size adaptation strategy with adjustment of randomization ratio

In clinical trials, sample size reestimation is a useful strategy for mitigating the risk of uncertainty in design assumptions and ensuring sufficient power for the final analysis. In particular, sample size reestimation based on unblinded interim effect size can often lead to sample size increase, and statistical adjustment is usually needed for the final analysis to ensure that type I error rate is appropriately controlled. In current literature, sample size reestimation and corresponding type I error control are discussed in the context of maintaining the original randomization ratio across treatment groups, which we refer to as “proportional increase.” In practice, not all studies are designed based on an optimal randomization ratio due to practical reasons. In such cases, when sample size is to be increased, it is more efficient to allocate the additional subjects such that the randomization ratio is brought closer to an optimal ratio. In this research, we propose an adaptive randomization ratio change when sample size increase is warranted. We refer to this strategy as “nonproportional increase,” as the number of subjects increased in each treatment group is no longer proportional to the original randomization ratio. The proposed method boosts power not only through the increase of the sample size, but also via efficient allocation of the additional subjects. The control of type I error rate is shown analytically. Simulations are performed to illustrate the theoretical results.     

Isolation of primary human hepatocytes from normal and diseased liver tissue: a one hundred liver experience

Successful and consistent isolation of primary human hepatocytes remains a challenge for both cell-based therapeutics/transplantation and laboratory research. Several centres around the world have extensive experience in the isolation of human hepatocytes from non-diseased livers obtained from donor liver surplus to surgical requirement or at hepatic resection for tumours. These livers are an important but limited source of cells for therapy or research. The capacity to isolate cells from diseased liver tissue removed at transplantation would substantially increase availability of cells for research. However no studies comparing the outcome of human hepatocytes isolation from diseased and non-diseased livers presently exist. Here we report our experience isolating human hepatocytes from organ donors, non-diseased resected liver and cirrhotic tissue. We report the cell yields and functional qualities of cells isolated from the different types of liver and demonstrate that a single rigorous protocol allows the routine harvest of good quality primary hepatocytes from the most commonly accessible human liver tissue samples.     

Sample size/power calculation for case-cohort studies

In epidemiologic studies and disease prevention trials, interest often involves estimation of the relationship between some disease endpoints and individual exposure. In some studies, due to the rarity of the disease and the cost in collecting the exposure information for the entire cohort, a case-cohort design, which consists of a small random sample of the whole cohort and all the diseased subjects, is often used. Previous work has focused on analyzing data from the case-cohort design and few have discussed the sample size issues. In this article, we describe two tests for the case-cohort design, which can be treated as a natural generalization of log-rank test in the full cohort design. We derive an explicit form for power/sample size calculation based on these two tests. A number of simulation studies have been used to illustrate the efficiency of the tests for the case-cohort design. An example is provided on how to use the formula.     

Sample size determination for classifiers based on single-nucleotide polymorphisms

Single-nucleotide polymorphisms (SNPs), believed to determine human differences, are widely used to predict risk of diseases. Typically, clinical samples are limited and/or the sampling cost is high. Thus, it is essential to determine an adequate sample size needed to build a classifier based on SNPs. Such a classifier would facilitate correct classifications, while keeping the sample size to a minimum, thereby making the studies cost-effective. For coded SNP data from 2 classes, an optimal classifier and an approximation to its probability of correct classification (PCC) are derived. A linear classifier is constructed and an approximation to its PCC is also derived. These approximations are validated through a variety of Monte Carlo simulations. A sample size determination algorithm based on the criterion, which ensures that the difference between the 2 approximate PCCs is below a threshold, is given and its effectiveness is illustrated via simulations. For the HapMap data on Chinese and Japanese populations, a linear classifier is built using 51 independent SNPs, and the required total sample sizes are determined using our algorithm, as the threshold varies. For example, when the threshold value is 0.05, our algorithm determines a total sample size of 166 (83 for Chinese and 83 for Japanese) that satisfies the criterion.     

Kinship index variations among populations and thresholds for familial searching

Current familial searching strategies are developed primarily based on autosomal STR loci, since most of the offender profiles in the forensic DNA databases do not contain Y-STR or mitochondrial DNA data. There are generally two familial searching methods, Identity-by-State (IBS) based methods or kinship index (KI) based methods. The KI based method is an analytically superior method because the allele frequency information is considered as opposed to solely allele counting. However, multiple KIs should be calculated if the unknown forensic profile may be attributed to multiple possible relevant populations. An important practical issue is the KI threshold to select for limiting the list of candidates from a search. There are generally three strategies of setting the KI threshold for familial searching: (1) SWGDAM recommendation 6; (2) minimum KI≥KI threshold; and (3) maximum KI≥KI threshold. These strategies were evaluated and compared by using both simulation data and empirical data. The minimum KI will tend to be closer to the KI appropriate for the population of which the forensic profile belongs. The minimum KI≥KI threshold performs better than the maximum KI≥KI threshold. The SWGDAM strategy may be too stringent for familial searching with large databases (e.g., 1 million or more profiles), because its KI thresholds depend on the database size and the KI thresholds of large databases have a higher probability to exclude true relatives than smaller databases. Minimum KI≥KI threshold strategy is a better option, as it provides the flexibility to adjust the KI threshold according to a pre-determined number of candidates or false positive/negative rates. Joint use of both IBS and KI does not significantly reduce the chance of including true relatives in a candidate list, but does provide a higher efficiency of familial searching.     

A modified sign test for comparing paired ROC curves

We develop a permutation test for assessing a difference in the areas under the curve (AUCs) in a paired setting where both modalities are given to each diseased and nondiseased subject. We propose that permutations be made between subjects specifically by shuffling the diseased/nondiseased labels of the subjects within each modality. As these permutations are made within modality, the permutation test is valid even if both modalities are measured on different scales. We show that our permutation test is a sign test for the symmetry of an underlying discrete distribution whose size remains valid under the assumption of equal AUCs. We demonstrate the operating characteristics of our test via simulation and show that our test is equal in power to a permutation test recently proposed by Bandos and others (2005).     

Sample Size Considerations for GEE Analyses of Three‐Level Cluster Randomized Trials

Summary Cluster randomized trials in health care may involve three instead of two levels, for instance, in trials where different interventions to improve quality of care are compared. In such trials, the intervention is implemented in health care units (“clusters”) and aims at changing the behavior of health care professionals working in this unit (“subjects”), while the effects are measured at the patient level (“evaluations”). Within the generalized estimating equations approach, we derive a sample size formula that accounts for two levels of clustering: that of subjects within clusters and that of evaluations within subjects. The formula reveals that sample size is inflated, relative to a design with completely independent evaluations, by a multiplicative term that can be expressed as a product of two variance inflation factors, one that quantifies the impact of within‐subject correlation of evaluations on the variance of subject‐level means and the other that quantifies the impact of the correlation between subject‐level means on the variance of the cluster means. Power levels as predicted by the sample size formula agreed well with the simulated power for more than 10 clusters in total, when data were analyzed using bias‐corrected estimating equations for the correlation parameters in combination with the model‐based covariance estimator or the sandwich estimator with a finite sample correction.     

Comparison of disease prevalence in two populations in the presence of misclassification

Comparing disease prevalence in two groups is an important topic in medical research, and prevalence rates are obtained by classifying subjects according to whether they have the disease. Both high‐cost infallible gold‐standard classifiers or low‐cost fallible classifiers can be used to classify subjects. However, statistical analysis that is based on data sets with misclassifications leads to biased results. As a compromise between the two classification approaches, partially validated sets are often used in which all individuals are classified by fallible classifiers, and some of the individuals are validated by the accurate gold‐standard classifiers. In this article, we develop several reliable test procedures and approximate sample size formulas for disease prevalence studies based on the difference between two disease prevalence rates with two independent partially validated series. Empirical studies show that (i) the Score test produces close‐to‐nominal level and is preferred in practice; and (ii) the sample size formula based on the Score test is also fairly accurate in terms of the empirical power and type I error rate, and is hence recommended. A real example from an aplastic anemia study is used to illustrate the proposed methodologies.     

Two-stage case–control designs for rare genetic variants

The search for the association of rare genetic variants with common diseases is of high interest, yet challenging because of cost considerations. We present an efficient two-stage design that uses diseased cases to first screen for rare variants at stage-1. If too few cases are found to carry any variants, the study stops. Otherwise, the selected variants are screened at stage-2 in a larger set of cases and controls, and the frequency of variants is compared between cases and controls by an exact test that corrects for the stage-1 ascertainment. Simulations show that our new method provides conservative Type-I error rates, similar to the conservative aspect of Fisher’s exact test. We show that the probability of stopping at stage-1 increases with a smaller number of cases screened at stage-1, a larger stage-1 continuation threshold, or a smaller carrier probability. Our simulations also show how these factors impact the power at stage-2. To balance stopping early when there are few variant carriers versus continuation to stage-2 when the variants have a reasonable effect size on the phenotype, we provide guidance on designing an optimal study that minimizes the expected sample size when the null hypothesis is true, yet achieves the desired power.     

Probability of detecting disease-associated single nucleotide polymorphisms in case-control genome-wide association studies

Some case-control genome-wide association studies (CCGWASs) select promising single nucleotide polymorphisms (SNPs) by ranking corresponding p-values, rather than by applying the same p-value threshold to each SNP. For such a study, we define the detection probability (DP) for a specific disease-associated SNP as the probability that the SNP will be "T-selected," namely have one of the top T largest chi-square values (or smallest p-values) for trend tests of association. The corresponding proportion positive (PP) is the fraction of selected SNPs that are true disease-associated SNPs. We study DP and PP analytically and via simulations, both for fixed and for random effects models of genetic risk, that allow for heterogeneity in genetic risk. DP increases with genetic effect size and case-control sample size and decreases with the number of nondisease-associated SNPs, mainly through the ratio of T to N, the total number of SNPs. We show that DP increases very slowly with T, and the increment in DP per unit increase in T declines rapidly with T. DP is also diminished if the number of true disease SNPs exceeds T. For a genetic odds ratio per minor disease allele of 1.2 or less, even a CCGWAS with 1000 cases and 1000 controls requires T to be impractically large to achieve an acceptable DP, leading to PP values so low as to make the study futile and misleading. We further calculate the sample size of the initial CCGWAS that is required to minimize the total cost of a research program that also includes follow-up studies to examine the T-selected SNPs. A large initial CCGWAS is desirable if genetic effects are small or if the cost of a follow-up study is large.     

Iwao’s patchiness regression through the origin: biological importance and efficiency of sampling applications

Iwao’s mean crowding-mean density relation can be treated both as a linear function describing the biological characteristics of a species at a population level, or a regression model fitted to empirical data (Iwao’s patchiness regression). In this latter form its parameters are commonly used to construct sampling plans for insect pests, which are characteristically patchily distributed or overdispersed. It is shown in this paper that modifying both the linear function and statistical model to force the intercept or lower functional limit through the origin results in more intuitive biological interpretation of parameters and better sampling economy. Firstly, forcing the function through the origin has the effect of ensuring that zero crowding occurs when zero individuals occupy a patch. Secondly, it ensures that negative values of the intercept, which do not yield an intuitive biological interpretation, will not arise. It is shown analytically that sequential sampling plans based on regression through the origin should be more efficient compared to plans based on conventional regression. For two overdispersed data sets, through-origin based plans collected a significantly lower sample size during validation than plans based on conventional regression, but the improvement in sampling efficiency was not large enough to be of practical benefit. No difference in sample size was observed when through-origin and conventional regression based plans were validated using underdispersed data. A field researcher wishing to adopt a through-origin form of Iwao’s regression for the biological reasons outlined above can therefore be confident that their sampling strategies will not be affected by doing so.     

Dose-response and thresholds in mutagenicity studies: a statistical testing approach

The analysis of dose-response relationships is an important objective in toxicology, and one in which both modelling and testing approaches are used. One particular question is whether a threshold exists at low doses. The concept of a pragmatic threshold is used, i.e. low doses with biologically unimportant effects are assumed to be threshold doses. "Biologically unimportant" means, in statistical terms, a lower effect than the effect of the negative control, or at least a just-tolerable margin delta higher than the effect of the negative control. Therefore, threshold doses can be tested in terms of a one-sided hypothesis of equivalence. A new approach is proposed, assuming, at the least, that the low dose is a threshold dose, and the highest dose is superior to the negative control. By analogy to the k-fold rule commonly used in mutagenicity studies, tests on ratio-to-control are used. The a priori definition of the threshold margin is inherently needed. A further approach proposes the analysis of dose-response relationships by means of order-restricted inference (the so-called trend test). A modification of a multiple-contrast test is used, in which only those contrasts are included that are sensitive for no effects at low doses. A further modification treats the complicated, but real, problem of simultaneous existence of a threshold, a monotonic increase, and a downturn effect at high dose(s). A parametric procedure is considered, together with an extension for proportions. The important problem of a priori sample size definition is discussed. The approaches are demonstrated by means of examples based on real data.     

Accounting for variability in sample size estimation with applications to nonadherence and estimation of variance and effect size

We consider sample size calculations for testing differences in means between two samples and allowing for different variances in the two groups. Typically, the power functions depend on the sample size and a set of parameters assumed known, and the sample size needed to obtain a prespecified power is calculated. Here, we account for two sources of variability: we allow the sample size in the power function to be a stochastic variable, and we consider estimating the parameters from preliminary data. An example of the first source of variability is nonadherence (noncompliance). We assume that the proportion of subjects who will adhere to their treatment regimen is not known before the study, but that the proportion is a stochastic variable with a known distribution. Under this assumption, we develop simple closed form sample size calculations based on asymptotic normality. The second source of variability is in parameter estimates that are estimated from prior data. For example, we account for variability in estimating the variance of the normal response from existing data which are assumed to have the same variance as the study for which we are calculating the sample size. We show that we can account for the variability of the variance estimate by simply using a slightly larger nominal power in the usual sample size calculation, which we call the calibrated power. We show that the calculation of the calibrated power depends only on the sample size of the existing data, and we give a table of calibrated power by sample size. Further, we consider the calculation of the sample size in the rarer situation where we account for the variability in estimating the standardized effect size from some existing data. This latter situation, as well as several of the previous ones, is motivated by sample size calculations for a Phase II trial of a malaria vaccine candidate.     

Sample Size Determination for Grouped Exponential Observations: A Cost Function Approach

Calculating the required sample size for a desired power at a given type I error level, we often assume that we know the exact time of all subject responses whenever they occur during our study period. It is very common, however, in practice that we only monitor subjects periodically and, therefore, we know only whether responses occur or not during an interval. This paper includes a quantitative discussion of the effect resulting from data grouping or interval censoring on the required sample size when we have two treatment groups. Furthermore, with the goal of exploring the optimum in the number of subjects, the number of examinations per subject for test responses, and the total length of a study time period, this paper also provides a general guideline about how to determine these to minimize the total cost of a study for a desired power at a given α-level. A specified linear cost function that incorporates the costs of obtaining subjects, periodic examinations for test responses of subjects, and the total length of a study period, is assumed, primarily for illustrative purpose.     

Small Vaccination Experiments with Binary Outcome: The Paradox of Increasing Power with Decreasing Sample Size and/or Increasing Imbalance

In animal vaccination experiments with binary outcome (diseased/non diseased), the comparison of the vaccinated and control group is often based on the Fisher exact test. A tool for the evaluation of different designs is proposed, based on the expected power of the Fisher exact test. The expected power can sometimes unexpectedly increase with decreasing sample size and/or increasing imbalance. The reasons for these peculiar results are explained and compared to the results of two other types of tests: the unconditional test and the randomisation test. In a vaccination experiment with a restricted number of animals it is shown to be important to consider expected power in order to choose the most appropriate design.