Testing the Intraclass Version of Kappa Coeffcient of Agreement with Binary Scale and Sample Size Determination

The intraclass version of kappa coefficient has been commonly applied as a measure of agreement for two ratings per subject with binary outcome in reliability studies. We present an efficient statistic for testing the strength of kappa agreement using likelihood scores, and derive asymptotic power and sample size formula. Exact evaluation shows that the score test is generally conservative and more powerful than a method based on a chi‐square goodness‐of‐fit statistic (Donner and Eliasziw , 1992, Statistics in Medicine 11 , 1511–1519). In particular, when the research question is one directional, the one‐sided score test is substantially more powerful and the reduction in sample size is appreciable.     

POWSIM: a computer program for assessing statistical power when testing for genetic differentiation

Knowledge of statistical power is essential for sampling design and data evaluation when testing for genetic differentiation. Yet, such information is typically missing in studies of conservation and evolutionary genetics, most likely because of complex interactions between the many factors that affect power. powsim is a 32‐bit Windows/DOS simulation‐based computer program that estimates power (and α error) for chi‐square and Fisher's exact tests when evaluating the hypothesis of genetic homogeneity. Optional combinations include the number of samples, sample sizes, number of loci and alleles, allele frequencies, and degree of differentiation (quantified as F_ST). powsim is available at http://www.zoologi.su.se/~ryman .     

Testing for Homogeneity in Meta‐Analysis I. The One‐Parameter Case: Standardized Mean Difference

Summary Meta‐analysis seeks to combine the results of several experiments in order to improve the accuracy of decisions. It is common to use a test for homogeneity to determine if the results of the several experiments are sufficiently similar to warrant their combination into an overall result. Cochran’s Q statistic is frequently used for this homogeneity test. It is often assumed that Q follows a chi‐square distribution under the null hypothesis of homogeneity, but it has long been known that this asymptotic distribution for Q is not accurate for moderate sample sizes. Here, we present an expansion for the mean of Q under the null hypothesis that is valid when the effect and the weight for each study depend on a single parameter, but for which neither normality nor independence of the effect and weight estimators is needed. This expansion represents an order O(1/n) correction to the usual chi‐square moment in the one‐parameter case. We apply the result to the homogeneity test for meta‐analyses in which the effects are measured by the standardized mean difference (Cohen’s d‐statistic). In this situation, we recommend approximating the null distribution of Q by a chi‐square distribution with fractional degrees of freedom that are estimated from the data using our expansion for the mean of Q. The resulting homogeneity test is substantially more accurate than the currently used test. We provide a program available at the Paper Information link at the Biometrics website http://www.biometrics.tibs.org for making the necessary calculations.     

New Ranked Set Sampling for One‐Parameter Family of Distributions

Bhoj (1997c) proposed a new ranked set sampling (NRSS) procedure for a specific two‐parameter family of distributions when the sample size is even. This NRSS procedure can be applied to one‐parameter family of distributions when the sample size is even. However, this procedure cannot be used if the sample size is odd. Therefore, in this paper, we propose a modified version of the NRSS procedure which can be used for one‐parameter distributions when the sample size is odd. Simple estimator for the parameter based on proposed NRSS is derived. The relative precisions of this estimator are higher than those of other estimators which are based on other ranked set sampling procedures and the best linear unbiased estimator using all order statistics.     

Adaptive patient enrichment designs in therapeutic trials

The utility of clinical trial designs with adaptive patient enrichment is investigated in an adequate and well‐controlled trial setting. The overall treatment effect is the weighted average of the treatment effects in the mutually exclusive subsets of the originally intended entire study population. The adaptive enrichment approaches permit assessment of treatment effect that may be applicable to specific nested patient (sub)sets due to heterogeneous patient characteristics and/or differential response to treatment, e.g. a responsive patient subset versus a lack of beneficial patient subset, in all patient (sub)sets studied. The adaptive enrichment approaches considered include three adaptive design scenarios: (i) total sample size fixed and with futility stopping, (ii) sample size adaptation and futility stopping, and (iii) sample size adaptation without futility stopping. We show that regardless of whether the treatment effect eventually assessed is applicable to the originally studied patient population or only to the nested patient subsets; it is possible to devise an adaptive enrichment approach that statistically outperforms one‐size‐fits‐all fixed design approach and the fixed design with a pre‐specified multiple test procedure. We emphasize the need of additional studies to replicate the finding of a treatment effect in an enriched patient subset. The replication studies are likely to need fewer number of patients because of an identified treatment effect size that is larger than the diluted overall effect size. The adaptive designs, when applicable, are along the line of efficiency consideration in a drug development program.     

Cluster Randomization Trials: A Simulation Study

A simulation study is conducted to compare several methods that test the common log odds ratio in multiple 2 × 2 tables when the data are correlated within clusters. Allowing cluster size to vary within each table, we evaluate the unadjusted Mantel‐Haenszel chi‐square statistic (χ²_MH), the adjusted Mantel‐Haenszel chi‐square statistics of Rao and Scott using both an unpooled design effect (χ²_RSN) and a pooled design effect (χ²_RSP), the adjusted Mantel‐Haenszel chi‐square statistic of Donald and Donner (χ²_DD), the chi‐square statistic using the GEE approach (χ²_GEE), the adjusted Mantel‐Haenszel chi‐square statistic of Begg (χ²_B), the Wald (χ²_W), the robust Wald (χ²_RW), the score (χ²_S), the robust score (χ²_RS), and the adjusted Mantel‐Haenszel chi‐square statistics of Zhang and Boos (χ²_ZBP and χ²_ZBN). The test statistics above are compared in terms of empirical significance levels and empirical power levels. The robust score statistic χ²_RS and the adjusted Mantel‐Haenszel chi‐square statistics of Zhang and Boos (χ²_ZBP and χ²_ZBN) generally have empirical significance levels closer to the nominal value than the other statistics. These three statistics have similar empirical power levels when the intracluster correlation is zero or the cluster sizes are balanced. χ²_RS performs better in terms of empirical power levels when a positive intracluster correlation exists in the imbalance setting.     

An Investigation on the Allelic Chi‐Square Test Used in Genetic Association Studies

Case‐control studies are primary study designs used in genetic association studies. Sasieni (Biometrics 1997, 53, 1253–1261) pointed out that the allelic chi‐square test used in genetic association studies is invalid when Hardy‐Weinberg equilibrium (HWE) is violated in a combined population. It is important to know how much type I error rate is deviated from the nominal level under violated HWE. We examine bounds of type I error rate of the allelic chi‐square test. We also investigate power of the goodness‐of‐fit test for HWE which can be used as a guideline for selecting an appropriate test between the allelic chi‐square test and the modified allelic chi‐square test, the latter of which was proposed for cases of violated HWE. In small samples, power is not large enough to detect the Wright's inbreeding model of small values of inbreeding coefficient. Therefore, when the null hypothesis of HWE is barely accepted, the modified test should be considered as an alternative method. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Comparing two independent binomial proportions by a modified chi square test

When using the chi square test to compare the proportions of successes in two independent binomial samples, a new continuity correction is proposed, which equals half the size of the smaller sample. Exact computations of unconditional tail-probabilities, together with a theoretical argument, show that this correction is more appropriate than the YATES correction. This version of the chi square test is still conservative, but less so, and as a consequence more powerful, than the YATES corrected test.     

Statistical and energetic analysis of side-chain conformations in oligopeptides

The distributions of side-chain conformations in 258 crystal structures of oligopeptides have been analyzed. The sample contains 321 residues having side chains that extend beyond the C beta atom. Statistically observed preferences of side-chain dihedral angles are summarized and correlated with stereochemical and energetic constraints. The distributions are compared with observed distributions in proteins of known X-ray structures and with computed minimum-energy conformations of amino acid derivatives. The distributions are similar in all three sets of data, and they appear to be governed primarily by intraresidue interactions. In side chains with no beta-branching, the most important interactions that determine chi 1 are those between the C gamma H2 group and atoms of the neighboring peptide groups. As a result, the g- conformation (chi 1 congruent to -60 degrees) occurs most frequently for rotation around the C alpha-C beta bond in oligopeptides, followed by the t conformation (chi 1 congruent to 180 degrees), while the g+ conformation (chi 1 congruent to 60 degrees) is least favored. In residues with beta-branching, steric repulsions between the C gamma H2 or C gamma H3 groups and backbone atoms govern the distribution of chi 1. The extended (t) conformation is highly favored for rotation around the C beta-C gamma and C gamma-C delta bonds in unbranched side chains, because the t conformer has a lower energy than the g+ and g- conformers in hydrocarbon chains. This study of the observed side-chain conformations has led to a refinement of one of the energy parameters used in empirical conformational energy computations.     

Sample Size in the Study of Behaviour

The choice of an appropriate sample size for a study is a notoriously neglected topic in behavioural research, even though it is of utmost importance and the rules of action are more than clear – or are they? They may be clear if a formal power analysis is concerned. However, with the educated guesswork usually applied in behavioural studies there are various trade‐offs, and the degrees of freedom are extensive. An analysis of 119 original studies haphazardly chosen from five leading behavioural journals suggests that the selected sample size reflects an influence of constraints more often than a rational optimization process. As predicted, field work involves greater samples than studies conducted in captivity, and invertebrates are used in greater numbers than vertebrates when the approach is similar. However, it seems to be less important for determining the number of subjects if the study employs observational or experimental means. This is surprising because in contrast to mere observations, experiments allow to reduce random variation in the data, which is an essential precondition for economizing on sample size. By pointing to inconsistent patterns the intention of this article is to induce thought and discussion among behavioural researchers on this crucial issue, where apparently neither standard procedures are applied nor conventions have yet been established. This is an issue of concern for authors, referees and editors alike.     

Is the Exact Test Better than the Asymptotic Test for Testing Marginal Homogeneity in 2 × 2 Tables?

McNemar test is commonly used to test for the marginal homogeneity in 2 × 2 contingency tables. McNemar test is an asymptotic test based either on standard normal distribution or on the chi‐square distribution. When the total sample size is small, an exact version of McNemar test is available based on the binomial probabilities. The example in the paper came from a clinical study to investigate the effect of epidermal growth factor for children who had microvillus inclusion diseases. There were only six observations available. The test results differ between the exact test and the asymptotic test. It is a common belief that with this small sample size the exact test be used. However, we claim that McNemar test performs better than the exact test even when the sample size is small. In order to investigate the performances of McNemar test and the exact test, we identify the parameters that affect the test results and then perform sensitivity analysis. In addition, through Monte Carlo simulation studies we compare the empirical sizes and powers of these tests as well as other asymptotic tests such as Wald test and the likelihood ratio test.     

Homogeneity/Heterogeneity Hypotheses for Standardized Mortality Ratios Based on Minimum Power‐divergence Estimators

This paper analyzes the power divergence estimators when homogeneity/heterogeneity hypotheses among standardized mortality ratios (SMRs) are taken into account. A Monte Carlo study shows that when the standard mortality rate is not external, that is it is estimated from the sample data, these estimators have a good performance even for small sample sets and in particular the minimum chi‐square estimators have a better behavior compared to the classical maximum likelihood estimators. In order to make decisions under homogeneity/heterogeneity hypotheses of SMRs we propose some test‐statistics which consider the minimum power divergence estimators. Through a numerical example focused on SMRs of melanoma mortality ratios in different regions of the US, a homogeneity/heterogeneity study is illustrated.     

Accounting for missing data in the estimation of contemporary genetic effective population size (Ne)

Theoretical models are often applied to population genetic data sets without fully considering the effect of missing data. Researchers can deal with missing data by removing individuals that have failed to yield genotypes and/or by removing loci that have failed to yield allelic determinations, but despite their best efforts, most data sets still contain some missing data. As a consequence, realized sample size differs among loci, and this poses a problem for unbiased methods that must explicitly account for random sampling error. One commonly used solution for the calculation of contemporary effective population size (N_e) is to calculate the effective sample size as an unweighted mean or harmonic mean across loci. This is not ideal because it fails to account for the fact that loci with different numbers of alleles have different information content. Here we consider this problem for genetic estimators of contemporary effective population size (N_e). To evaluate bias and precision of several statistical approaches for dealing with missing data, we simulated populations with known N_e and various degrees of missing data. Across all scenarios, one method of correcting for missing data (fixed‐inverse variance‐weighted harmonic mean) consistently performed the best for both single‐sample and two‐sample (temporal) methods of estimating N_e and outperformed some methods currently in widespread use. The approach adopted here may be a starting point to adjust other population genetics methods that include per‐locus sample size components.     

GEM: a Gaussian Evolutionary Method for predicting protein side-chain conformations

We have developed an evolutionary approach to predicting protein side-chain conformations. This approach, referred to as the Gaussian Evolutionary Method (GEM), combines both discrete and continuous global search mechanisms. The former helps speed up convergence by reducing the size of rotamer space, whereas the latter, integrating decreasing-based Gaussian mutations and self-adaptive Gaussian mutations, continuously adapts dihedrals to optimal conformations. We tested our approach on 38 proteins ranging in size from 46 to 325 residues and showed that the results were comparable to those using other methods. The average accuracies of our predictions were 80% for chi(1), 66% for chi(1 + 2), and 1.36 A for the root mean square deviation of side-chain positions. We found that if our scoring function was perfect, the prediction accuracy was also essentially perfect. However, perfect prediction could not be achieved if only a discrete search mechanism was applied. These results suggest that GEM is robust and can be used to examine the factors limiting the accuracy of protein side-chain prediction methods. Furthermore, it can be used to systematically evaluate and thus improve scoring functions.     

Plot shape effects on plant species diversity measurements

Abstract. Question: Do rectangular sample plots record more plant species than square plots as suggested by both empirical and theoretical studies? Location: Grasslands, shrublands and forests in the Mediterranean‐climate region of California, USA. Methods: We compared three 0.1‐ha sampling designs that differed in the shape and dispersion of 1‐m² and 100‐m² nested subplots. We duplicated an earlier study that compared the Whittaker sample design, which had square clustered subplots, with the modified Whittaker design, which had dispersed rectangular subplots. To sort out effects of dispersion from shape we used a third design that overlaid square subplots on the modified Whittaker design. Also, using data from published studies we extracted species richness values for 400‐m² subplots that were either square or 1:4 rectangles partially overlaid on each other from desert scrub in high and low rainfall years, chaparral, sage scrub, oak savanna and coniferous forests with and without fire. Results: We found that earlier empirical reports of more than 30% greater richness with rectangles were due to the confusion of shape effects with spatial effects, coupled with the use of cumulative number of species as the metric for comparison. Average species richness was not significantly different between square and 1:4 rectangular sample plots at either 1‐ or 100‐m². Pairwise comparisons showed no significant difference between square and rectangular samples in all but one vegetation type, and that one exhibited significantly greater richness with squares. Our three intensive study sites appear to exhibit some level of self‐similarity at the scale of 400 m², but, contrary to theoretical expectations, we could not detect plot shape effects on species richness at this scale. Conclusions: At the 0.1‐ha scale or lower there is no evidence that plot shape has predictable effects on number of species recorded from sample plots. We hypothesize that for the mediterranean‐climate vegetation types studied here, the primary reason that 1:4 rectangles do not sample greater species richness than squares is because species turnover varies along complex environmental gradients that are both parallel and perpendicular to the long axis of rectangular plots. Reports in the literature of much greater species richness recorded for highly elongated rectangular strips than for squares of the same area are not likely to be fair comparisons because of the dramatically different periphery/area ratio, which includes a much greater proportion of species that are using both above and below‐ground niche space outside the sample area.     

Blinded versus unblinded estimation of a correlation coefficient to inform interim design adaptations

Regulatory authorities require that the sample size of a confirmatory trial is calculated prior to the start of the trial. However, the sample size quite often depends on parameters that might not be known in advance of the study. Misspecification of these parameters can lead to under‐ or overestimation of the sample size. Both situations are unfavourable as the first one decreases the power and the latter one leads to a waste of resources. Hence, designs have been suggested that allow a re‐assessment of the sample size in an ongoing trial. These methods usually focus on estimating the variance. However, for some methods the performance depends not only on the variance but also on the correlation between measurements. We develop and compare different methods for blinded estimation of the correlation coefficient that are less likely to introduce operational bias when the blinding is maintained. Their performance with respect to bias and standard error is compared to the unblinded estimator. We simulated two different settings: one assuming that all group means are the same and one assuming that different groups have different means. Simulation results show that the naïve (one‐sample) estimator is only slightly biased and has a standard error comparable to that of the unblinded estimator. However, if the group means differ, other estimators have better performance depending on the sample size per group and the number of groups.     

Heterogeneous treatment effects in stratified clinical trials with time‐to‐event endpoints

When analyzing clinical trials with a stratified population, homogeneity of treatment effects is a common assumption in survival analysis. However, in the context of recent developments in clinical trial design, which aim to test multiple targeted therapies in corresponding subpopulations simultaneously, the assumption that there is no treatment‐by‐stratum interaction seems inappropriate. It becomes an issue if the expected sample size of the strata makes it unfeasible to analyze the trial arms individually. Alternatively, one might choose as primary aim to prove efficacy of the overall (targeted) treatment strategy. When testing for the overall treatment effect, a violation of the no‐interaction assumption renders it necessary to deviate from standard methods that rely on this assumption. We investigate the performance of different methods for sample size calculation and data analysis under heterogeneous treatment effects. The commonly used sample size formula by Schoenfeld is compared to another formula by Lachin and Foulkes, and to an extension of Schoenfeld's formula allowing for stratification. Beyond the widely used (stratified) Cox model, we explore the lognormal shared frailty model, and a two‐step analysis approach as potential alternatives that attempt to adjust for interstrata heterogeneity. We carry out a simulation study for a trial with three strata and violations of the no‐interaction assumption. The extension of Schoenfeld's formula to heterogeneous strata effects provides the most reliable sample size with respect to desired versus actual power. The two‐step analysis and frailty model prove to be more robust against loss of power caused by heterogeneous treatment effects than the stratified Cox model and should be preferred in such situations.     

Discovering biogeographic and ecological clusters with a graph theoretic spin on factor analysis

Factor analysis (FA) has the advantage of highlighting each semi‐distinct cluster of samples in a data set with one axis at a time, as opposed to simply arranging samples across axes to represent gradients. However, in the case of presence–absence data it is confounded by absences when gradients are long. No statistical model can cope with this problem because the raw data simply do not present underlying information about the length of such gradients. Here I propose an easy way to tease out this information. It is a simple emendation of FA called stepping down, which involves giving an absence a negative value when the missing species nowhere co‐occurs with the species found in the relevant sample. Specifically, a binary co‐occurrence graph is created, and the magnitude of negative values is made a function of how far the graph must be traversed in order to link the missing species with each species that is present. Simulations show that standard FA yields inferior results to FA based on stepped‐down matrices in terms of mapping clusters into axes one‐by‐one. Standard FA is also uninformative when applied to a global bat inventory data set. Step‐down FA (SDFA) easily flags the main biogeographic groupings. Methods like correspondence analysis, non‐metric multidimensional scaling, and Bayesian latent variable modelling are not commensurate with SDFA because they do not seek to find a one‐to‐one mapping of axes and clusters. Stepping down seems promising as a means of illustrating clusters of samples, especially when there are subtle or complex discontinuities in gradients.     

Asymptotic distribution of the lod score for familial data

Using a linkage model with mixed parental mating types between a trait locus and a marker locus the asymptotic null distribution of the statistic U = 2ln(10)Z(theta) was stimulated and compared to the chi square type distribution function 1/2 + 1/2 Pr [chi 2(1) less than mu]. The stimulation results show that the chi square approximation fits the asymptotic null distribution well when both loci are predominated by either one of the two alleles at their loci, respectively, or the linkage phase tends to disequilibrium.