Cluster Randomization Trials: A Simulation Study

A simulation study is conducted to compare several methods that test the common log odds ratio in multiple 2 × 2 tables when the data are correlated within clusters. Allowing cluster size to vary within each table, we evaluate the unadjusted Mantel‐Haenszel chi‐square statistic (χ²_MH), the adjusted Mantel‐Haenszel chi‐square statistics of Rao and Scott using both an unpooled design effect (χ²_RSN) and a pooled design effect (χ²_RSP), the adjusted Mantel‐Haenszel chi‐square statistic of Donald and Donner (χ²_DD), the chi‐square statistic using the GEE approach (χ²_GEE), the adjusted Mantel‐Haenszel chi‐square statistic of Begg (χ²_B), the Wald (χ²_W), the robust Wald (χ²_RW), the score (χ²_S), the robust score (χ²_RS), and the adjusted Mantel‐Haenszel chi‐square statistics of Zhang and Boos (χ²_ZBP and χ²_ZBN). The test statistics above are compared in terms of empirical significance levels and empirical power levels. The robust score statistic χ²_RS and the adjusted Mantel‐Haenszel chi‐square statistics of Zhang and Boos (χ²_ZBP and χ²_ZBN) generally have empirical significance levels closer to the nominal value than the other statistics. These three statistics have similar empirical power levels when the intracluster correlation is zero or the cluster sizes are balanced. χ²_RS performs better in terms of empirical power levels when a positive intracluster correlation exists in the imbalance setting.     

Remarks on the conservatism of sigma(0 -- E)2/E in survival data.

This note presents theoretical and numerical calculations which investigate the adequacy of the sigma(0 -- E)2/E approximation (Peto and Pike [1973]) when applying the Mantel/Haenszel summary chisquare statistic to the comparison of r life tables in the manner suggested by Mantel [1966], Peto and Peto [1972], and Cox [1972]. These indicate that conservatism is mild unless there are marked differences in the withdrawal patterns in the r tables. Such differences may, however, be anticipated in the case of a generalization of the life table procedure suggested by Crowley [1973], Mantel and Byar [1974], and Turnbull, Brown, and Hu [1974], wherein individuals are moved from one life table to another according to changes in their treatment (or other) statuses.     

A Method of Moments Estimator for Random Effect Multivariate Meta‐Analysis

Summary Meta‐analysis is a powerful approach to combine evidence from multiple studies to make inference about one or more parameters of interest, such as regression coefficients. The validity of the fixed effect model meta‐analysis depends on the underlying assumption that all studies in the meta‐analysis share the same effect size. In the presence of heterogeneity, the fixed effect model incorrectly ignores the between‐study variance and may yield false positive results. The random effect model takes into account both within‐study and between‐study variances. It is more conservative than the fixed effect model and should be favored in the presence of heterogeneity. In this paper, we develop a noniterative method of moments estimator for the between‐study covariance matrix in the random effect model multivariate meta‐analysis. To our knowledge, it is the first such method of moments estimator in the matrix form. We show that our estimator is a multivariate extension of DerSimonian and Laird’s univariate method of moments estimator, and it is invariant to linear transformations. In the simulation study, our method performs well when compared to existing random effect model multivariate meta‐analysis approaches. We also apply our method in the analysis of a real data example.     

Design and Analysis of Crossover Trials for Absorbing Binary Endpoints

Summary The crossover is a popular and efficient trial design used in the context of patient heterogeneity to assess the effect of treatments that act relatively quickly and whose benefit disappears with discontinuation. Each patient can serve as her own control as within‐individual treatment and placebo responses are compared. Conventional wisdom is that these designs are not appropriate for absorbing binary endpoints, such as death or HIV infection. We explore the use of crossover designs in the context of these absorbing binary endpoints and show that they can be more efficient than the standard parallel group design when there is heterogeneity in individuals' risks. We also introduce a new two‐period design where first period “survivors” are rerandomized for the second period. This design combines the crossover design with the parallel design and achieves some of the efficiency advantages of the crossover design while ensuring that the second period groups are comparable by randomization. We discuss the validity of the new designs and evaluate both a mixture model and a modified Mantel–Haenszel test for inference. The mixture model assumes no carryover or period effects while the Mantel–Haenszel approach conditions out period effects. Simulations are used to compare the different designs and an example is provided to explore practical issues in implementation.     

Describing Ordinal Odds Ratios for Stratified r × c Tables

For an r × ctable with ordinal responses, odds ratios are commonly used to describe the relationship between the row and column variables. This article shows two types of ordinal odds ratios where local‐global odds ratios are used to compare several groups on a c‐category ordinal response and a global odds ratio is used to measure the global association between a pair of ordinal responses. When there is a stratification factor, we consider Mantel‐Haenszel (MH) type estimators of these odds ratios to summarize the association from several strata. Like the ordinary MH estimator of the common odds ratio for several 2 × 2 contingency tables, the estimators are used when the association is not expected to vary drastically among the strata. Also, the estimators are consistent under the ordinary asymptotic framework in which the number of strata is fixed and also under sparse asymptotics in which the number of strata grows with the sample size. Compared to the maximum likelihood estimators, simulations find that the MH type estimators perform better especially when each stratum has few observations. This article provides variances and covariances formulae for the local‐global odds ratios estimators and applies the bootstrap method to obtain a standard error for the global odds ratio estimator. At the end, we discuss possible ways of testing the homogeneity assumption.     

Multivariate meta‐analysis with an increasing number of parameters

Meta‐analysis can average estimates of multiple parameters, such as a treatment's effect on multiple outcomes, across studies. Univariate meta‐analysis (UVMA) considers each parameter individually, while multivariate meta‐analysis (MVMA) considers the parameters jointly and accounts for the correlation between their estimates. The performance of MVMA and UVMA has been extensively compared in scenarios with two parameters. Our objective is to compare the performance of MVMA and UVMA as the number of parameters, p, increases. Specifically, we show that (i) for fixed‐effect (FE) meta‐analysis, the benefit from using MVMA can substantially increase as p increases; (ii) for random effects (RE) meta‐analysis, the benefit from MVMA can increase as p increases, but the potential improvement is modest in the presence of high between‐study variability and the actual improvement is further reduced by the need to estimate an increasingly large between study covariance matrix; and (iii) when there is little to no between‐study variability, the loss of efficiency due to choosing RE MVMA over FE MVMA increases as p increases. We demonstrate these three features through theory, simulation, and a meta‐analysis of risk factors for non‐Hodgkin lymphoma.     

The analysis of stratified 2 x 2 contingency tables

We consider the problem of testing for independence against the consistent superiority of one treatment over another when the response variable is binary and is compared across two treatments in each of several strata. Specifically, we consider the randomized clinical trial setting. A number of issues arise in this context. First, should tables be combined if there are small or zero margins? Second, should one assume a common odds ratio across strata? Third, if the odds ratios differ across strata, then how does the standard test (based on a common odds ratio) perform? Fourth, are there other analyzes that are more appropriate for handling a situation in which the odds ratios may differ across strata? In addressing these issues we find that the frequently used Cochran–Mantel–Haenszel test may have a poor power profile, despite being optimal when the odds ratios are common. We develop novel tests that are analogous to the Smirnov, modified Smirnov, convex hull, and adaptive tests that have been proposed for ordered categorical data. (© 2006 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Inference for binomial probability based on dependent Bernoulli random variables with applications to meta‐analysis and group level studies

We study bias arising as a result of nonlinear transformations of random variables in random or mixed effects models and its effect on inference in group‐level studies or in meta‐analysis. The findings are illustrated on the example of overdispersed binomial distributions, where we demonstrate considerable biases arising from standard log‐odds and arcsine transformations of the estimated probability , both for single‐group studies and in combining results from several groups or studies in meta‐analysis. Our simulations confirm that these biases are linear in ρ, for small values of ρ, the intracluster correlation coefficient. These biases do not depend on the sample sizes or the number of studies K in a meta‐analysis and result in abysmal coverage of the combined effect for large K. We also propose bias‐correction for the arcsine transformation. Our simulations demonstrate that this bias‐correction works well for small values of the intraclass correlation. The methods are applied to two examples of meta‐analyses of prevalence.     

Random-effects meta-analysis models for the odds ratio in the case of rare events under different data-generating models: A simulation study

Meta-analysis of binary data is challenging when the event under investigation is rare, and standard models for random-effects meta-analysis perform poorly in such settings. In this simulation study, we investigate the performance of different random-effects meta-analysis models in terms of point and interval estimation of the pooled log odds ratio in rare events meta-analysis. First and foremost, we evaluate the performance of a hypergeometric-normal model from the family of generalized linear mixed models (GLMMs), which has been recommended, but has not yet been thoroughly investigated for rare events meta-analysis. Performance of this model is compared to performance of the beta-binomial model, which yielded favorable results in previous simulation studies, and to the performance of models that are frequently used in rare events meta-analysis, such as the inverse variance model and the Mantel–Haenszel method. In addition to considering a large number of simulation parameters inspired by real-world data settings, we study the comparative performance of the meta-analytic models under two different data-generating models (DGMs) that have been used in past simulation studies. The results of this study show that the hypergeometric-normal GLMM is useful for meta-analysis of rare events when moderate to large heterogeneity is present. In addition, our study reveals important insights with regard to the performance of the beta-binomial model under different DGMs from the binomial-normal family. In particular, we demonstrate that although misalignment of the beta-binomial model with the DGM affects its performance, it shows more robustness to the DGM than its competitors.     

Stratified Fisher's exact test and its sample size calculation

Chi‐squared test has been a popular approach to the analysis of a 2 × 2 table when the sample sizes for the four cells are large. When the large sample assumption does not hold, however, we need an exact testing method such as Fisher's test. When the study population is heterogeneous, we often partition the subjects into multiple strata, so that each stratum consists of homogeneous subjects and hence the stratified analysis has an improved testing power. While Mantel–Haenszel test has been widely used as an extension of the chi‐squared test to test on stratified 2 × 2 tables with a large‐sample approximation, we have been lacking an extension of Fisher's test for stratified exact testing. In this paper, we discuss an exact testing method for stratified 2 × 2 tables that is simplified to the standard Fisher's test in single 2 × 2 table cases, and propose its sample size calculation method that can be useful for designing a study with rare cell frequencies.     

Bivariate random‐effects meta‐analysis models for diagnostic test accuracy studies using arcsine‐based transformations

Diagnostic or screening tests are widely used in medical fields to classify patients according to their disease status. Several statistical models for meta‐analysis of diagnostic test accuracy studies have been developed to synthesize test sensitivity and specificity of a diagnostic test of interest. Because of the correlation between test sensitivity and specificity, modeling the two measures using a bivariate model is recommended. In this paper, we extend the current standard bivariate linear mixed model (LMM) by proposing two variance‐stabilizing transformations: the arcsine square root and the Freeman–Tukey double arcsine transformation. We compared the performance of the proposed methods with the standard method through simulations using several performance measures. The simulation results showed that our proposed methods performed better than the standard LMM in terms of bias, root mean square error, and coverage probability in most of the scenarios, even when data were generated assuming the standard LMM. We also illustrated the methods using two real data sets.     

Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data

The problem of combining information from separate trials is a key consideration when performing a meta‐analysis or planning a multicentre trial. Although there is a considerable journal literature on meta‐analysis based on individual patient data (IPD), i.e. a one‐step IPD meta‐analysis, versus analysis based on summary data, i.e. a two‐step IPD meta‐analysis, recent articles in the medical literature indicate that there is still confusion and uncertainty as to the validity of an analysis based on aggregate data. In this study, we address one of the central statistical issues by considering the estimation of a linear function of the mean, based on linear models for summary data and for IPD. The summary data from a trial is assumed to comprise the best linear unbiased estimator, or maximum likelihood estimator of the parameter, along with its covariance matrix. The setup, which allows for the presence of random effects and covariates in the model, is quite general and includes many of the commonly employed models, for example, linear models with fixed treatment effects and fixed or random trial effects. For this general model, we derive a condition under which the one‐step and two‐step IPD meta‐analysis estimators coincide, extending earlier work considerably. The implications of this result for the specific models mentioned above are illustrated in detail, both theoretically and in terms of two real data sets, and the roles of balance and heterogeneity are highlighted. Our analysis also shows that when covariates are present, which is typically the case, the two estimators coincide only under extra simplifying assumptions, which are somewhat unrealistic in practice.     

Effects of sample size,number of markers,and allelic richness on the detection of spatial genetic pattern

The influence of study design on the ability to detect the effects of landscape pattern on gene flow is one of the most pressing methodological gaps in landscape genetic research. To investigate the effect of study design on landscape genetics inference, we used a spatially‐explicit, individual‐based program to simulate gene flow in a spatially continuous population inhabiting a landscape with gradual spatial changes in resistance to movement. We simulated a wide range of combinations of number of loci, number of alleles per locus and number of individuals sampled from the population. We assessed how these three aspects of study design influenced the statistical power to successfully identify the generating process among competing hypotheses of isolation‐by‐distance, isolation‐by‐barrier, and isolation‐by‐landscape resistance using a causal modelling approach with partial Mantel tests. We modelled the statistical power to identify the generating process as a response surface for equilibrium and non‐equilibrium conditions after introduction of isolation‐by‐landscape resistance. All three variables (loci, alleles and sampled individuals) affect the power of causal modelling, but to different degrees. Stronger partial Mantel r correlations between landscape distances and genetic distances were found when more loci were used and when loci were more variable, which makes comparisons of effect size between studies difficult. Number of individuals did not affect the accuracy through mean equilibrium partial Mantel r, but larger samples decreased the uncertainty (increasing the precision) of equilibrium partial Mantel r estimates. We conclude that amplifying more (and more variable) loci is likely to increase the power of landscape genetic inferences more than increasing number of individuals.     

Performance of the Peto odds ratio compared to the usual odds ratio estimator in the case of rare events

For the calculation of relative measures such as risk ratio (RR) and odds ratio (OR) in a single study, additional approaches are required for the case of zero events. In the case of zero events in one treatment arm, the Peto odds ratio (POR) can be calculated without continuity correction, and is currently the relative effect estimation method of choice for binary data with rare events. The aim of this simulation study is a variegated comparison of the estimated OR and estimated POR with the true OR in a single study with two parallel groups without confounders in data situations where the POR is currently recommended. This comparison was performed by means of several performance measures, that is the coverage, confidence interval (CI) width, mean squared error (MSE), and mean percentage error (MPE). We demonstrated that the estimator for the POR does not outperform the estimator for the OR for all the performance measures investigated. In the case of rare events, small treatment effects and similar group sizes, we demonstrated that the estimator for the POR performed better than the estimator for the OR only regarding the coverage and MPE, but not the CI width and MSE. For larger effects and unbalanced group size ratios, the coverage and MPE of the estimator for the POR were inappropriate. As in practice the true effect is unknown, the POR method should be applied only with the utmost caution.     

Complementary Log–Log Regression for the Estimation of Covariate‐Adjusted Prevalence Ratios in the Analysis of Data from Cross‐Sectional Studies

We assessed complementary log–log (CLL) regression as an alternative statistical model for estimating multivariable‐adjusted prevalence ratios (PR) and their confidence intervals. Using the delta method, we derived an expression for approximating the variance of the PR estimated using CLL regression. Then, using simulated data, we examined the performance of CLL regression in terms of the accuracy of the PR estimates, the width of the confidence intervals, and the empirical coverage probability, and compared it with results obtained from log–binomial regression and stratified Mantel–Haenszel analysis. Within the range of values of our simulated data, CLL regression performed well, with only slight bias of point estimates of the PR and good confidence interval coverage. In addition, and importantly, the computational algorithm did not have the convergence problems occasionally exhibited by log–binomial regression. The technique is easy to implement in SAS (SAS Institute, Cary, NC), and it does not have the theoretical and practical issues associated with competing approaches. CLL regression is an alternative method of binomial regression that warrants further assessment.     

Assessment of identity disequilibrium and its relation to empirical heterozygosity fitness correlations: a meta‐analysis

Heterozygosity fitness correlations (HFCs) have frequently been used to detect inbreeding depression, under the assumption that genome‐wide heterozygosity is a good proxy for inbreeding. However, meta‐analyses of the association between fitness measures and individual heterozygosity have shown that often either no correlations are observed or the effect sizes are small. One of the reasons for this may be the absence of variance in inbreeding, a requisite for generating general‐effect HFCs. Recent work has highlighted identity disequilibrium (ID) as a measure that may capture variance in the level of inbreeding within a population; however, no thorough assessment of ID in natural populations has been conducted. In this meta‐analysis, we assess the magnitude of ID (as measured by the g₂ statistic) from 50 previously published HFC studies and its relationship to the observed effect sizes of those studies. We then assess how much power the studies had to detect general‐effect HFCs, and the number of markers that would have been needed to generate a high expected correlation (r² = 0.9) between observed heterozygosity and inbreeding. Across the majority of studies, g₂ values were not significantly different than zero. Despite this, we found that the magnitude of g₂ was associated with the average effect sizes observed in a population, even when point estimates were nonsignificant. These low values of g₂ translated into low expected correlations between heterozygosity and inbreeding and suggest that many more markers than typically used are needed to robustly detect HFCs.     

A stage‐wise approach for the analysis of multi‐environment trials

Plant breeders and variety testing agencies routinely test candidate genotypes (crop varieties, lines, test hybrids) in multiple environments. Such multi‐environment trials can be efficiently analysed by mixed models. A single‐stage analysis models the entire observed data at the level of individual plots. This kind of analysis is usually considered as the gold standard. In practice, however, it is more convenient to use a two‐stage approach, in which experiments are first analysed per environment, yielding adjusted means per genotype, which are then summarised across environments in the second stage. Stage‐wise approaches suggested so far are approximate in that they cannot fully reproduce a single‐stage analysis, except in very simple cases, because the variance–covariance matrix of adjusted means from individual environments needs to be approximated by a diagonal matrix. This paper proposes a fully efficient stage‐wise method, which carries forward the full variance–covariance matrix of adjusted means from the individual environments to the analysis across the series of trials. Provided the variance components are known, this method can fully reproduce the results of a single‐stage analysis. Computations are made efficient by a diagonalisation of the residual variance–covariance matrix, which necessitates a corresponding linear transformation of both the first‐stage estimates (e.g. adjusted means and regression slopes for plot covariates) and the corresponding design matrices for fixed and random effects. We also exemplify the extension of the general approach to a three‐stage analysis. The method is illustrated using two datasets, one real and the other simulated. The proposed approach has close connections with meta‐analysis, where environments correspond to centres and genotypes to medical treatments. We therefore compare our theoretical results with recently published results from a meta‐analysis.     

Bayesian Meta‐Analysis of the Accuracy of a Test for Tuberculous Pleuritis in the Absence of a Gold Standard Reference

Summary Absence of a perfect reference test is an acknowledged source of bias in diagnostic studies. In the case of tuberculous pleuritis, standard reference tests such as smear microscopy, culture and biopsy have poor sensitivity. Yet meta‐analyses of new tests for this disease have always assumed the reference standard is perfect, leading to biased estimates of the new test’s accuracy. We describe a method for joint meta‐analysis of sensitivity and specificity of the diagnostic test under evaluation, while considering the imperfect nature of the reference standard. We use a Bayesian hierarchical model that takes into account within‐ and between‐study variability. We show how to obtain pooled estimates of sensitivity and specificity, and how to plot a hierarchical summary receiver operating characteristic curve. We describe extensions of the model to situations where multiple reference tests are used, and where index and reference tests are conditionally dependent. The performance of the model is evaluated using simulations and illustrated using data from a meta‐analysis of nucleic acid amplification tests (NAATs) for tuberculous pleuritis. The estimate of NAAT specificity was higher and the sensitivity lower compared to a model that assumed that the reference test was perfect.     

Soil carbon stock change following afforestation in Northern Europe: a meta‐analysis

Northern Europe supports large soil organic carbon (SOC) pools and has been subjected to high frequency of land‐use changes during the past decades. However, this region has not been well represented in previous large‐scale syntheses of land‐use change effects on SOC, especially regarding effects of afforestation. Therefore, we conducted a meta‐analysis of SOC stock change following afforestation in Northern Europe. Response ratios were calculated for forest floors and mineral soils (0–10 cm and 0–20/30 cm layers) based on paired control (former land use) and afforested plots. We analyzed the influence of forest age, former land‐use, forest type, and soil textural class. Three major improvements were incorporated in the meta‐analysis: analysis of major interaction groups, evaluation of the influence of nonindependence between samples according to study design, and mass correction. Former land use was a major factor contributing to changes in SOC after afforestation. In former croplands, SOC change differed between soil layers and was significantly positive (20%) in the 0–10 cm layer. Afforestation of former grasslands had a small negative (nonsignificant) effect indicating limited SOC change following this land‐use change within the region. Forest floors enhanced the positive effects of afforestation on SOC, especially with conifers. Meta‐estimates calculated for the periods <30 years and >30 years since afforestation revealed a shift from initial loss to later gain of SOC. The interaction group analysis indicated that meta‐estimates in former land‐use, forest type, and soil textural class alone were either offset or enhanced when confounding effects among variable classes were considered. Furthermore, effect sizes were slightly overestimated if sample dependence was not accounted for and if no mass correction was performed. We conclude that significant SOC sequestration in Northern Europe occurs after afforestation of croplands and not grasslands, and changes are small within a 30‐year perspective.