Designing and analyzing clinical trials with composite outcomes: consideration of possible treatment differences between the individual outcomes

When the individual outcomes within a composite outcome appear to have different treatment effects, either in magnitude or direction, researchers may question the validity or appropriateness of using this composite outcome as a basis for measuring overall treatment effect in a randomized controlled trial. The question remains as to how to distinguish random variation in estimated treatment effects from important heterogeneity within a composite outcome. This paper suggests there may be some utility in directly testing the assumption of homogeneity of treatment effect across the individual outcomes within a composite outcome. We describe a treatment heterogeneity test for composite outcomes based on a class of models used for the analysis of correlated data arising from the measurement of multiple outcomes for the same individuals. Such a test may be useful in planning a trial with a primary composite outcome and at trial end with final analysis and presentation. We demonstrate how to determine the statistical power to detect composite outcome treatment heterogeneity using the POISE Trial data. Then we describe how this test may be incorporated into a presentation of trial results with composite outcomes. We conclude that it may be informative for trialists to assess the consistency of treatment effects across the individual outcomes within a composite outcome using a formalized methodology and the suggested test represents one option.     

A comparison of strategies for Markov chain Monte Carlo computation in quantitative genetics

In quantitative genetics, Markov chain Monte Carlo (MCMC) methods are indispensable for statistical inference in non-standard models like generalized linear models with genetic random effects or models with genetically structured variance heterogeneity. A particular challenge for MCMC applications in quantitative genetics is to obtain efficient updates of the high-dimensional vectors of genetic random effects and the associated covariance parameters. We discuss various strategies to approach this problem including reparameterization, Langevin-Hastings updates, and updates based on normal approximations. The methods are compared in applications to Bayesian inference for three data sets using a model with genetically structured variance heterogeneity.     

Low-Order Non-Spatial Effects Dominate Second-Order Spatial Effects in the Texture Quantifier Analysis of 18F-FDG-PET Images

BackgroundThere is increasing interest in applying image texture quantifiers to assess the intra-tumor heterogeneity observed in FDG-PET images of various cancers. Use of these quantifiers as prognostic indicators of disease outcome and/or treatment response has yielded inconsistent results. We study the general applicability of some well-established texture quantifiers to the image data unique to FDG-PET.MethodsWe first created computer-simulated test images with statistical properties consistent with clinical image data for cancers of the uterine cervix. We specifically isolated second-order statistical effects from low-order effects and analyzed the resulting variation in common texture quantifiers in response to contrived image variations. We then analyzed the quantifiers computed for FIGOIIb cervical cancers via receiver operating characteristic (ROC) curves and via contingency table analysis of detrended quantifier values.ResultsWe found that image texture quantifiers depend strongly on low-effects such as tumor volume and SUV distribution. When low-order effects are controlled, the image texture quantifiers tested were not able to discern only the second-order effects. Furthermore, the results of clinical tumor heterogeneity studies might be tunable via choice of patient population analyzed.ConclusionSome image texture quantifiers are strongly affected by factors distinct from the second-order effects researchers ostensibly seek to assess via those quantifiers.     

Density-dependence by habitat heterogeneity: individual quality versus territory quality

The aim of this comment is to review the ecological issues concerning the role of individual and habitat heterogeneity as possible mechanisms explaining density-dependent fecundity in animal populations. Our intention is to discuss different approaches to determine whether or not studied populations are subjected to density-dependent processes and which mechanisms are involved. We show that because individual quality (e.g. measured in terms of age of breeding individuals) and territory quality can be correlated, untangling both effects on fecundity is frequently a difficult task. We discuss the misuse of statistical methods and other problems related to the specific characteristics of the studied populations that can have a strong influence on the conclusions reached by researchers working in this field.     

Application of random effects to the study of resource selection by animals

1. Resource selection estimated by logistic regression is used increasingly in studies to identify critical resources for animal populations and to predict species occurrence. 2. Most frequently, individual animals are monitored and pooled to estimate population-level effects without regard to group or individual-level variation. Pooling assumes that both observations and their errors are independent, and resource selection is constant given individual variation in resource availability. 3. Although researchers have identified ways to minimize autocorrelation, variation between individuals caused by differences in selection or available resources, including functional responses in resource selection, have not been well addressed. 4. Here we review random-effects models and their application to resource selection modelling to overcome these common limitations. We present a simple case study of an analysis of resource selection by grizzly bears in the foothills of the Canadian Rocky Mountains with and without random effects. 5. Both categorical and continuous variables in the grizzly bear model differed in interpretation, both in statistical significance and coefficient sign, depending on how a random effect was included. We used a simulation approach to clarify the application of random effects under three common situations for telemetry studies: (a) discrepancies in sample sizes among individuals; (b) differences among individuals in selection where availability is constant; and (c) differences in availability with and without a functional response in resource selection. 6. We found that random intercepts accounted for unbalanced sample designs, and models with random intercepts and coefficients improved model fit given the variation in selection among individuals and functional responses in selection. Our empirical example and simulations demonstrate how including random effects in resource selection models can aid interpretation and address difficult assumptions limiting their generality. This approach will allow researchers to appropriately estimate marginal (population) and conditional (individual) responses, and account for complex grouping, unbalanced sample designs and autocorrelation.     

The impact of stopping rules on heterogeneity of results in overviews of clinical trials.

This paper explores the extent to which application of statistical stopping rules in clinical trials can create an artificial heterogeneity of treatment effects in overviews (meta-analyses) of related trials. For illustration, we concentrate on overviews of identically designed group sequential trials, using either fixed nominal or O'Brien and Fleming two-sided boundaries. Some analytic results are obtained for two-group designs and simulation studies are otherwise used, with the following overall findings. The use of stopping rules leads to biased estimates of treatment effect so that the assessment of heterogeneity of results in an overview of trials, some of which have used stopping rules, is confounded by this bias. If the true treatment effect being studied is small, as is often the case, then artificial heterogeneity is introduced, thus increasing the Type I error rate in the test of homogeneity. This could lead to erroneous use of a random effects model, producing exaggerated estimates and confidence intervals. However, if the true mean effect is large, then between-trial heterogeneity may be underestimated. When undertaking or interpreting overviews, one should ascertain whether stopping rules have been used (either formally or informally) and should consider whether their use might account for any heterogeneity found.     

Statistical inference for valued-edge networks: the generalized exponential random graph model

Across the sciences, the statistical analysis of networks is central to the production of knowledge on relational phenomena. Because of their ability to model the structural generation of networks based on both endogenous and exogenous factors, exponential random graph models are a ubiquitous means of analysis. However, they are limited by an inability to model networks with valued edges. We address this problem by introducing a class of generalized exponential random graph models capable of modeling networks whose edges have continuous values (bounded or unbounded), thus greatly expanding the scope of networks applied researchers can subject to statistical analysis.     

SIMEX variance component tests in generalized linear mixed measurement error models

In the analysis of clustered data with covariates measured with error, a problem of common interest is to test for correlation within clusters and heterogeneity across clusters. We examined this problem in the framework of generalized linear mixed measurement error models. We propose using the simulation extrapolation (SIMEX) method to construct a score test for the null hypothesis that all variance components are zero. A key feature of this SIMEX score test is that no assumptions need to be made regarding the distributions of the random effects and the unobserved covariates. We illustrate this test by analyzing Framingham heart disease data and evaluate its performance by simulation. We also propose individual SIMEX score tests for testing the variance components separately. Both tests can be easily implemented using existing statistical software.     

Classifying tissue samples from measurements on cells with within-class tissue sample heterogeneity

We consider here the problem of classifying a macro-level object based on measurements of embedded (micro-level) observations within each object, for example, classifying a patient based on measurements on a collection of a random number of their cells. Classification problems with this hierarchical, nested structure have not received the same statistical understanding as the general classification problem. Some heuristic approaches have been developed and a few authors have proposed formal statistical models. We focus on the problem where heterogeneity exists between the macro-level objects within a class. We propose a model-based statistical methodology that models the log-odds of the macro-level object belonging to a class using a latent-class variable model to account for this heterogeneity. The latent classes are estimated by clustering the macro-level object density estimates. We apply this method to the detection of patients with cervical neoplasia based on quantitative cytology measurements on cells in a Papanicolaou smear. Quantitative cytology is much cheaper and potentially can take less time than the current standard of care. The results show that the automated quantitative cytology using the proposed method is roughly equivalent to clinical cytopathology and shows significant improvement over a statistical model that does not account for the heterogeneity of the data.     

On the replication of genetic associations: timing can be everything!

The failure of researchers to replicate genetic-association findings is most commonly attributed to insufficient statistical power, population stratification, or various forms of between-study heterogeneity or environmental influences.(1) Here, we illustrate another potential cause for nonreplications that has so far not received much attention in the literature. We illustrate that the strength of a genetic effect can vary by age, causing "age-varying associations." If not taken into account during the design and the analysis of a study, age-varying genetic associations can cause nonreplication. By using the 100K SNP scan of the Framingham Heart Study, we identified an age-varying association between a SNP in ROBO1 and obesity and hypothesized an age-gene interaction. This finding was followed up in eight independent samples comprising 13,584 individuals. The association was replicated in five of the eight studies, showing an age-dependent relationship (one-sided combined p = 3.92 x 10(-9), combined p value from pediatric cohorts = 2.21 x 10(-8), combined p value from adult cohorts = 0.00422). Furthermore, this study illustrates that it is difficult for cross-sectional study designs to detect age-varying associations. If the specifics of age- or time-varying genetic effects are not considered in the selection of both the follow-up samples and in the statistical analysis, important genetic associations may be missed.     

Dispersal and individual quality in a long lived species

The idea of differences in individual quality has been put forward in numerous long-term studies in long-lived species to explain differences in lifetime production among individuals. Despite the important role of individual heterogeneity in vital rates in demography, population dynamics and life history theory, the idea of "individual quality" is elusive. It is sometimes assumed to be a static or dynamic individual characteristic. When considered as a dynamic trait, it is sometimes assumed to vary deterministically or stochastically, or to be confounded with the characteristics of the habitat. We addressed heterogeneity in reproductive performance among individuals established in higher-quality habitat in a long-lived seabird species. We used approaches to statistical inference based on individual random effects permitting quantification of heterogeneity in populations and assessment of individual variation from the population mean. We found evidence of heterogeneity in breeding probability, not success probability. We assessed the influence of dispersal on individual reproductive potential. Dispersal is likely to be destabilizing in species with high site and mate fidelity. We detected heterogeneity after dispersal, not before. Individuals may perform well regardless of quality before destabilization, including those that recruited in higher-quality habitat by chance, but only higher-quality individuals may be able to overcome the consequences of dispersal. Importantly, results differed when accounting for individual heterogeneity (an increase in mean breeding probability when individuals dispersed), or not (a decrease in mean breeding probability). In the latter case, the decrease in mean breeding probability may result from a substantial decrease in breeding probability in a few individuals and a slight increase in others. In other words, the pattern observed at the population mean level may not reflect what happens in the majority of individuals.     

Revisiting the Effect of Capture Heterogeneity on Survival Estimates in Capture-Mark-Recapture Studies: Does It Matter?

Recently developed capture-mark-recapture methods allow us to account for capture heterogeneity among individuals in the form of discrete mixtures and continuous individual random effects. In this article, we used simulations and two case studies to evaluate the effectiveness of continuously distributed individual random effects at removing potential bias due to capture heterogeneity, and to evaluate in what situation the added complexity of these models is justified. Simulations and case studies showed that ignoring individual capture heterogeneity generally led to a small negative bias in survival estimates and that individual random effects effectively removed this bias. As expected, accounting for capture heterogeneity also led to slightly less precise survival estimates. Our case studies also showed that accounting for capture heterogeneity increased in importance towards the end of study. Though ignoring capture heterogeneity led to a small bias in survival estimates, such bias may greatly impact management decisions. We advocate reducing potential heterogeneity at the sampling design stage. Where this is insufficient, we recommend modelling individual capture heterogeneity in situations such as when a large proportion of the individuals has a low detection probability (e.g. in the presence of floaters) and situations where the most recent survival estimates are of great interest (e.g. in applied conservation).     

Establishment of combined analytical method to extract the genes of interest from transcriptome data

Techniques for analyzing genome-wide expression profiles, such as the microarray technique and next-generation sequencers, have been developed. While these techniques can provide a lot of information about gene expression, selection of genes of interest is complicated because of excessive gene expression data. Thus, many researchers use statistical methods or fold change as screening tools for finding gene sets whose expression is altered between groups, which may result in the loss of important information. In the present study, we aimed to establish a combined method for selecting genes of interest with a small magnitude of alteration in gene expression by coupling with proteome analysis. We used hypercholesterolemic rats to examine the effects of a crude herbal drug on gene expression and proteome profiles. We could not select genes of interest by using standard methods. However, by coupling with proteome analysis, we found several effects of the crude herbal drug on gene expression. Our results suggest that this method would be useful in selecting gene sets with expressions that do not show a large magnitude of alteration.     

Familial Resemblance for Plasma Leptin: Sample Homogeneity across Adiposity and Ethnic Groups

Objective: Previous studies show a wide range in the percentage of variance in leptin levels attributable to genetic factors. These studies differ markedly with respect to ethnicity, study design, and statistical methodology. Therefore, the purpose of this study was to investigate heterogeneity hypotheses across ethnic groups and by adiposity level, using the same statistical methods. Research Methods and Procedures: Samples included black vs. white (HERITAGE Family Study) and random vs. obese (Québec Family Study) individuals from 432 families (1432 individuals). Heritability for leptin, alternatively adjusted for age and sex and then for age, sex, and adiposity was estimated with the use of familial correlations. Heterogeneity in the magnitude of the familial resemblance between samples and the effect of adjusting for adiposity was explored. Results: Heritability did not vary across samples stratified by adiposity level or ethnic group or across adjustment schemes. Maximal heritability, the percentage of additive phenotypic variability due to all familial sources, was 32%. Discussion: Whereas leptin and adiposity were highly correlated within individuals, removing the effects of adiposity did not significantly alter the magnitude of the familial component for leptin. Moreover, this effect did not vary as a function of ethnicity (black vs. white) or adiposity level. Thus, no evidence for heterogeneity was detected. However, a comparison among previous studies raises questions concerning possible genetic heterogeneity in other ethnic groups in which complex interactions among leptin, adiposity, and diabetes status may be important.     

Looking for a needle in a haystack: inference about individual fitness components in a heterogeneous population

Studies of wild vertebrates have provided evidence of substantial differences in lifetime reproduction among individuals and the sequences of life history ‘states’ during life (breeding, nonbreeding, etc.). Such differences may reflect ‘fixed’ differences in fitness components among individuals determined before, or at the onset of reproductive life. Many retrospective life history studies have translated this idea by assuming a ‘latent’ unobserved heterogeneity resulting in a fixed hierarchy among individuals in fitness components. Alternatively, fixed differences among individuals are not necessarily needed to account for observed levels of individual heterogeneity in life histories. Individuals with identical fitness traits may stochastically experience different outcomes for breeding and survival through life that lead to a diversity of ‘state’ sequences with some individuals living longer and being more productive than others, by chance alone. The question is whether individuals differ in their underlying fitness components in ways that cannot be explained by observable ‘states’ such as age, previous breeding success, etc. Here, we compare statistical models that represent these opposing hypotheses, and mixtures of them, using data from kittiwakes. We constructed models that accounted for observed covariates, individual random effects (unobserved heterogeneity), first‐order Markovian transitions between observed states, or combinations of these features. We show that individual sequences of states are better accounted for by models incorporating unobserved heterogeneity than by models including first‐order Markov processes alone, or a combination of both. If we had not considered individual heterogeneity, models including Markovian transitions would have been the best performing ones. We also show that inference about age‐related changes in fitness components is sensitive to incorporation of underlying individual heterogeneity in models. Our approach provides insight into the sources of individual heterogeneity in life histories, and can be applied to other data sets to examine the ubiquity of our results across the tree of life.     

黄土高原小流域土壤养分的空间异质性

利用地理信息系统的空间分析功能,通过地统计学的半变异函数定量研究了黄土高原典型小流域土壤养分的空间异质性特征。结果表明;土壤有机质,全氮,有效氮,全磷和有效磷的理论模型均为球状模型,由随机因素引起的空间变异占空间总变异的比例小,其值分别为13.333%,10.938%,22.000%,9.091%和27.536%,反映5种养分具有较强的空间自相关格局,但它们的空间自相关范围具有明显的差异。土壤全氮和有效氮变程小,分别是90m和110m,有机质次之,变程是120m,而土壤全磷和有效磷的变程最大为160m,研究成果将有效地指导土壤的取样设计,以及进行土壤养分的空间内插和制图。     

A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses

Background

Heterogeneity has a key role in meta-analysis methods and can greatly affect conclusions. However, true levels of heterogeneity are unknown and often researchers assume homogeneity. We aim to: a) investigate the prevalence of unobserved heterogeneity and the validity of the assumption of homogeneity; b) assess the performance of various meta-analysis methods; c) apply the findings to published meta-analyses.

Methods and Findings

We accessed 57,397 meta-analyses, available in the Cochrane Library in August 2012. Using simulated data we assessed the performance of various meta-analysis methods in different scenarios. The prevalence of a zero heterogeneity estimate in the simulated scenarios was compared with that in the Cochrane data, to estimate the degree of unobserved heterogeneity in the latter. We re-analysed all meta-analyses using all methods and assessed the sensitivity of the statistical conclusions. Levels of unobserved heterogeneity in the Cochrane data appeared to be high, especially for small meta-analyses. A bootstrapped version of the DerSimonian-Laird approach performed best in both detecting heterogeneity and in returning more accurate overall effect estimates. Re-analysing all meta-analyses with this new method we found that in cases where heterogeneity had originally been detected but ignored, 17–20% of the statistical conclusions changed. Rates were much lower where the original analysis did not detect heterogeneity or took it into account, between 1% and 3%.

Conclusions

When evidence for heterogeneity is lacking, standard practice is to assume homogeneity and apply a simpler fixed-effect meta-analysis. We find that assuming homogeneity often results in a misleading analysis, since heterogeneity is very likely present but undetected. Our new method represents a small improvement but the problem largely remains, especially for very small meta-analyses. One solution is to test the sensitivity of the meta-analysis conclusions to assumed moderate and large degrees of heterogeneity. Equally, whenever heterogeneity is detected, it should not be ignored.     

Interpreting parameters in the logistic regression model with random effects

Logistic regression with random effects is used to study the relationship between explanatory variables and a binary outcome in cases with nonindependent outcomes. In this paper, we examine in detail the interpretation of both fixed effects and random effects parameters. As heterogeneity measures, the random effects parameters included in the model are not easily interpreted. We discuss different alternative measures of heterogeneity and suggest using a median odds ratio measure that is a function of the original random effects parameters. The measure allows a simple interpretation, in terms of well-known odds ratios, that greatly facilitates communication between the data analyst and the subject-matter researcher. Three examples from different subject areas, mainly taken from our own experience, serve to motivate and illustrate different aspects of parameter interpretation in these models.     

Bayesian nonparametric meta-analysis using Polya tree mixture models

Summary .   A common goal in meta-analysis is estimation of a single effect measure using data from several studies that are each designed to address the same scientific inquiry. Because studies are typically conducted in geographically disperse locations, recent developments in the statistical analysis of meta-analytic data involve the use of random effects models that account for study-to-study variability attributable to differences in environments, demographics, genetics, and other sources that lead to heterogeneity in populations. Stemming from asymptotic theory, study-specific summary statistics are modeled according to normal distributions with means representing latent true effect measures. A parametric approach subsequently models these latent measures using a normal distribution, which is strictly a convenient modeling assumption absent of theoretical justification. To eliminate the influence of overly restrictive parametric models on inferences, we consider a broader class of random effects distributions. We develop a novel hierarchical Bayesian nonparametric Polya tree mixture (PTM) model. We present methodology for testing the PTM versus a normal random effects model. These methods provide researchers a straightforward approach for conducting a sensitivity analysis of the normality assumption for random effects. An application involving meta-analysis of epidemiologic studies designed to characterize the association between alcohol consumption and breast cancer is presented, which together with results from simulated data highlight the performance of PTMs in the presence of nonnormality of effect measures in the source population.