Testing small study effects in multivariate meta-analysis

Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias, and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection processes can be involved in the reporting of multivariate outcomes. For example, some studies may be completely unpublished while others may selectively report multiple outcomes. In this paper, we propose a score test as an overall test of small study effects in multivariate meta-analysis. Two detailed case studies are given to demonstrate the advantage of the proposed test over various naive applications of univariate tests in practice. Through simulation studies, the proposed test is found to retain nominal Type I error rates with considerable power in moderate sample size settings. Finally, we also evaluate the concordance between the proposed tests with the naive application of univariate tests by evaluating 44 systematic reviews with multiple outcomes from the Cochrane Database.     

The impact of imputation on meta-analysis of genome-wide association studies

Genotype imputation is often used in the meta-analysis of genome-wide association studies (GWAS), for combining data from different studies and/or genotyping platforms, in order to improve the ability for detecting disease variants with small to moderate effects. However, how genotype imputation affects the performance of the meta-analysis of GWAS is largely unknown. In this study, we investigated the effects of genotype imputation on the performance of meta-analysis through simulations based on empirical data from the Framingham Heart Study. We found that when fix-effects models were used, considerable between-study heterogeneity was detected when causal variants were typed in only some but not all individual studies, resulting in up to ～25% reduction of detection power. For certain situations, the power of the meta-analysis can be even less than that of individual studies. Additional analyses showed that the detection power was slightly improved when between-study heterogeneity was partially controlled through the random-effects model, relative to that of the fixed-effects model. Our study may aid in the planning, data analysis, and interpretation of GWAS meta-analysis results when genotype imputation is necessary.     

The interactive effects of plant microbial symbionts: a review and meta-analysis

In nature, plants often associate with multiple symbionts concurrently, yet the effects of tripartite symbioses are not well understood. We expected synergistic growth responses from plants associating with functionally distinct symbionts. In contrast, symbionts providing similar benefits to a host may reduce host plant growth. We reviewed studies investigating the effect of multiple interactions on host plant performance. Additionally, we conducted a meta-analysis on the studies that performed controlled manipulations of the presence of two microbial symbionts. Using response ratios, we investigated the effects on plants of pairs of symbionts (mycorrhizal fungi, fungal endophytes, and nitrogen-fixers). The results did not support the view that arbuscular mycorrhizal (AM) fungi and rhizobia should interact synergistically. In contrast, we found the joint effects of fungal endophytes and arbuscular mycorrhizal fungi to be greater than expected given their independent effects. This increase in plant performance only held for antagonistic endophytes, whose negative effects were alleviated when in association with AM fungi, while the impact of beneficial endophytes was not altered by infection with AM fungi. Generalizations from the meta-analysis were limited by the substantial variation within types of interactions and the data available, highlighting the need for more research on a range of plant systems.     

How Often Does an Individual Trial Agree with Its Corresponding Meta-Analysis? A Meta-Epidemiologic Study

Objective

A meta-analysis may provide more conclusive results than a single trial. The major cost of meta-analysis is the time of waiting before the meta-analysis becomes available and resources spent on consequent trials that may not be necessary. The objective of this study is to address how often the result of a single trial, in particular the first trial, differs from that of its corresponding meta-analysis so as to reduce unnecessary waiting time and subsequent trials.

Study Design and Settings

A meta-epidemiologic study was conducted by collecting meta-analyses from the Cochrane Database of Systematic Reviews and five major medical journals. Effect size of a single trial was compared with that of its corresponding meta-analysis. The single trial includes the first trial, last trial and any trial randomly selected from the meta-analysis.

Results

647 meta-analyses are included and the median number of trials in a meta-analysis is 5. 233 (36.0%) meta-analyses have the first trial with a statistically significant result. When the first trial is statistically significant, 84.1% (95% CI: 79.4%, 88.8%) of the corresponding meta-analyses is both in the same direction and statistically significant. When the first trial is statistically insignificant, 57.9% (95% CI: 53.2%, 62.8%) of the meta-analysis is also statistically insignificant regardless of direction. The median number of years is 6.5 years from the first to the 5^th trial.

Conclusion

The conclusion of the first trial that the treatment is effective or harmful is mostly likely correct. A statistically significant trial agrees more often with its corresponding meta-analysis than a large trial. These findings imply that particularly in some urgent, life-saving or other critical circumstances for which no other effective methods are available, cautious recommendation based on the significant result of the first trial seems justifiable and could start use of an effective intervention by 5–8 years earlier.     

General Framework for Meta-analysis of Rare Variants in Sequencing Association Studies

We propose a general statistical framework for meta-analysis of gene- or region-based multimarker rare variant association tests in sequencing association studies. In genome-wide association studies, single-marker meta-analysis has been widely used to increase statistical power by combining results via regression coefficients and standard errors from different studies. In analysis of rare variants in sequencing studies, region-based multimarker tests are often used to increase power. We propose meta-analysis methods for commonly used gene- or region-based rare variants tests, such as burden tests and variance component tests. Because estimation of regression coefficients of individual rare variants is often unstable or not feasible, the proposed method avoids this difficulty by calculating score statistics instead that only require fitting the null model for each study and then aggregating these score statistics across studies. Our proposed meta-analysis rare variant association tests are conducted based on study-specific summary statistics, specifically score statistics for each variant and between-variant covariance-type (linkage disequilibrium) relationship statistics for each gene or region. The proposed methods are able to incorporate different levels of heterogeneity of genetic effects across studies and are applicable to meta-analysis of multiple ancestry groups. We show that the proposed methods are essentially as powerful as joint analysis by directly pooling individual level genotype data. We conduct extensive simulations to evaluate the performance of our methods by varying levels of heterogeneity across studies, and we apply the proposed methods to meta-analysis of rare variant effects in a multicohort study of the genetics of blood lipid levels.     

Saw Palmetto Berry as a Treatment for BPH

Phytotherapeutic agents are often prescribed in Europe for the treatment of benign prostatic hyperplasia with lower urinary tract symptoms and are commonly used in the United States in over-the-counter preparations. Saw palmetto berry is the most popular of these agents, and in vitro some studies suggest that liposterolic extract of the plant has antiandrogenic effects that inhibit the type 1 and type 2 isoenzymes of 5alpha-reductase; however there are no clinical studies that show any decrease in serum dihydrotestosterone or prostate-specific antigen. Its efficacy in the treatment of lower urinary tract symptoms has not been conclusively proven. Clinical efficacy was suggested by a meta-analysis of Permixon, a formulation of saw palmetto, but the meta-analysis was done on suboptimal studies. One trial supports the equivalency of Permixon to finasteride in treating moderate to severe symptoms of benign prostatic hyperplasia, with less decrease in sexual function. However, without a control/placebo arm, the actual efficacy of the agents cannot be determined. Other than occasional gastrointestinal upset, no other side effects have been reported.     

Design,Analysis, and Reporting of Crossover Trials for Inclusion in a Meta-Analysis

Randomized crossover trials are clinical experiments in which participants are assigned randomly to a sequence of treatments and each participant serves as his/her own control in estimating treatment effect. We need a better understanding of the validity of their results to enable recommendations as to which crossover trials can be included in meta-analysis and for development of reporting guidelines.

Objective

To evaluate the characteristics of the design, analysis, and reporting of crossover trials for inclusion in a meta-analysis of treatment for primary open-angle glaucoma and to provide empirical evidence to inform the development of tools to assess the validity of the results from crossover trials and reporting guidelines.

Methods

We searched MEDLINE, EMBASE, and Cochrane’s CENTRAL register for randomized crossover trials for a systematic review and network meta-analysis we are conducting. Two individuals independently screened the search results for eligibility and abstracted data from each included report.

Results

We identified 83 crossover trials eligible for inclusion. Issues affecting the risk of bias in crossover trials, such as carryover, period effects and missing data, were often ignored. Some trials failed to accommodate the within-individual differences in the analysis. For a large proportion of the trials, the authors tabulated the results as if they arose from a parallel design. Precision estimates properly accounting for the paired nature of the design were often unavailable from the study reports; consequently, to include trial findings in a meta-analysis would require further manipulation and assumptions.

Conclusions

The high proportion of poorly reported analyses and results has the potential to affect whether crossover data should or can be included in a meta-analysis. There is pressing need for reporting guidelines for crossover trials.     

A Multi-Trait,Meta-analysis for Detecting Pleiotropic Polymorphisms for Stature,Fatness and Reproduction in Beef Cattle

Polymorphisms that affect complex traits or quantitative trait loci (QTL) often affect multiple traits. We describe two novel methods (1) for finding single nucleotide polymorphisms (SNPs) significantly associated with one or more traits using a multi-trait, meta-analysis, and (2) for distinguishing between a single pleiotropic QTL and multiple linked QTL. The meta-analysis uses the effect of each SNP on each of n traits, estimated in single trait genome wide association studies (GWAS). These effects are expressed as a vector of signed t-values (t) and the error covariance matrix of these t values is approximated by the correlation matrix of t-values among the traits calculated across the SNP (V). Consequently, t''V⁻¹t is approximately distributed as a chi-squared with n degrees of freedom. An attractive feature of the meta-analysis is that it uses estimated effects of SNPs from single trait GWAS, so it can be applied to published data where individual records are not available. We demonstrate that the multi-trait method can be used to increase the power (numbers of SNPs validated in an independent population) of GWAS in a beef cattle data set including 10,191 animals genotyped for 729,068 SNPs with 32 traits recorded, including growth and reproduction traits. We can distinguish between a single pleiotropic QTL and multiple linked QTL because multiple SNPs tagging the same QTL show the same pattern of effects across traits. We confirm this finding by demonstrating that when one SNP is included in the statistical model the other SNPs have a non-significant effect. In the beef cattle data set, cluster analysis yielded four groups of QTL with similar patterns of effects across traits within a group. A linear index was used to validate SNPs having effects on multiple traits and to identify additional SNPs belonging to these four groups.     

Elevated yolk androgen levels and the expression of multiple sexually selected male characters

Maternal hormones in bird eggs modulate not only offspring development, but recently it has also been shown that these effects can persist into adult life. A number of long-lasting effects concern traits of which the expression or development is modulated by androgens. This suggests that the nature of yolk hormone-mediated maternal effects may be organizational. Maternal androgens may therefore play an important role in sexual selection, since the expression of sexually selected male characters is often androgen-dependent.We experimentally manipulated the yolk androgen concentrations of European Starling (Sturnus vulgaris) eggs. Subsequently we followed 49 unrelated males from hatching until year of first reproduction. We investigated the expression of multiple sexually selected male characters (song, beak color and throat feather characteristics), taking into account whether a trait is androgen-dependent.Elevated levels of yolk androgens affected the length of the embryonic period, but did not modify the expression of either androgen-dependent or androgen-independent sexually selected male characters including song phenotype at adulthood. Thus the most important function of yolk androgens in starlings and possibly other bird species may relate to the early developmental period. The outcome of our study together with the results of our meta-analysis indicates that the effects of yolk androgens on sexually selected male characters may be comparatively small. Our results suggest that this may relate to the numerous other environmental and/or genetic factors influencing the expression of sexually selected male characters.     

Cell Specific eQTL Analysis without Sorting Cells

The functional consequences of trait associated SNPs are often investigated using expression quantitative trait locus (eQTL) mapping. While trait-associated variants may operate in a cell-type specific manner, eQTL datasets for such cell-types may not always be available. We performed a genome-environment interaction (GxE) meta-analysis on data from 5,683 samples to infer the cell type specificity of whole blood cis-eQTLs. We demonstrate that this method is able to predict neutrophil and lymphocyte specific cis-eQTLs and replicate these predictions in independent cell-type specific datasets. Finally, we show that SNPs associated with Crohn’s disease preferentially affect gene expression within neutrophils, including the archetypal NOD2 locus.     

A subset-based approach improves power and interpretation for the combined analysis of genetic association studies of heterogeneous traits

Pooling genome-wide association studies (GWASs) increases power but also poses methodological challenges because studies are often heterogeneous. For example, combining GWASs of related but distinct traits can provide promising directions for the discovery of loci with small but common pleiotropic effects. Classical approaches for meta-analysis or pooled analysis, however, might not be suitable for such analysis because individual variants are likely to be associated with only a subset of the traits or might demonstrate effects in different directions. We propose a method that exhaustively explores subsets of studies for the presence of true association signals that are in either the same direction or possibly opposite directions. An efficient approximation is used for rapid evaluation of p values. We present two illustrative applications, one for a meta-analysis of separate case-control studies of six distinct cancers and another for pooled analysis of a case-control study of glioma, a class of brain tumors that contains heterogeneous subtypes. Both the applications and additional simulation studies demonstrate that the proposed methods offer improved power and more interpretable results when compared to traditional methods for the analysis of heterogeneous traits. The proposed framework has applications beyond genetic association studies.     

The Contribution of Systematic Review and Meta-Analysis Methods to Human Health Risk Assessment: Neurobehavioral Effects of Manganese

Current methods of human health risk assessment may lack transparency in respect of identification, review, and synthesis of potentially relevant human and animal evidence. The nature, degree, and source of uncertainties are often unclear. This article aims to demonstrate the contribution that systematic review and meta-analysis methods can make to providing more structured, transparent, and systematic risk assessments. We focus on disparities between five risk assessments for neurobehavioral effects of manganese, and then illustrate advantages of a systematic approach. Fifty-five human epidemiological studies and 37 animal experiments were identified. Where appropriate, meta-analysis methods demonstrated consistent adverse effects associated with manganese exposure across species. In particular, there was reduced activity in subjects exposed to manganese, although exposed rats tended to be more active than controls. Limitations of exposure measurement and reporting restricted use of more quantitative methods of evidence synthesis. From a methodological viewpoint, we conclude that systematic review and meta-analysis methods can contribute to a more systematic and transparent human health risk assessment making more efficient use of available evidence, compared to current methods of risk assessment. More complex methods could encompass further differences between relevant studies and so further improve the risk assessment process.     

RankAggreg, an R package for weighted rank aggregation

Background  

Researchers in the field of bioinformatics often face a challenge of combining several ordered lists in a proper and efficient manner. Rank aggregation techniques offer a general and flexible framework that allows one to objectively perform the necessary aggregation. With the rapid growth of high-throughput genomic and proteomic studies, the potential utility of rank aggregation in the context of meta-analysis becomes even more apparent. One of the major strengths of rank-based aggregation is the ability to combine lists coming from different sources and platforms, for example different microarray chips, which may or may not be directly comparable otherwise.     

Systematic reviews: a primer for plastic surgery research

Clinicians rely on review articles to keep current with the rapid accumulation of medical and surgical literature. Traditional expert reviews, however, often suffer from inherent personal biases and may not reflect a true synthesis of the existing literature on a particular subject. Systematic reviews are structured, scientific articles that address the shortcomings of traditional reviews by adhering to strict, reproducible methods and recommended guidelines. The methods are designed to eliminate possible sources of bias, ensure as complete a review of the existing literature as possible, and present the results in a way that is useful for its intended audience. Systematic reviews may at times include a quantitative synthesis of the available data in the form of a meta-analysis. Meta-analysis is a statistical tool for combining the numerical results of separate studies to obtain a summary outcome with increased precision due to the larger, combined number of patients. Meta-analyses may be particularly helpful when individual study results are conflicting and the existing literature is inconclusive. The validity of meta-analysis, however, is highly dependent on the quality of data available in the literature. In its strictest form, meta-analysis is used to combine data from only randomized controlled clinical trials. Because randomized controlled clinical trials are infrequently performed in plastic surgery research, this article will focus on systematic reviews to provide the readers with a useful guide in performing this field of study.     

Meta-analysis of gene expression microarrays with missing replicates

Background  

Many different microarray experiments are publicly available today. It is natural to ask whether different experiments for the same phenotypic conditions can be combined using meta-analysis, in order to increase the overall sample size. However, some genes are not measured in all experiments, hence they cannot be included or their statistical significance cannot be appropriately estimated in traditional meta-analysis. Nonetheless, these genes, which we refer to as incomplete genes, may also be informative and useful.     

Meta-Analysis Identifies Gene-by-Environment Interactions as Demonstrated in a Study of 4,965 Mice

Identifying environmentally-specific genetic effects is a key challenge in understanding the structure of complex traits. Model organisms play a crucial role in the identification of such gene-by-environment interactions, as a result of the unique ability to observe genetically similar individuals across multiple distinct environments. Many model organism studies examine the same traits but under varying environmental conditions. For example, knock-out or diet-controlled studies are often used to examine cholesterol in mice. These studies, when examined in aggregate, provide an opportunity to identify genomic loci exhibiting environmentally-dependent effects. However, the straightforward application of traditional methodologies to aggregate separate studies suffers from several problems. First, environmental conditions are often variable and do not fit the standard univariate model for interactions. Additionally, applying a multivariate model results in increased degrees of freedom and low statistical power. In this paper, we jointly analyze multiple studies with varying environmental conditions using a meta-analytic approach based on a random effects model to identify loci involved in gene-by-environment interactions. Our approach is motivated by the observation that methods for discovering gene-by-environment interactions are closely related to random effects models for meta-analysis. We show that interactions can be interpreted as heterogeneity and can be detected without utilizing the traditional uni- or multi-variate approaches for discovery of gene-by-environment interactions. We apply our new method to combine 17 mouse studies containing in aggregate 4,965 distinct animals. We identify 26 significant loci involved in High-density lipoprotein (HDL) cholesterol, many of which are consistent with previous findings. Several of these loci show significant evidence of involvement in gene-by-environment interactions. An additional advantage of our meta-analysis approach is that our combined study has significantly higher power and improved resolution compared to any single study thus explaining the large number of loci discovered in the combined study.     

Life history of the harlequin ladybird, <Emphasis Type="Italic">Harmonia axyridis</Emphasis>: a global meta-analysis

The harlequin ladybird, Harmonia axyridis, is an important natural enemy of aphids throughout the world, but is now also considered an invasive alien species. We performed a meta-analysis of published life history data to address the question whether invading populations in Europe and North America have life history parameters that differ from native populations in Asia, explaining the beetle’s invasion success in new territories. In this meta-analysis, we accounted for important covariables that are often reported in published studies such as temperature, food source (aphids or eggs of Ephestia kuehniella), strain (laboratory or field populations) and photoperiod. Temperature was a key factor having consistent large effects on development rate, survival and reproductive characteristics of H. axyridis. Food source, strain, and photoperiod had effects on some, but not all characteristics, and their overall effect across characteristics was minor. Individuals of invasive populations had a shorter pre-oviposition period and higher fecundity at low temperatures than those of native populations, and a greater longevity across all temperatures. No differences in survival were found between native and invasive populations, while differences in development rate were not consistent, with opposing results obtained according to the way development rate was measured in trials reported in the literature. Results of this meta-analysis support the hypothesis that the life history of the beetle has changed during its invasion into North America and Europe. Invasive populations had a shorter pre-oviposition period and higher fecundity at low temperatures, as well as a greater longevity across all temperatures than native populations. These differences may partially explain the invasive success of H. axyridis.     

Meta-analysis of studies with missing data

Summary .  Consider a meta-analysis of studies with varying proportions of patient-level missing data, and assume that each primary study has made certain missing data adjustments so that the reported estimates of treatment effect size and variance are valid. These estimates of treatment effects can be combined across studies by standard meta-analytic methods, employing a random-effects model to account for heterogeneity across studies. However, we note that a meta-analysis based on the standard random-effects model will lead to biased estimates when the attrition rates of primary studies depend on the size of the underlying study-level treatment effect. Perhaps ignorable within each study, these types of missing data are in fact not ignorable in a meta-analysis. We propose three methods to correct the bias resulting from such missing data in a meta-analysis: reweighting the DerSimonian–Laird estimate by the completion rate; incorporating the completion rate into a Bayesian random-effects model; and inference based on a Bayesian shared-parameter model that includes the completion rate. We illustrate these methods through a meta-analysis of 16 published randomized trials that examined combined pharmacotherapy and psychological treatment for depression.     

A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses

Background

Heterogeneity has a key role in meta-analysis methods and can greatly affect conclusions. However, true levels of heterogeneity are unknown and often researchers assume homogeneity. We aim to: a) investigate the prevalence of unobserved heterogeneity and the validity of the assumption of homogeneity; b) assess the performance of various meta-analysis methods; c) apply the findings to published meta-analyses.

Methods and Findings

We accessed 57,397 meta-analyses, available in the Cochrane Library in August 2012. Using simulated data we assessed the performance of various meta-analysis methods in different scenarios. The prevalence of a zero heterogeneity estimate in the simulated scenarios was compared with that in the Cochrane data, to estimate the degree of unobserved heterogeneity in the latter. We re-analysed all meta-analyses using all methods and assessed the sensitivity of the statistical conclusions. Levels of unobserved heterogeneity in the Cochrane data appeared to be high, especially for small meta-analyses. A bootstrapped version of the DerSimonian-Laird approach performed best in both detecting heterogeneity and in returning more accurate overall effect estimates. Re-analysing all meta-analyses with this new method we found that in cases where heterogeneity had originally been detected but ignored, 17–20% of the statistical conclusions changed. Rates were much lower where the original analysis did not detect heterogeneity or took it into account, between 1% and 3%.

Conclusions

When evidence for heterogeneity is lacking, standard practice is to assume homogeneity and apply a simpler fixed-effect meta-analysis. We find that assuming homogeneity often results in a misleading analysis, since heterogeneity is very likely present but undetected. Our new method represents a small improvement but the problem largely remains, especially for very small meta-analyses. One solution is to test the sensitivity of the meta-analysis conclusions to assumed moderate and large degrees of heterogeneity. Equally, whenever heterogeneity is detected, it should not be ignored.