Inference for meta‐analysis with a suspected temporal trend

There is sometimes a clear evidence of a strong secular trend in the treatment effect of studies included in a meta‐analysis. In such cases, estimating the present‐day treatment effect by meta‐regression is both reasonable and straightforward. We however consider the more common situation where a secular trend is suspected, but is not strongly statistically significant. Typically, this lack of significance is due to the small number of studies included in the analysis, so that a meta‐regression could give wild point estimates. We introduce an empirical Bayes meta‐analysis methodology, which shrinks the secular trend toward zero. This has the effect that treatment effects are adjusted for trend, but where the evidence from data is weak, wild results are not obtained. We explore several frequentist approaches and a fully Bayesian method is also implemented. A measure of trend analogous to I² is described, and exact significance tests for trend are given. Our preferred method is one based on penalized or h‐likelihood, which is computationally simple, and allows invariance of predictions to the (arbitrary) choice of time origin. We suggest that a trendless standard random effects meta‐analysis should routinely be supplemented with an h‐likelihood analysis as a sensitivity analysis.     

A Method of Moments Estimator for Random Effect Multivariate Meta‐Analysis

Summary Meta‐analysis is a powerful approach to combine evidence from multiple studies to make inference about one or more parameters of interest, such as regression coefficients. The validity of the fixed effect model meta‐analysis depends on the underlying assumption that all studies in the meta‐analysis share the same effect size. In the presence of heterogeneity, the fixed effect model incorrectly ignores the between‐study variance and may yield false positive results. The random effect model takes into account both within‐study and between‐study variances. It is more conservative than the fixed effect model and should be favored in the presence of heterogeneity. In this paper, we develop a noniterative method of moments estimator for the between‐study covariance matrix in the random effect model multivariate meta‐analysis. To our knowledge, it is the first such method of moments estimator in the matrix form. We show that our estimator is a multivariate extension of DerSimonian and Laird’s univariate method of moments estimator, and it is invariant to linear transformations. In the simulation study, our method performs well when compared to existing random effect model multivariate meta‐analysis approaches. We also apply our method in the analysis of a real data example.     

A matrix‐based method of moments for fitting the multivariate random effects model for meta‐analysis and meta‐regression

Multivariate meta‐analysis is becoming more commonly used. Methods for fitting the multivariate random effects model include maximum likelihood, restricted maximum likelihood, Bayesian estimation and multivariate generalisations of the standard univariate method of moments. Here, we provide a new multivariate method of moments for estimating the between‐study covariance matrix with the properties that (1) it allows for either complete or incomplete outcomes and (2) it allows for covariates through meta‐regression. Further, for complete data, it is invariant to linear transformations. Our method reduces to the usual univariate method of moments, proposed by DerSimonian and Laird, in a single dimension. We illustrate our method and compare it with some of the alternatives using a simulation study and a real example.     

Enabling regional management in a changing climate through Bayesian meta‐analysis of a large‐scale disturbance

Aim Quantifying and predicting change in large ecosystems is an important research objective for applied ecologists as human disturbance effects become increasingly evident at regional and global scales. However, studies used to make inferences about large‐scale change are frequently of uneven quality and few in number, having been undertaken to study local, rather than global, change. Our aim is to improve the quality of inferences that can be made in meta‐analyses of large‐scale disturbance by integrating studies of varying quality in a unified modelling framework that is informative for both local and regional management. Innovation Here we improve conventionally structured meta‐analysis methods by including imputation of unknown study variances and the use of Bayesian factor potentials. The approach is a coherent framework for integrating data of varying quality across multiple studies while facilitating belief statements about the uncertainty in parameter estimates and the probable outcome of future events. The approach is applied to a regional meta‐analysis of the effects of loss of coral cover on species richness and the abundance of coral‐dependent fishes in the western Indian Ocean (WIO) before and after a mass bleaching event in 1998. Main conclusions Our Bayesian approach to meta‐analysis provided greater precision of parameter estimates than conventional weighted linear regression meta‐analytical techniques, allowing us to integrate all available data from 66 available study locations in the WIO across multiple scales. The approach thereby: (1) estimated uncertainty in site‐level estimates of change, (2) provided a regional estimate for future change at any given site in the WIO, and (3) provided a probabilistic belief framework for future management of reef resources at both local and regional scales.     

Formalizing the definition of meta‐analysis in Molecular Ecology

Meta‐analysis, the statistical synthesis of pertinent literature to develop evidence‐based conclusions, is relatively new to the field of molecular ecology, with the first meta‐analysis published in the journal Molecular Ecology in 2003 (Slate & Phua 2003). The goal of this article is to formalize the definition of meta‐analysis for the authors, editors, reviewers and readers of Molecular Ecology by completing a review of the meta‐analyses previously published in this journal. We also provide a brief overview of the many components required for meta‐analysis with a more specific discussion of the issues related to the field of molecular ecology, including the use and statistical considerations of Wright's F_ST and its related analogues as effect sizes in meta‐analysis. We performed a literature review to identify articles published as ‘meta‐analyses’ in Molecular Ecology, which were then evaluated by at least two reviewers. We specifically targeted Molecular Ecology publications because as a flagship journal in this field, meta‐analyses published in Molecular Ecology have the potential to set the standard for meta‐analyses in other journals. We found that while many of these reviewed articles were strong meta‐analyses, others failed to follow standard meta‐analytical techniques. One of these unsatisfactory meta‐analyses was in fact a secondary analysis. Other studies attempted meta‐analyses but lacked the fundamental statistics that are considered necessary for an effective and powerful meta‐analysis. By drawing attention to the inconsistency of studies labelled as meta‐analyses, we emphasize the importance of understanding the components of traditional meta‐analyses to fully embrace the strengths of quantitative data synthesis in the field of molecular ecology.     

Comparison of One‐Step and Two‐Step Meta‐Analysis Models Using Individual Patient Data

The problem of combining information from separate trials is a key consideration when performing a meta‐analysis or planning a multicentre trial. Although there is a considerable journal literature on meta‐analysis based on individual patient data (IPD), i.e. a one‐step IPD meta‐analysis, versus analysis based on summary data, i.e. a two‐step IPD meta‐analysis, recent articles in the medical literature indicate that there is still confusion and uncertainty as to the validity of an analysis based on aggregate data. In this study, we address one of the central statistical issues by considering the estimation of a linear function of the mean, based on linear models for summary data and for IPD. The summary data from a trial is assumed to comprise the best linear unbiased estimator, or maximum likelihood estimator of the parameter, along with its covariance matrix. The setup, which allows for the presence of random effects and covariates in the model, is quite general and includes many of the commonly employed models, for example, linear models with fixed treatment effects and fixed or random trial effects. For this general model, we derive a condition under which the one‐step and two‐step IPD meta‐analysis estimators coincide, extending earlier work considerably. The implications of this result for the specific models mentioned above are illustrated in detail, both theoretically and in terms of two real data sets, and the roles of balance and heterogeneity are highlighted. Our analysis also shows that when covariates are present, which is typically the case, the two estimators coincide only under extra simplifying assumptions, which are somewhat unrealistic in practice.     

A population‐averaged approach to diagnostic test meta‐analysis

The meta‐analysis of diagnostic accuracy studies is often of interest in screening programs for many diseases. The typical summary statistics for studies chosen for a diagnostic accuracy meta‐analysis are often two dimensional: sensitivities and specificities. The common statistical analysis approach for the meta‐analysis of diagnostic studies is based on the bivariate generalized linear‐mixed model (BGLMM), which has study‐specific interpretations. In this article, we present a population‐averaged (PA) model using generalized estimating equations (GEE) for making inference on mean specificity and sensitivity of a diagnostic test in the population represented by the meta‐analytic studies. We also derive the marginalized counterparts of the regression parameters from the BGLMM. We illustrate the proposed PA approach through two dataset examples and compare performance of estimators of the marginal regression parameters from the PA model with those of the marginalized regression parameters from the BGLMM through Monte Carlo simulation studies. Overall, both marginalized BGLMM and GEE with sandwich standard errors maintained nominal 95% confidence interval coverage levels for mean specificity and mean sensitivity in meta‐analysis of 25 of more studies even under misspecification of the covariance structure of the bivariate positive test counts for diseased and nondiseased subjects.     

Bayesian Meta‐Analysis of the Accuracy of a Test for Tuberculous Pleuritis in the Absence of a Gold Standard Reference

Summary Absence of a perfect reference test is an acknowledged source of bias in diagnostic studies. In the case of tuberculous pleuritis, standard reference tests such as smear microscopy, culture and biopsy have poor sensitivity. Yet meta‐analyses of new tests for this disease have always assumed the reference standard is perfect, leading to biased estimates of the new test’s accuracy. We describe a method for joint meta‐analysis of sensitivity and specificity of the diagnostic test under evaluation, while considering the imperfect nature of the reference standard. We use a Bayesian hierarchical model that takes into account within‐ and between‐study variability. We show how to obtain pooled estimates of sensitivity and specificity, and how to plot a hierarchical summary receiver operating characteristic curve. We describe extensions of the model to situations where multiple reference tests are used, and where index and reference tests are conditionally dependent. The performance of the model is evaluated using simulations and illustrated using data from a meta‐analysis of nucleic acid amplification tests (NAATs) for tuberculous pleuritis. The estimate of NAAT specificity was higher and the sensitivity lower compared to a model that assumed that the reference test was perfect.     

Case–Control Studies of Gene–Environment Interaction: Bayesian Design and Analysis

Summary With increasing frequency, epidemiologic studies are addressing hypotheses regarding gene‐environment interaction. In many well‐studied candidate genes and for standard dietary and behavioral epidemiologic exposures, there is often substantial prior information available that may be used to analyze current data as well as for designing a new study. In this article, first, we propose a proper full Bayesian approach for analyzing studies of gene–environment interaction. The Bayesian approach provides a natural way to incorporate uncertainties around the assumption of gene–environment independence, often used in such an analysis. We then consider Bayesian sample size determination criteria for both estimation and hypothesis testing regarding the multiplicative gene–environment interaction parameter. We illustrate our proposed methods using data from a large ongoing case–control study of colorectal cancer investigating the interaction of N‐acetyl transferase type 2 (NAT2) with smoking and red meat consumption. We use the existing data to elicit a design prior and show how to use this information in allocating cases and controls in planning a future study that investigates the same interaction parameters. The Bayesian design and analysis strategies are compared with their corresponding frequentist counterparts.     

Selection models with monotone weight functions in meta analysis

Publication bias, the fact that studies identified for inclusion in a meta analysis do not represent all studies on the topic of interest, is commonly recognized as a threat to the validity of a meta analysis. One way to explicitly model publication bias is via weighted probability distributions. We adopt the non-parametric approach initially introduced by Dear and Begg (1992) but impose that the weight function w is monotonely non-increasing as a function of the p-value. Since in meta analysis one typically only has few studies or "observations," regularization of the estimation problem seems sensible. In addition, virtually all parametric weight functions proposed so far in the literature are in fact decreasing. We discuss how to estimate a decreasing weight function in the above model and illustrate the new methodology on two well-known examples. Some basic properties of the log-likelihood function and computation of a p-value quantifying the evidence against the null hypothesis of a constant weight function are indicated. In addition, we provide an approximate selection bias adjusted profile likelihood confidence interval for the treatment effect. The corresponding software and the data sets used to illustrate it are provided as the R package selectMeta (Rufibach, 2011).     

Disease signatures are robust across tissues and experiments

Meta‐analyses combining gene expression microarray experiments offer new insights into the molecular pathophysiology of disease not evident from individual experiments. Although the established technical reproducibility of microarrays serves as a basis for meta‐analysis, pathophysiological reproducibility across experiments is not well established. In this study, we carried out a large‐scale analysis of disease‐associated experiments obtained from NCBI GEO, and evaluated their concordance across a broad range of diseases and tissue types. On evaluating 429 experiments, representing 238 diseases and 122 tissues from 8435 microarrays, we find evidence for a general, pathophysiological concordance between experiments measuring the same disease condition. Furthermore, we find that the molecular signature of disease across tissues is overall more prominent than the signature of tissue expression across diseases. The results offer new insight into the quality of public microarray data using pathophysiological metrics, and support new directions in meta‐analysis that include characterization of the commonalities of disease irrespective of tissue, as well as the creation of multi‐tissue systems models of disease pathology using public data.     

Using aggregate data to estimate the standard error of a treatment–covariate interaction in an individual patient data meta‐analysis

Subgroup analyses are important to medical research because they shed light on the heterogeneity of treatment effectts. A treatment–covariate interaction in an individual patient data (IPD) meta‐analysis is the most reliable means to estimate how a subgroup factor modifies a treatment's effectiveness. However, owing to the challenges in collecting participant data, an approach based on aggregate data might be the only option. In these circumstances, it would be useful to assess the relative efficiency and power loss of a subgroup analysis without patient‐level data. We present methods that use aggregate data to estimate the standard error of an IPD meta‐analysis’ treatment–covariate interaction for regression models of a continuous or dichotomous patient outcome. Numerical studies indicate that the estimators have good accuracy. An application to a previously published meta‐regression illustrates the practical utility of the methodology.     

Formalized synthesis opportunities for ecology: systematic reviews and meta‐analyses

Narrative reviews are dead. Long live systematic reviews (and meta‐analyses). Synthesis in many forms is now a driving force in ecology. Advances in open data for ecology and new tools provide vastly improved capacity for novel, emergent knowledge synthesis in our discipline. Systematic reviews and meta‐analyses are two formal synthesis opportunities for ecologists that are now accepted as traditional publications, but the scope of validated syntheses will continue to expand. To date, systematic reviews are rarely used whilst the rate of meta‐analyses published in ecological journals is increasing exponentially. Systematic reviews provide an overview of the literature landscape for a topic, and meta‐analyses examine the strength of evidence integrated across different studies. Effective synthesis benefits from both approaches, but better data reporting and additional advances in the culture of sharing data, code, analytics, workflows, methods and also ideas will further energize these efforts. At this junction, synthetic efforts that include systematic reviews and meta‐analyses should continue as stand‐alone publications. This is a necessary step in the evolution of synthesis in our discipline. Nonetheless, they are still evolving tools, and meta‐analyses in particular are simply an extended set of statistical tests. Admittedly, understanding the statistics and assumptions influence how we conduct synthesis much as statistical choices often shape experimental design, i.e. ANOVA versus regression‐based experiments, but statistics do not make the paper. Current steps – primary research articles need to more effectively report evidence, sharing scientific products should expand, systematic reviews should be used to identify research gaps/delineate literature landscapes, and meta‐analyses should be used to examine evidence patterns to further predictive ecology.     

Ignoring non‐English‐language studies may bias ecological meta‐analyses

Meta‐analysis plays a crucial role in syntheses of quantitative evidence in ecology and biodiversity conservation. The reliability of estimates in meta‐analyses strongly depends on unbiased sampling of primary studies. Although earlier studies have explored potential biases in ecological meta‐analyses, biases in reported statistical results and associated study characteristics published in different languages have never been tested in environmental sciences. We address this knowledge gap by systematically searching published meta‐analyses and comparing effect‐size estimates between English‐ and Japanese‐language studies included in existing meta‐analyses. Of the 40 published ecological meta‐analysis articles authored by those affiliated to Japanese institutions, we find that three meta‐analysis articles searched for studies in the two languages and involved sufficient numbers of English‐ and Japanese‐language studies, resulting in four eligible meta‐analyses (i.e., four meta‐analyses conducted in the three meta‐analysis articles). In two of the four, effect sizes differ significantly between the English‐ and Japanese‐language studies included in the meta‐analyses, causing considerable changes in overall mean effect sizes and even their direction when Japanese‐language studies are excluded. The observed differences in effect sizes are likely attributable to systematic differences in reported statistical results and associated study characteristics, particularly taxa and ecosystems, between English‐ and Japanese‐language studies. Despite being based on a small sample size, our findings suggest that ignoring non‐English‐language studies may bias outcomes of ecological meta‐analyses, due to systematic differences in study characteristics and effect‐size estimates between English‐ and non‐English languages. We provide a list of actions that meta‐analysts could take in the future to reduce the risk of language bias.     

A novel meta‐analytical approach to improve systematic review of rates and patterns of microevolution

A classic topic in ecology and evolution, phenotypic microevolution of quantitative traits has received renewed attention in the face of rapid global environmental change. However, for plants, synthesis has been hampered by the limited use of standard metrics, which makes it difficult to systematize empirical information. Here we demonstrate the advantages of incorporating meta‐analysis tools to the review of microevolutionary rates. We perform a systematic survey of the plant literature on microevolution of quantitative traits over known periods of time, based on the scopus database. We quantify the amount of change by standard mean difference and develop a set of effect sizes to analyze such data. We show that applying meta‐analysis tools to a systematic literature review allows the extraction of a much larger volume of information than directly calculating microevolutionary rates. We also propose derived meta‐analysis effect sizes (h, LG and LR) which are appropriate for the study of evolutionary patterns, the first being similar to haldanes, the second and third allowing the application of a preexisting analytical framework for the inference of evolutionary mechanisms. This novel methodological development is applicable to the study of microevolution in any taxa. To pilot test it, we built an open‐access database of 1,711 microevolutionary rates of 152 angiosperm species from 128 studies documenting population changes in quantitative traits following an environmental novelty with a known elapsed time (<260 years). The performance of the metrics proposed (h, LG and LR) is similar to that of preexisting ones, and at the same time they bring the advantages of lower estimation bias and higher number of usable observations typical of meta‐analysis.     

Accumulating evidence in ecology: Once is not enough

Many published studies in ecological science are viewed as stand‐alone investigations that purport to provide new insights into how ecological systems behave based on single analyses. But it is rare for results of single studies to provide definitive results, as evidenced in current discussions of the “reproducibility crisis” in science. The key step in science is the comparison of hypothesis‐based predictions with observations, where the predictions are typically generated by hypothesis‐specific models. Repeating this step allows us to gain confidence in the predictive ability of a model, and its corresponding hypothesis, and thus to accumulate evidence and eventually knowledge. This accumulation may occur via an ad hoc approach, via meta‐analyses, or via a more systematic approach based on the anticipated evolution of an information state. We argue the merits of this latter approach, provide an example, and discuss implications for designing sequences of studies focused on a particular question. We conclude by discussing current data collection programs that are preadapted to use this approach and argue that expanded use would increase the rate of learning in ecology, as well as our confidence in what is learned.     

Mixed‐Effects State‐Space Models for Analysis of Longitudinal Dynamic Systems

Summary The rapid development of new biotechnologies allows us to deeply understand biomedical dynamic systems in more detail and at a cellular level. Many of the subject‐specific biomedical systems can be described by a set of differential or difference equations that are similar to engineering dynamic systems. In this article, motivated by HIV dynamic studies, we propose a class of mixed‐effects state‐space models based on the longitudinal feature of dynamic systems. State‐space models with mixed‐effects components are very flexible in modeling the serial correlation of within‐subject observations and between‐subject variations. The Bayesian approach and the maximum likelihood method for standard mixed‐effects models and state‐space models are modified and investigated for estimating unknown parameters in the proposed models. In the Bayesian approach, full conditional distributions are derived and the Gibbs sampler is constructed to explore the posterior distributions. For the maximum likelihood method, we develop a Monte Carlo EM algorithm with a Gibbs sampler step to approximate the conditional expectations in the E‐step. Simulation studies are conducted to compare the two proposed methods. We apply the mixed‐effects state‐space model to a data set from an AIDS clinical trial to illustrate the proposed methodologies. The proposed models and methods may also have potential applications in other biomedical system analyses such as tumor dynamics in cancer research and genetic regulatory network modeling.