A bound for publication bias based on the fraction of unpublished studies

Publication bias in meta-analysis is usually modeled in terms of an accept/reject selection procedure in which the selected studies are the "published" studies and the rejected studies are the "unpublished" studies. One possible selection mechanism is to suppose that only studies that report an estimated treatment effect exceeding (or falling short of) some threshold are accepted. We show that, with appropriate choice of thresholds, this attains the maximum bias among all selection mechanisms in which the probability of selection increases with study size. It is impossible to estimate the selection mechanism from the observed studies alone: this result leads to a "worst-case" sensitivity analysis for publication bias, which is remarkably easy to implement in practice. The method is illustrated using data on the effectiveness of prophylactic corticosteroids.     

Assessing the implications of publication bias for two popular estimates of between-study variance in meta-analysis

Perhaps the greatest threat to the validity of a meta-analysis is the possibility of publication bias, where studies with interesting or statistically significant results are more likely to be published. This obviously impacts on inference concerning the treatment effect but also has implications for estimates of between-study variance. Two popular and established estimation methods are considered and formulae for assessing the implications of the bias are provided in terms of a general process for selecting studies. Meta-analysts, concerned that publication bias may be present, can use these as part of a sensitivity analysis to assess the robustness of their estimates of between-study variance using any selection process that is likely to be used in practice. The procedure is illustrated using a meta-analysis of clinical trials concerning the effectiveness of endoscopic sclerotherapy for preventing death in patients with cirrhosis and oesophagogastric varices.     

Adjusting for publication bias in meta-analysis via inverse probability weighting using clinical trial registries

Publication bias is a major concern in conducting systematic reviews and meta-analyses. Various sensitivity analysis or bias-correction methods have been developed based on selection models, and they have some advantages over the widely used trim-and-fill bias-correction method. However, likelihood methods based on selection models may have difficulty in obtaining precise estimates and reasonable confidence intervals, or require a rather complicated sensitivity analysis process. Herein, we develop a simple publication bias adjustment method by utilizing the information on conducted but still unpublished trials from clinical trial registries. We introduce an estimating equation for parameter estimation in the selection function by regarding the publication bias issue as a missing data problem under the missing not at random assumption. With the estimated selection function, we introduce the inverse probability weighting (IPW) method to estimate the overall mean across studies. Furthermore, the IPW versions of heterogeneity measures such as the between-study variance and the I² measure are proposed. We propose methods to construct confidence intervals based on asymptotic normal approximation as well as on parametric bootstrap. Through numerical experiments, we observed that the estimators successfully eliminated bias, and the confidence intervals had empirical coverage probabilities close to the nominal level. On the other hand, the confidence interval based on asymptotic normal approximation is much wider in some scenarios than the bootstrap confidence interval. Therefore, the latter is recommended for practical use.     

Effects of Mutation on Selection Limits in Finite Populations with Multiple Alleles

The ultimate response to directional selection (i.e., the selection limit) under recurrent mutation is analyzed by a diffusion approximation for a population in which there are k possible alleles at a locus. The limit mainly depends on two scaled parameters S (= 4Ns sigma a) and theta (= 4Nu) and k, the number of alleles, where N is the effective population size, u is the mutation rate, s is the selection coefficient, and sigma 2a is the variance of allelic effects. When the selection pressure is weak (S less than or equal to 0.5), the limit is given approximately by 2S sigma a[1 - (1 + c2)/k]/(theta + 1) for additive effects of alleles, where c is the coefficient of variation of the mutation rates among alleles. For strong selection, other approximations are devised to analyze the limit in different parameter regions. The effect of mutation on selection limits largely relies on the potential of mutation to introduce new and better alleles into the population. This effect is, however, bounded under the present model. Unequal mutation rates among alleles tend to reduce the selection limit, and can have a substantial effect only for small numbers of alleles and weak selection. The selection limit decreases as the mutation rate increases.     

Effect of residual stress and heterogeneity on circumferential stress in the arterial wall

Quantifying the stress distribution through the arterial wall is essential to studies of arterial growth and disease. Previous studies have shown that both residual stress, as measured by opening angle, and differing material properties for the media-intima and the adventitial layers affect the transmural circumferential stress (sigma theta) distribution. Because a lack of comprehensive data on a single species and artery has led to combinations from multiple sources, this study determined the sensitivity of sigma theta to published variations in both opening angle and layer thickness data. We fit material properties to previously published experimental data for pressure-diameter relations and opening angles of rabbit carotid artery, and predicted sigma theta through the arterial wall at physiologic conditions. Using a one-layer model, the ratio of sigma theta at the internal wall to the mean sigma theta decreased from 2.34 to 0.98 as the opening angle increased from 60 to 130 deg. In a two-layer model using a 95 deg opening angle, mean sigma theta in the adventitia increased (112 percent for 25 percent adventitia) and mean sigma theta in the media decreased (47 percent for 25 percent adventitia). These results suggest that both residual stress and wall layers have important effects on transmural stress distribution. Thus, experimental measurements of loading curves, opening angles, and wall composition from the same species and artery are needed to accurately predict the transmural stress distribution in the arterial wall.     

Selection models with monotone weight functions in meta analysis

Publication bias, the fact that studies identified for inclusion in a meta analysis do not represent all studies on the topic of interest, is commonly recognized as a threat to the validity of a meta analysis. One way to explicitly model publication bias is via weighted probability distributions. We adopt the non-parametric approach initially introduced by Dear and Begg (1992) but impose that the weight function w is monotonely non-increasing as a function of the p-value. Since in meta analysis one typically only has few studies or "observations," regularization of the estimation problem seems sensible. In addition, virtually all parametric weight functions proposed so far in the literature are in fact decreasing. We discuss how to estimate a decreasing weight function in the above model and illustrate the new methodology on two well-known examples. Some basic properties of the log-likelihood function and computation of a p-value quantifying the evidence against the null hypothesis of a constant weight function are indicated. In addition, we provide an approximate selection bias adjusted profile likelihood confidence interval for the treatment effect. The corresponding software and the data sets used to illustrate it are provided as the R package selectMeta (Rufibach, 2011).     

Relationships fade with time: a meta-analysis of temporal trends in publication in ecology and evolution.

Both significant positive and negative relationships between the magnitude of research findings (their 'effect size') and their year of publication have been reported in a few areas of biology. These trends have been attributed to Kuhnian paradigm shifts, scientific fads and bias in the choice of study systems. Here we test whether or not these isolated cases reflect a more general trend. We examined the relationship using effect sizes extracted from 44 peer-reviewed meta-analyses covering a wide range of topics in ecological and evolutionary biology. On average, there was a small but significant decline in effect size with year of publication. For the original empirical studies there was also a significant decrease in effect size as sample size increased. However, the effect of year of publication remained even after we controlled for sampling effort. Although these results have several possible explanations, it is suggested that a publication bias against non-significant or weaker findings offers the most parsimonious explanation. As in the medical sciences, non-significant results may take longer to publish and studies with both small sample sizes and non-significant results may be less likely to be published.     

PICALM rs3851179 Variant Confers Susceptibility to Alzheimer’s Disease in Chinese Population

The association between PICALM rs3851179 variant and Alzheimer’s disease (AD) has been well established by previous genome-wide association studies (GWAS) and candidate gene studies in European population. Recent studies investigated the association between PICALM rs3851179 and AD susceptibility in Chinese population. However, these studies reported consistent and inconsistent results. Here, we selected 9435 samples including 3704 AD cases and 5731 controls from previous studies and evaluated this association using a meta-analysis method for additive model. We did not observe significant genetic heterogeneity in Chinese population. Our results indicate significant association between PICALM rs3851179 and AD in Chinese population. The sensitivity analysis indicates that the association between rs3851179 and AD did not vary substantially. The regression analysis suggests no significant publication bias. In summary, this updated meta-analysis highlights the involvement of PICALM rs3851179 variant in Alzheimer’s disease susceptibility in Chinese population.     

Osmotic flow equations for leaky porous membranes

A basic set of equations describing the flows of volume (Jv) and solute (Js) across a leaky porous membrane, coupled to the differences of osmotic and hydrostatic pressures d pi and dP has been derived by using general frictional theory. Denoting the mean pore concentration of solute by c*s and the hydraulic and diffusive conductances by Lp and Ps/RT the equations take the form Jv = LpdP + sigma sLp d pi Js = c*s(1 - sigma f)Jv + Ps d pi/RT sigma s = theta (1 - DsVs/DwVw - Ds/Dos) sigma f = 1 - theta DsVs/DwVw - Ds/Dos in which Dw and Ds are the diffusion coefficients for water and solute in the pore and Dos that for free solution. The relation between the reflection coefficients sigma s and sigma f for osmosis and ultrafiltration is then given by sigma s = sigma f - (1- theta)(1 - Ds/Dos), where theta is the diffusive-driven:pressure-driven flow ratio. These equations follow from the fact that in leaky pores osmosis occurs by diffusion alone and that there cannot be any Onsager symmetry leading to sigma s = sigma f. Symmetry holds in the limits where either the pore is small, when sigma s = sigma f = 1, or where the pore is large when sigma s = sigma f = 0.     

Helicobacter pylori infection and colorectal cancer risk: a meta-analysis

BACKGROUND: Several studies suggested an association between Helicobacter pylori infection and colorectal carcinoma or adenoma risk. However, different authors reported quite varying estimates. We carried out a systematic review and meta-analysis of published studies investigating this association and paid special attention to the possibility of publication bias and sources of heterogeneity between studies. Materials and METHODS: An extensive literature search and cross-referencing were performed to identify all published studies. Summary estimates were obtained using random-effects models. The presence of possible publication bias was assessed using different statistical approaches. RESULTS: In a meta-analysis of the 11 identified human studies, published between 1991 and 2002, a summary odds ratio of 1.4 (95% CI, 1.1-1.8) was estimated for the association between H. pylori infection and colorectal cancer risk. The graphical funnel plot appeared asymmetrical, but the formal statistical evaluations did not provide strong evidence of publication bias. The proportion of variation of study results because of heterogeneity was small (36.5%). CONCLUSIONS: The results of our meta-analysis are consistent with a possible small increase in risk of colorectal cancer because of H. pylori infection. However, the possibility of some publication bias cannot be ruled out, although it could not be statistically confirmed. Larger, better designed and better controlled studies are needed to clarify the situation.     

Publication bias: evidence of delayed publication in a cohort study of clinical research projects.

OBJECTIVES: To determine the extent to which publication is influenced by study outcome. DESIGN: A cohort of studies submitted to a hospital ethics committee over 10 years were examined retrospectively by reviewing the protocols and by questionnaire. The primary method of analysis was Cox''s proportional hazards model. SETTING: University hospital, Sydney, Australia. STUDIES: 748 eligible studies submitted to Royal Prince Alfred Hospital Ethics Committee between 1979 and 1988. MAIN OUTCOME MEASURES: Time to publication. RESULTS: Response to the questionnaire was received for 520 (70%) of the eligible studies. Of the 218 studies analysed with tests of significance, those with positive results (P < 0.05) were much more likely to be published than those with negative results (P > or = 0.10) (hazard ratio 2.32 (95% confidence interval 1.47 to 3.66), P = 0.0003), with a significantly shorter time to publication (median 4.8 v 8.0 years). This finding was even stronger for the group of 130 clinical trials (hazard ratio 3.13 (1.76 to 5.58). P = 0.0001), with median times to publication of 4.7 and 8.0 years respectively. These results were not materially changed after adjusting for other significant predictors of publication. Studies with indefinite conclusions (0.05 < or = P < 0.10) tended to have an even lower publication rate and longer time to publication than studies with negative results (hazard ratio 0.39 (0.13 to 1.12), P = 0.08). For the 103 studies in which outcome was rated qualitatively, there was no clear cut evidence of publication bias, although the number of studies in this group was not large. CONCLUSIONS: This study confirms the evidence of publication bias found in other studies and identifies delay in publication as an additional important factor. The study results support the need for prospective registration of trials to avoid publication bias and also support restricting the selection of trials to those started before a common date in undertaking systematic reviews.     

Testing small study effects in multivariate meta-analysis

Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias, and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection processes can be involved in the reporting of multivariate outcomes. For example, some studies may be completely unpublished while others may selectively report multiple outcomes. In this paper, we propose a score test as an overall test of small study effects in multivariate meta-analysis. Two detailed case studies are given to demonstrate the advantage of the proposed test over various naive applications of univariate tests in practice. Through simulation studies, the proposed test is found to retain nominal Type I error rates with considerable power in moderate sample size settings. Finally, we also evaluate the concordance between the proposed tests with the naive application of univariate tests by evaluating 44 systematic reviews with multiple outcomes from the Cochrane Database.     

Bias and error in using survey records for ponderosa pine landscape restoration

Aim Public land survey records are commonly used to reconstruct historical forest structure over large landscapes. Reconstruction studies have been criticized for using absolute measures of forest attributes, such as density and basal area, because of potential selection bias by surveyors and unknown measurement error. Current methods to identify bias are based upon statistical techniques whose assumptions may be violated for survey data. Our goals were to identify and directly estimate common sources of bias and error, and to test the accuracy of statistical methods to identify them. Location Forests in the western USA: Mogollon Plateau, Arizona; Blue Mountains, Oregon; Front Range, Colorado. Methods We quantified both selection bias and measurement error for survey data in three ponderosa pine landscapes by directly comparing measurements of bearing trees in survey notes with remeasurements of bearing trees at survey corners (384 corners and 812 trees evaluated). Results Selection bias was low in all areas and there was little variability among surveyors. Surveyors selected the closest tree to the corner 95% to 98% of the time, and hence bias may have limited impacts on reconstruction studies. Bourdo’s methods were able to successfully detect presence or absence of bias most of the time, but do not measure the rate of bias. Recording and omission errors were common but highly variable among surveyors. Measurements for bearing trees made by surveyors were generally accurate. Most bearings were less than 5° in error and most distances were within 5% of our remeasurements. Many, but not all, surveyors in the western USA probably estimated diameter of bearing trees at stump height (0.3 m). These estimates deviated from reconstructed diameters by a mean absolute error of 7.0 to 10.6 cm. Main conclusions Direct comparison of survey data at relocated corners is the only method that can determine if bias and error are meaningful. Data from relocated trees show that biased selection of trees is not likely to be an important source of error. Many surveyor errors would have no impact on reconstruction studies, but omission errors have the potential to have a large impact on results. We suggest how to reduce potential errors through data screening.     

Three-armed trials including placebo and no-treatment groups may be subject to publication bias: systematic review

Background

It has been argued that placebos may not have important clinical impacts in general. However, there is increasing evidence of a publication bias among trials published in journals. Therefore, we explored the potential for publication bias in randomized trials with active treatment, placebo, and no-treatment groups.

Methods

Three-armed randomized trials of acupuncture, acupoint stimulation, and transcutaneous electrical stimulation were obtained from electronic databases. Effect sizes between treatment and placebo groups were calculated for treatment effect, and effect sizes between placebo and no-treatment groups were calculated for placebo effect. All data were then analyzed for publication bias.

Results

For the treatment effect, small trials with fewer than 100 patients per arm showed more benefits than large trials with at least 100 patients per arm in acupuncture and acupoint stimulation. For the placebo effect, no differences were found between large and small trials. Further analyses showed that the treatment effect in acupuncture and acupoint stimulation may be subject to publication bias because study design and any known factors of heterogeneity were not associated with the small study effects. In the simulation, the magnitude of the placebo effect was smaller than that calculated after considering publication bias.

Conclusions

Randomized three-armed trials, which are necessary for estimating the placebo effect, may be subject to publication bias. If the magnitude of the placebo effect is assessed in an intervention, the potential for publication bias should be investigated using data related to the treatment effect.     

Applications of a parametric model for informative censoring

In this article, we explore the use of a parametric model (for analyzing survival data) which is defined to allow sensitivity analysis for the presence of informative censoring. The dependence between the failure and the censoring processes is expressed through a parameter delta and a general bias function B(t, theta). We calculate the expectation of the potential bias due to informative censoring, which is an overall measure of how misleading our results might be if censoring is actually nonignorable. Bounds are also calculated for quantities of interest, e.g., parameter of the distribution of the failure process, which do not depend on the choice of the bias function for fixed delta. An application that relates to systematic lupus erythematosus data illustrates how additional information can result in reducing the uncertainty on estimates of the location parameter. Sensitivity analysis on a relative risk parameter is also explored.     

Exploring Regional Variation in Roost Selection by Bats: Evidence from a Meta-Analysis

Background and Aims

Tree diameter, tree height and canopy closure have been described by previous meta-analyses as being important characteristics in roost selection by cavity-roosting bats. However, size and direction of effects for these characteristics varied greatly among studies, also referred to as heterogeneity. Potential sources of heterogeneity have not been investigated in previous meta-analyses, which are explored by correlating additional covariates (moderator variables). We tested whether effect sizes from 34 studies were consistent enough to reject the null hypothesis that trees selected by bats did not significantly differ in their characteristics from randomly selected trees. We also examined whether heterogeneity in tree diameter effect sizes was correlated to moderator variables such as sex, bat species, habitat type, elevation and mean summer temperature.

Methods

We used Hedges’ g standardized mean difference as the effect size for the most common characteristics that were encountered in the literature. We estimated heterogeneity indices, potential publication bias, and spatial autocorrelation of our meta-data. We relied upon meta-regression and multi-model inference approaches to evaluate the effects of moderator variables on heterogeneity in tree diameter effect sizes.

Results

Tree diameter, tree height, snag density, elevation, and canopy closure were significant characteristics of roost selection by cavity-roosting bats. Size and direction of effects varied greatly among studies with respect to distance to water, tree density, slope, and bark remaining on trunks. Inclusion of mean summer temperature and sex in meta-regressions further explained heterogeneity in tree diameter effect sizes.

Conclusions

Regional differences in roost selection for tree diameter were related to mean summer temperature. Large diameter trees play a central role in roost selection by bats, especially in colder regions, where they are likely to provide a warm and stable microclimate for reproductive females. Records of summer temperature fluctuations inside and outside tree cavities that are used by bats should be included in future research.     

Sensitivity analysis for informative censoring in parametric survival models

Most statistical methods for censored survival data assume there is no dependence between the lifetime and censoring mechanisms, an assumption which is often doubtful in practice. In this paper we study a parametric model which allows for dependence in terms of a parameter delta and a bias function B(t, theta). We propose a sensitivity analysis on the estimate of the parameter of interest for small values of delta. This parameter measures the dependence between the lifetime and the censoring mechanisms. Its size can be interpreted in terms of a correlation coefficient between the two mechanisms. A medical example suggests that even a small degree of dependence between the failure and censoring processes can have a noticeable effect on the analysis.     

The genealogy of a sequence subject to purifying selection at multiple sites

We investigate the effect of purifying selection at multiple sites on both the shape of the genealogy and the distribution of mutations on the tree. We find that the primary effect of purifying selection on a genealogy is to shift the distribution of mutations on the tree, whereas the shape of the tree remains largely unchanged. This result is relevant to the large number of coalescent estimation procedures, which generally assume neutrality for segregating polymorphisms--applying these estimators to evolutionarily constrained sequences could lead to a significant degree of bias. We also estimate the statistical power of several neutrality tests in detecting weak to moderate purifying selection and find that the power is quite good for some parameter combinations. This result contrasts with previous studies, which predicted low statistical power because of the minor effect that weak purifying selection has on the shape of a genealogy. Finally, we investigate the effect of Hill-Robertson interference among linked deleterious mutations on patterns of molecular variation. We find that dependence among selected loci can substantially reduce the efficacy of even fairly strong purifying selection.     

Estimating effective population size or mutation rate with microsatellites

Microsatellites are short tandem repeats that are widely dispersed among eukaryotic genomes. Many of them are highly polymorphic; they have been used widely in genetic studies. Statistical properties of all measures of genetic variation at microsatellites critically depend upon the composite parameter theta = 4Nmicro, where N is the effective population size and micro is mutation rate per locus per generation. Since mutation leads to expansion or contraction of a repeat number in a stepwise fashion, the stepwise mutation model has been widely used to study the dynamics of these loci. We developed an estimator of theta, theta; (F), on the basis of sample homozygosity under the single-step stepwise mutation model. The estimator is unbiased and is much more efficient than the variance-based estimator under the single-step stepwise mutation model. It also has smaller bias and mean square error (MSE) than the variance-based estimator when the mutation follows the multistep generalized stepwise mutation model. Compared with the maximum-likelihood estimator theta; (L) by, theta; (F) has less bias and smaller MSE in general. theta; (L) has a slight advantage when theta is small, but in such a situation the bias in theta; (L) may be more of a concern.     

Publication bias in ecology and evolution: an empirical assessment using the 'trim and fill' method

Recent reviews of specific topics, such as the relationship between male attractiveness to females and fluctuating asymmetry or attractiveness and the expression of secondary sexual characters, suggest that publication bias might be a problem in ecology and evolution. In these cases, there is a significant negative correlation between the sample size of published studies and the magnitude or strength of the research findings (formally the ‘effect size’). If all studies that are conducted are equally likely to be published, irrespective of their findings, there should not be a directional relationship between effect size and sample size; only a decrease in the variance in effect size as sample size increases due to a reduction in sampling error. One interpretation of these reports of negative correlations is that studies with small sample sizes and weaker findings (smaller effect sizes) are less likely to be published. If the biological literature is systematically biased this could undermine the attempts of reviewers to summarise actual biology relationships by inflating estimates of average effect sizes. But how common is this problem? And does it really effect the general conclusions of literature reviews? Here, we examine data sets of effect sizes extracted from 40 peer‐reviewed, published meta‐analyses. We estimate how many studies are missing using the newly developed ‘trim and fill’ method. This method uses asymmetry in plots of effect size against sample size (‘funnel plots’) to detect ‘missing’ studies. For random‐effect models of meta‐analysis 38% (15/40) of data sets had a significant number of ‘missing’ studies. After correcting for potential publication bias, 21% (8/38) of weighted mean effects were no longer significantly greater than zero, and 15% (5/34) were no longer statistically robust when we used random‐effects models in a weighted meta‐analysis. The mean correlation between sample size and the magnitude of standardised effect size was also significantly negative (r_s=‐0.20, P < 0‐0001). Individual correlations were significantly negative (P < 0.10) in 35% (14/40) of cases. Publication bias may therefore effect the main conclusions of at least 15–21% of meta‐analyses. We suggest that future literature reviews assess the robustness of their main conclusions by correcting for potential publication bias using the ‘trim and fill’ method.