Evidential statistics as a statistical modern synthesis to support 21st century science

During the 20th century, population ecology and science in general relied on two very different statistical paradigms to solve its inferential problems: error statistics (also referred to as classical statistics and frequentist statistics) and Bayesian statistics. A great deal of good science was done using these tools, but both schools suffer from technical and philosophical difficulties. At the turning of the 21st century (Royall in Statistical evidence: a likelihood paradigm. Chapman & Hall, London, 1997 ; Lele in The nature of scientific evidence: statistical, philosophical and empirical considerations. The University of Chicago Press, Chicago, pp 191–216, 2004a ), evidential statistics emerged as a seriously contending paradigm. Drawing on and refining elements from error statistics, likelihoodism, Bayesian statistics, information criteria, and robust methods, evidential statistics is a statistical modern synthesis that smoothly incorporates model identification, model uncertainty, model comparison, parameter estimation, parameter uncertainty, pre-data control of error, and post-data strength of evidence into a single coherent framework. We argue that evidential statistics is currently the most effective statistical paradigm to support 21st century science. Despite the power of the evidential paradigm, we think that there is no substitute for learning how to clarify scientific arguments with statistical arguments. In this paper we sketch and relate the conceptual bases of error statistics, Bayesian statistics and evidential statistics. We also discuss a number of misconceptions about the paradigms that have hindered practitioners, as well as some real problems with the error and Bayesian statistical paradigms solved by evidential statistics.     

Multiple two-sample testing under arbitrary covariance dependency with an application in imaging mass spectrometry

Large-scale hypothesis testing has become a ubiquitous problem in high-dimensional statistical inference, with broad applications in various scientific disciplines. One relevant application is constituted by imaging mass spectrometry (IMS) association studies, where a large number of tests are performed simultaneously in order to identify molecular masses that are associated with a particular phenotype, for example, a cancer subtype. Mass spectra obtained from matrix-assisted laser desorption/ionization (MALDI) experiments are dependent, when considered as statistical quantities. False discovery proportion (FDP) estimation and  control under arbitrary dependency structure among test statistics is an active topic in modern multiple testing research. In this context, we are concerned with the evaluation of associations between the binary outcome variable (describing the phenotype) and multiple predictors derived from MALDI measurements. We propose an inference procedure in which the correlation matrix of the test statistics is utilized. The approach is based on multiple marginal models. Specifically, we fit a marginal logistic regression model for each predictor individually. Asymptotic joint normality of the stacked vector of the marginal regression coefficients is established under standard regularity assumptions, and their (limiting) correlation matrix is estimated. The proposed method extracts common factors from the resulting empirical correlation matrix. Finally, we estimate the realized FDP of a thresholding procedure for the marginal p-values. We demonstrate a practical application of the proposed workflow to MALDI IMS data in an oncological context.     

A Bayesian approach to efficient differential allocation for resampling-based significance testing

Background  

Large-scale statistical analyses have become hallmarks of post-genomic era biological research due to advances in high-throughput assays and the integration of large biological databases. One accompanying issue is the simultaneous estimation of p-values for a large number of hypothesis tests. In many applications, a parametric assumption in the null distribution such as normality may be unreasonable, and resampling-based p-values are the preferred procedure for establishing statistical significance. Using resampling-based procedures for multiple testing is computationally intensive and typically requires large numbers of resamples.     

Estimating Uncertainty in Ecosystem Budget Calculations

Ecosystem nutrient budgets often report values for pools and fluxes without any indication of uncertainty, which makes it difficult to evaluate the significance of findings or make comparisons across systems. We present an example, implemented in Excel, of a Monte Carlo approach to estimating error in calculating the N content of vegetation at the Hubbard Brook Experimental Forest in New Hampshire. The total N content of trees was estimated at 847 kg ha⁻¹ with an uncertainty of 8%, expressed as the standard deviation divided by the mean (the coefficient of variation). The individual sources of uncertainty were as follows: uncertainty in allometric equations (5%), uncertainty in tissue N concentrations (3%), uncertainty due to plot variability (6%, based on a sample of 15 plots of 0.05 ha), and uncertainty due to tree diameter measurement error (0.02%). In addition to allowing estimation of uncertainty in budget estimates, this approach can be used to assess which measurements should be improved to reduce uncertainty in the calculated values. This exercise was possible because the uncertainty in the parameters and equations that we used was made available by previous researchers. It is important to provide the error statistics with regression results if they are to be used in later calculations; archiving the data makes resampling analyses possible for future researchers. When conducted using a Monte Carlo framework, the analysis of uncertainty in complex calculations does not have to be difficult and should be standard practice when constructing ecosystem budgets.     

Testing statistical significance scores of sequence comparison methods with structure similarity

Background  

In the past years the Smith-Waterman sequence comparison algorithm has gained popularity due to improved implementations and rapidly increasing computing power. However, the quality and sensitivity of a database search is not only determined by the algorithm but also by the statistical significance testing for an alignment. The e-value is the most commonly used statistical validation method for sequence database searching. The CluSTr database and the Protein World database have been created using an alternative statistical significance test: a Z-score based on Monte-Carlo statistics. Several papers have described the superiority of the Z-score as compared to the e-value, using simulated data. We were interested if this could be validated when applied to existing, evolutionary related protein sequences.     

Bayesian statistics in genetics: a guide for the uninitiated.

Statistical analyses are used in many fields of genetic research. Most geneticists are taught classical statistics, which includes hypothesis testing, estimation and the construction of confidence intervals; this framework has proved more than satisfactory in many ways. What does a Bayesian framework have to offer geneticists? Its utility lies in offering a more direct approach to some questions and the incorporation of prior information. It can also provide a more straightforward interpretation of results. The utility of a Bayesian perspective, especially for complex problems, is becoming increasingly clear to the statistics community; geneticists are also finding this framework useful and are increasingly utilizing the power of this approach.     

Simultaneous and selective inference: Current successes and future challenges

The previous decade can be viewed as a second golden for era Multiple Comparisons research. I argue that much of the success stems from our being able to address real current needs. At the same time, this success generated a plethora of concepts for error rate and power, as well as multiplicity of methods for addressing them. These confuse the users of our methodology and pose a threat. To avoid the threat, it is our responsibility to match our theoretical goals to the goals of the users of statistics. Only then should we match the methods to the theoretical goals. Considerations related to such needs are discussed: simultaneous inference or selective inference, testing or estimation, decision making or scientific reporting. I then further argue that the vitality of our field in the future - as a research area - depends upon our ability to continue and address the real needs of statistical analyses in current problems. Two application areas offering new challenges have received less attention in our community to date are discussed. Safety analysis in clinical trials, where I offer an aggregated safety assessment methodology and functional Magnetic Resonance Imaging.     

An alternative foundation for the planning and evaluation of linkage analysis. II. Implications for multiple test adjustments

The 'multiple testing problem' currently bedevils the field of genetic epidemiology. Briefly stated, this problem arises with the performance of more than one statistical test and results in an increased probability of committing at least one Type I error. The accepted/conventional way of dealing with this problem is based on the classical Neyman-Pearson statistical paradigm and involves adjusting one's error probabilities. This adjustment is, however, problematic because in the process of doing that, one is also adjusting one's measure of evidence. Investigators have actually become wary of looking at their data, for fear of having to adjust the strength of the evidence they observed at a given locus on the genome every time they conduct an additional test. In a companion paper in this issue (Strug & Hodge I), we presented an alternative statistical paradigm, the 'evidential paradigm', to be used when planning and evaluating linkage studies. The evidential paradigm uses the lod score as the measure of evidence (as opposed to a p value), and provides new, alternatively defined error probabilities (alternative to Type I and Type II error rates). We showed how this paradigm separates or decouples the two concepts of error probabilities and strength of the evidence. In the current paper we apply the evidential paradigm to the multiple testing problem - specifically, multiple testing in the context of linkage analysis. We advocate using the lod score as the sole measure of the strength of evidence; we then derive the corresponding probabilities of being misled by the data under different multiple testing scenarios. We distinguish two situations: performing multiple tests of a single hypothesis, vs. performing a single test of multiple hypotheses. For the first situation the probability of being misled remains small regardless of the number of times one tests the single hypothesis, as we show. For the second situation, we provide a rigorous argument outlining how replication samples themselves (analyzed in conjunction with the original sample) constitute appropriate adjustments for conducting multiple hypothesis tests on a data set.     

Homologous recombination is unlikely to play a major role in influenza B virus evolution

Background

The role of migratory birds and of poultry trade in the dispersal of highly pathogenic H5N1 is still the topic of intense and controversial debate. In a recent contribution to this journal, Flint argues that the strict application of the scientific method can help to resolve this issue.

Discussion

We argue that Flint's identification of the scientific method with null hypothesis testing is misleading and counterproductive. There is far more to science than the testing of hypotheses; not only the justification, bur also the discovery of hypotheses belong to science. We also show why null hypothesis testing is weak and that Bayesian methods are a preferable approach to statistical inference. Furthermore, we criticize the analogy put forward by Flint between involuntary transport of poultry and long-distance migration.

Summary

To expect ultimate answers and unequivocal policy guidance from null hypothesis testing puts unrealistic expectations on a flawed approach to statistical inference and on science in general.     

Projecting Range Limits with Coupled Thermal Tolerance - Climate Change Models: An Example Based on Gray Snapper (Lutjanus griseus) along the U.S. East Coast

We couple a species range limit hypothesis with the output of an ensemble of general circulation models to project the poleward range limit of gray snapper. Using laboratory-derived thermal limits and statistical downscaling from IPCC AR4 general circulation models, we project that gray snapper will shift northwards; the magnitude of this shift is dependent on the magnitude of climate change. We also evaluate the uncertainty in our projection and find that statistical uncertainty associated with the experimentally-derived thermal limits is the largest contributor (∼ 65%) to overall quantified uncertainty. This finding argues for more experimental work aimed at understanding and parameterizing the effects of climate change and variability on marine species.     

Simultaneous modeling of multivariate heterogeneous responses and heteroskedasticity via a two-stage composite likelihood

Multivariate heterogeneous responses and heteroskedasticity have attracted increasing attention in recent years. In genome-wide association studies, effective simultaneous modeling of multiple phenotypes would improve statistical power and interpretability. However, a flexible common modeling system for heterogeneous data types can pose computational difficulties. Here we build upon a previous method for multivariate probit estimation using a two-stage composite likelihood that exhibits favorable computational time while retaining attractive parameter estimation properties. We extend this approach to incorporate multivariate responses of heterogeneous data types (binary and continuous), and possible heteroskedasticity. Although the approach has wide applications, it would be particularly useful for genomics, precision medicine, or individual biomedical prediction. Using a genomics example, we explore statistical power and confirm that the approach performs well for hypothesis testing and coverage percentages under a wide variety of settings. The approach has the potential to better leverage genomics data and provide interpretable inference for pleiotropy, in which a locus is associated with multiple traits.     

Uncertainty in Calculations of Net Primary Production for Grasslands

Net primary production (NPP) is a fundamental characteristic of all ecosystems and foundational to understanding the fluxes of energy and nutrients. Because NPP cannot be measured directly, researchers use field-measured surrogates as input variables in various equations designed to estimate ‘true NPP’. This has led to considerable debate concerning which equations most accurately estimate ‘true NPP’. This debate has influenced efforts to assess NPP in grasslands, with researchers often advocating more complex equations to avoid underestimation. However, this approach ignores the increase in statistical error associated with NPP estimates as a greater number of parameters and more complex mathematical functions are introduced into the equation. Using published grassland data and Monte Carlo simulation techniques, we assessed the relative variability in NPP estimates obtained using six different NPP estimation equations that varied in both the number of parameters and intricacy of mathematical operations. Our results indicated that more complex equations may result in greater uncertainty without reducing the probability of underestimation. The amount of uncertainty associated with estimates of NPP was influenced by the number of parameters as well as the variability in the data and the nature of the mathematical operations. For example, due to greater variability in the field-measured belowground data than aboveground data, estimates of belowground NPP tended to have more uncertainty than estimates of aboveground NPP. An analysis in which the input data were standardized allowed us to isolate the details of the calculations from the variability in the data in assessing the propagation of uncertainty. This analysis made clear that equations with product terms have the potential to magnify the uncertainty of the inputs in the estimates of NPP although this relationship was complicated by interactions with data variability and number of parameters. Our results suggest that more complex NPP estimation equations can increase uncertainty without necessarily reducing risk of underestimation. Because estimates can never be tested by comparison to “true NPP”, we recommend that researchers include an assessment of propagation of statistical error when evaluating the ‘best’ estimation method.     

Mate Familiarity Affects Pairing Behaviour in a Long‐Term Monogamous Lizard: Evidence from Detailed Bio‐Logging and a 31‐Year Field Study

Long‐term monogamy is most prevalent in birds but is also found in lizards. We combined a 31‐year field study of the long‐lived, monogamous Australian sleepy lizard, Tiliqua rugosa, with continuous behavioural observations through GPS data logging, in 1 yr, to investigate the duration of pair bonds, rates of partner change and whether either the reproductive performance hypothesis or the mate familiarity hypothesis could explain this remarkable long‐term monogamy. The reproductive performance hypothesis predicts higher reproductive success in more experienced parents, whereas the mate familiarity hypothesis suggests that effects of partner familiarity select for partner retention and long‐term monogamy. Rates of partner change were below 34% over a 5‐yr period and most sleepy lizards formed long‐term pair bonds: 31 partnerships lasted for more than 15 yr, 110 for more than 10 yr, and the recorded maximum was 27 yr (ongoing). In the year when we conducted detailed observations, familiar pairs mated significantly earlier than unfamiliar pairs. Previous pairing experience (total number of years paired with previous partners) had no significant effect. Early mating often equates to higher reproductive success, and we infer that is the case in sleepy lizards. Early mating of familiar pairs was not due to better body condition. We propose two suggestions about the proximate mechanisms that may allow familiar pair partners to mate earlier than unfamiliar partners. First, they may have improved coordination of their reproductive sexual cycles to reach receptivity earlier and thereby maximise fertilisation success. Second, they may forage more efficiently, benefiting from effective information transfer and/or cooperative predator detection. Those ideas need empirical testing in the future. Regardless of the mechanism, our observations of sleepy lizard pairing behaviour support the mate familiarity hypothesis, but not the reproductive performance hypothesis, as an explanation for its long‐term monogamous mating system.     

Bayesian characterization of uncertainty in species interaction strengths

Considerable effort has been devoted to the estimation of species interaction strengths. This effort has focused primarily on statistical significance testing and obtaining point estimates of parameters that contribute to interaction strength magnitudes, leaving the characterization of uncertainty associated with those estimates unconsidered. We consider a means of characterizing the uncertainty of a generalist predator’s interaction strengths by formulating an observational method for estimating a predator’s prey-specific per capita attack rates as a Bayesian statistical model. This formulation permits the explicit incorporation of multiple sources of uncertainty. A key insight is the informative nature of several so-called non-informative priors that have been used in modeling the sparse data typical of predator feeding surveys. We introduce to ecology a new neutral prior and provide evidence for its superior performance. We use a case study to consider the attack rates in a New Zealand intertidal whelk predator, and we illustrate not only that Bayesian point estimates can be made to correspond with those obtained by frequentist approaches, but also that estimation uncertainty as described by 95% intervals is more useful and biologically realistic using the Bayesian method. In particular, unlike in bootstrap confidence intervals, the lower bounds of the Bayesian posterior intervals for attack rates do not include zero when a predator–prey interaction is in fact observed. We conclude that the Bayesian framework provides a straightforward, probabilistic characterization of interaction strength uncertainty, enabling future considerations of both the deterministic and stochastic drivers of interaction strength and their impact on food webs.     

Knowledge and attitudes regarding genetic testing among Jordanians: An approach towards genomic medicine

BackgroundThe twenty first century can be called the genomic era referring to the rapid development of genetics, and the beginning of genomic medicine. An initial step towards genomic medicine is to evaluate the knowledge and attitude towards genetic testing among different populations. The aims of this study were to assess the genetic knowledge and attitude towards genetic testing among the Jordanian population and patients with immune diseases. In addition, we evaluated the association between knowledge, attitude and several demographic factors of the population.MethodsThis study was performed using an online questionnaire that was distributed to respondents from different regions of Jordan.ResultsA total of 1149 participants were recruited from the Jordanian population. Overall factual genetic knowledge of the participants was good (65.4%), with education level, working or studying in a health-related field and household average monthly income being significant predictors of factual knowledge scores (P = 0.03, P < 0.001 and P < 0.001, respectively). However, factual knowledge results revealed that scores of questions related to diseases were significantly higher than scores of gene-related scientific facts (P < 0.01). Participants of our study reported to have low perceived knowledge on medical uses (39.5%) and social consequences (23.9%) of genetic testing. Regarding the participants’ attitudes, favorable attitudes towards genetic testing were prevailing (91.5%). Favorable attitudes were more prominent among higher educated participants, and participants with higher scores of factual knowledge.ConclusionDespite the fact that our Jordanian-based study revealed a good level of genetic knowledge as well as a favorable attitude towards genetic testing, we realized an imbalance of knowledge between gene-related scientific facts and disease-related concepts as well as between factual and perceived genetic knowledge, which indicates the necessity of increasing the awareness about genetic testing in order to ensure that individuals can take informed decisions that help in the employment of personalized medicine.     

Bias-corrected variance estimation and hypothesis testing for spatial point and marked point processes using subsampling

Summary We introduce novel regression extrapolation based methods to correct the often large bias in subsampling variance estimation as well as hypothesis testing for spatial point and marked point processes. For variance estimation, our proposed estimators are linear combinations of the usual subsampling variance estimator based on subblock sizes in a continuous interval. We show that they can achieve better rates in mean squared error than the usual subsampling variance estimator. In particular, for n×n observation windows, the optimal rate of n^?2 can be achieved if the data have a finite dependence range. For hypothesis testing, we apply the proposed regression extrapolation directly to the test statistics based on different subblock sizes, and therefore avoid the need to conduct bias correction for each element in the covariance matrix used to set up the test statistics. We assess the numerical performance of the proposed methods through simulation, and apply them to analyze a tropical forest data set.     

Score Tests for Exploring Complex Models: Application to HIV Dynamics Models

In biostatistics, more and more complex models are being developed. This is particularly the case in system biology. Fitting complex models can be very time‐consuming, since many models often have to be explored. Among the possibilities are the introduction of explanatory variables and the determination of random effects. The particularity of this use of the score test is that the null hypothesis is not itself very simple; typically, some random effects may be present under the null hypothesis. Moreover, the information matrix cannot be computed, but only an approximation based on the score. This article examines this situation with the specific example of HIV dynamics models. We examine the score test statistics for testing the effect of explanatory variables and the variance of random effect in this complex situation. We study type I errors and the statistical powers of this score test statistics and we apply the score test approach to a real data set of HIV‐infected patients.     

How to obtain a precise and representative estimate for parameters in LCA

Background, Aim and Scope  Quite often there is need for precise and representative parameters in LCA studies. Probably the most relevant have direct influence on the functional unit, whose definition is crucial in the conduct of any LCA. Changes in the functional unit show directly in LCI and LCIA results. In comparative assertions, a bias in the functional unit may lead to a bias in the overall conclusions. Since quantitative data for the functional unit, such as geometric dimensions and specific weight, often vary, the question arises how to determine the functional unit, especially if a comparative assertion shall be representative for a region or market. Aim and scope of the study is to develop and apply methods for obtaining precise and representative estimates for the functional unit as one important parameter in an LCA study. Materials and Methods  Statistical sampling is applied in order to get empirical estimates for the weight of yoghurt cups, as a typical parameter for the functional unit. We used a two-stage sampling design, with stratified sampling in the first stage and three different sampling designs in the second stage, namely stratified, clustered, and a posteriori sampling. Sampling designs are motivated and described. In a case study, they are each used to determined a representative weight for 150 g yoghurt cups in Berlin, at the point of sale and within a specific time. In the first sampling stage, food markets are randomly selected, while in the second stage, yoghurt cups in these food markets are sampled. The sampling methods are applicable due to newly available internet data. These data sources and their shortcomings are described. Results  The random sampling procedure yields representative estimates, which are compared to figures for market leaders, i.e. yoghurt cups with very high occurrence in the supermarkets. While single types of yoghurt cups showed moderate uncertainty, representative estimates were highly precise. Discussion results show, for one, the performance of the applied statistical estimation procedures, and they show further that adding more information in the estimation procedure (on the shape of the cup, on the type of plastic, on the specific brand) helps reducing uncertainty. Conclusions  As conclusions, estimates and their uncertainty depend on the measurement procedure in a sensitive manner; any uncertainty information should be coupled with information on the measurement procedure, and it is recommended to use statistical sampling in order to reduce uncertainty for important parameters of an LCA study. Recommendations and Perspectives  Results for market leaders differed considerably from representative estimates. This implies to not use market leader data, or data with a high market share, as substitute for representative data in LCA studies. Statistical sampling has been barely used for Life Cycle Assessment. It turned out to be a feasible means for obtaining highly precise and representative estimates for the weight of yoghurt cups in the case study, based on empirical analysis. Further research is recommended in order to detect which parameters should best be investigated in LCA case studies; which data sources are available and recommended, and which sampling designs are appropriate for different application cases. ESS-Submission Editor: Seungdo Kim. PhD (kimseun@msu.edu)     

The precision of the hominid timescale estimated by relaxed clock methods

The chronological scenario of the evolution of hominoid primates has been thoroughly investigated since the advent of the molecular clock hypothesis. With the availability of genomic sequences for all hominid genera and other anthropoids, we may have reached the point at which the information from sequence data alone will not provide further evidence for the inference of the hominid evolution timescale. To verify this conjecture, we have compiled a genomic data set for all of the anthropoid genera. Our estimate places the Homo/Pan divergence at approximately 7.4 Ma, the Gorilla lineage divergence at approximately 9.7 Ma, the basal Hominidae divergence at 18.1 Ma and the basal Hominoidea divergence at 20.6 Ma. By inferring the theoretical limit distribution of posterior densities under a Bayesian framework, we show that it is unlikely that lengthier alignments or the availability of new genomic sequences will provide additional information to reduce the uncertainty associated with the divergence time estimates of the four hominid genera. A reduction of this uncertainty will be achieved only by the inclusion of more informative calibration priors.     

Trials and tribulations of statistical significance in biochemistry and omics

Over recent years many statisticians and researchers have highlighted that statistical inference would benefit from a better use and understanding of hypothesis testing, p-values, and statistical significance. We highlight three recommendations in the context of biochemical sciences. First recommendation: to improve the biological interpretation of biochemical data, do not use p-values (or similar test statistics) as thresholded values to select biomolecules. Second recommendation: to improve comparison among studies and to achieve robust knowledge, perform complete reporting of data. Third recommendation: statistical analyses should be reported completely with exact numbers (not as asterisks or inequalities). Owing to the high number of variables, a better use of statistics is of special importance in omic studies.