Equivalence analysis to support environmental safety assessment: Using nontarget organism count data from field trials with cisgenically modified potato

This paper considers the statistical analysis of entomological count data from field experiments with genetically modified (GM) plants. Such trials are carried out to assess environmental safety. Potential effects on nontarget organisms (NTOs), as indicators of biodiversity, are investigated. The European Food Safety Authority (EFSA) gives broad guidance on the environmental risk assessment (ERA) of GM plants. Field experiments must contain suitable comparator crops as a benchmark for the assessment of designated endpoints. In this paper, a detailed protocol is proposed to perform data analysis for the purpose of assessing environmental safety. The protocol includes the specification of a list of endpoints and their hierarchical relations, the specification of intended levels of data analysis, and the specification of provisional limits of concern to decide on the need for further investigation. The protocol emphasizes a graphical representation of estimates and confidence intervals for the ratio of mean abundances for the GM plant and its comparator crop. Interpretation relies mainly on equivalence testing in which confidence intervals are compared with the limits of concern. The proposed methodology is illustrated with entomological count data resulting from multiyear, multilocation field trials. A cisgenically modified potato line (with enhanced resistance to late blight disease) was compared to the original conventional potato variety in the Netherlands and Ireland in two successive years (2013, 2014). It is shown that the protocol encompasses alternative schemes for safety assessment resulting from different research questions and/or expert choices. Graphical displays of equivalence testing at several hierarchical levels and their interpretation are presented for one of these schemes. The proposed approaches should be of help in the ERA of GM or other novel plants.     

Two-stage design of sequencing studies for testing association with rare variants

Multiple rare variants have been suggested as accounting for some of the associations with common single nucleotide polymorphisms identified in genome-wide association studies or possibly some of the as yet undiscovered heritability. We consider the power of various approaches to designing substudies aimed at using next-generation sequencing technologies to discover novel variants and to select some subsets that are possibly causal for genotyping in the original case-control study and testing for association using various weighted sum indices. We find that the selection of variants based on the statistical significance of the case-control difference in the subsample yields good power for testing rare variant indices in the main study, and that multivariate models including both the summary index of rare variants and the associated common single nucleotide polymorphisms can distinguish which is the causal factor. By simulation, we explore the effects of varying the size of the discovery subsample, choice of index, and true causal model.     

Assessment of the food safety issues related to genetically modified foods

International consensus has been reached on the principles regarding evaluation of the food safety of genetically modified plants. The concept of substantial equivalence has been developed as part of a safety evaluation framework, based on the idea that existing foods can serve as a basis for comparing the properties of genetically modified foods with the appropriate counterpart. Application of the concept is not a safety assessment per se, but helps to identify similarities and differences between the existing food and the new product, which are then subject to further toxicological investigation. Substantial equivalence is a starting point in the safety evaluation, rather than an endpoint of the assessment. Consensus on practical application of the principle should be further elaborated. Experiences with the safety testing of newly inserted proteins and of whole genetically modified foods are reviewed, and limitations of current test methodologies are discussed. The development and validation of new profiling methods such as DNA microarray technology, proteomics, and metabolomics for the identification and characterization of unintended effects, which may occur as a result of the genetic modification, is recommended. The assessment of the allergenicity of newly inserted proteins and of marker genes is discussed. An issue that will gain importance in the near future is that of post-marketing surveillance of the foods derived from genetically modified crops. It is concluded, among others that, that application of the principle of substantial equivalence has proven adequate, and that no alternative adequate safety assessment strategies are available.     

POOR STATISTICAL PERFORMANCE OF THE MANTEL TEST IN PHYLOGENETIC COMPARATIVE ANALYSES

The Mantel test, based on comparisons of distance matrices, is commonly employed in comparative biology, but its statistical properties in this context are unknown. Here, we evaluate the performance of the Mantel test for two applications in comparative biology: testing for phylogenetic signal, and testing for an evolutionary correlation between two characters. We find that the Mantel test has poor performance compared to alternative methods, including low power and, under some circumstances, inflated type‐I error. We identify a remedy for the inflated type‐I error of three‐way Mantel tests using phylogenetic permutations; however, this test still has considerably lower power than independent contrasts. We recommend that use of the Mantel test should be restricted to cases in which data can only be expressed as pairwise distances among taxa.     

Evaluating manufacturing process profile comparability with multivariate equivalence testing: Case study of cell‐culture small scale model transfer

This article studies the Generalized Mahalanobis Distance (GMD) approach proposed by Hoffelder which measures the dissimilarity of two multivariate Gaussian distributions with arbitrary covariance matrices and unequal sample sizes. This investigation demonstrated that, with appropriate adjustment, the GMD approach can achieve the targeted nominal Type I error and provide sufficient power for testing equivalence between two profile populations. The adjusted GMD approach was applied to examine the equivalence of harvest profiles between a 12L small scale model and 2000L manufacturing scale in a transfer study performed at Sanofi Specialty Care Framingham Biologics. The harvest profiles were evaluated based on three critical parameters (Productivity, Lactate Production, and Total Cell Density) and deemed practically equivalent using a pre‐defined equivalence margin per the adjusted GMD approach. © 2017 American Institute of Chemical Engineers Biotechnol. Prog., 34:187–195, 2018     

The use of statistical tools in field testing of putative effects of genetically modified plants on nontarget organisms

To fulfill existing guidelines, applicants that aim to place their genetically modified (GM) insect‐resistant crop plants on the market are required to provide data from field experiments that address the potential impacts of the GM plants on nontarget organisms (NTO's). Such data may be based on varied experimental designs. The recent EFSA guidance document for environmental risk assessment (2010) does not provide clear and structured suggestions that address the statistics of field trials on effects on NTO's. This review examines existing practices in GM plant field testing such as the way of randomization, replication, and pseudoreplication. Emphasis is placed on the importance of design features used for the field trials in which effects on NTO's are assessed. The importance of statistical power and the positive and negative aspects of various statistical models are discussed. Equivalence and difference testing are compared, and the importance of checking the distribution of experimental data is stressed to decide on the selection of the proper statistical model. While for continuous data (e.g., pH and temperature) classical statistical approaches – for example, analysis of variance (ANOVA) – are appropriate, for discontinuous data (counts) only generalized linear models (GLM) are shown to be efficient. There is no golden rule as to which statistical test is the most appropriate for any experimental situation. In particular, in experiments in which block designs are used and covariates play a role GLMs should be used. Generic advice is offered that will help in both the setting up of field testing and the interpretation and data analysis of the data obtained in this testing. The combination of decision trees and a checklist for field trials, which are provided, will help in the interpretation of the statistical analyses of field trials and to assess whether such analyses were correctly applied.     

Colonization and/or mitochondrial selective sweeps across the North Atlantic intertidal assemblage revealed by multi‐taxa approximate Bayesian computation

Intertidal and subtidal communities of the western and eastern coasts of the North Atlantic Ocean were greatly affected by Pleistocene glaciations, with some taxa persisting on both coasts, and others recolonizing after being extirpated on one coast during the Last Glacial Maximum. In the original spirit of comparative phylogeography, we conducted a comparative analysis using mtDNA sequence data and a hierarchical approximate Bayesian computation (ABC) approach for testing these two scenarios across 12 intertidal and subtidal coastal invertebrates spanning the North Atlantic to determine the temporal dynamics of species membership of these two ephemeral communities. Conditioning on a low gene‐flow model, our results suggested that a colonization or mitochondrial selective sweep history was predominant across all taxa, with only the bivalve mollusc Mytilus edulis showing a history of trans‐Atlantic persistence. Conditioning on a high gene‐flow model weakened the support for this assemblage‐level demographic history. The predominance of a colonization‐type history also highlights concerns about analyses based on single‐locus data where genetic hitchhiking may be incorrectly inferred as colonization. In conclusion, driving factors in shifting species range distributions and membership of ephemeral coastal communities could be species‐specific environmental tolerances, species interactions, and/or stochastic demographic extinction. Through a re‐examination of a long‐standing question of North Atlantic phylogeography, we highlight the flexibility and statistical honesty of using a model‐based ABC approach.     

Using demographic model selection to untangle allopatric divergence and diversification mechanisms in the Rheum palmatum complex in the Eastern Asiatic Region

Allopatric divergence is often initiated by geological uplift and restriction to sky‐islands, climate oscillations, or river capture. However, it can be difficult to establish which mechanism was the most likely to generate the current phylogeographical structure of a species. Recently, genomic data in conjunction with a model testing framework have been applied to address this issue in animals. To test whether such an approach is also likely to be successful in plants, we used population genomic data of the Rheum palmatum complex from the Eastern Asiatic Region, in conjunction with biogeographical reconstruction and demographic model selection, to identify the potential mechanism(s) which have led to the current level of divergence. Our results indicate that the R. palmatum complex originated in the central Hengduan Mts and possibly in regions further east, and then dispersed westward and eastward resulting in genetically distinct lineages. Populations are likely to have diverged in refugia during climate oscillations followed by subsequent expansion and secondary contact. However, model simulations within the western lineage of the R. palmatum complex cannot reject a restriction to sky‐islands as a possible mechanism of diversification due to the genetically ambiguous position of one population. This highlights that genetically mixed populations might introduce ambiguity regarding the best diversification model in some cases. Although it might be possible to resolve this ambiguity using other data, sometimes this could prove to be difficult in complex biogeographical areas.     

Environmental risk assessment of GE plants under low-exposure conditions

The requirement for environmental risk assessment (ERA) of genetically engineered (GE) plants prior to large scale or commercial introduction into the environment is well established in national laws and regulations, as well as in international agreements. Since the first introductions of GE plants in commercial agriculture in the 1990s, a nearly universal paradigm has emerged for conducting these assessments based on a few guiding principles. These include the concept of case-by-case assessment, the use of comparative assessments, and a focus of the ERA on characteristics of the plant, the introduced trait, and the receiving environment as well as the intended use. In practice, however, ERAs for GE plants have frequently focused on achieving highly detailed characterizations of potential hazards at the expense of consideration of the relevant levels of exposure. This emphasis on exhaustive hazard characterization can lead to great difficulties when applied to ERA for GE plants under low-exposure conditions. This paper presents some relevant considerations for conducting an ERA for a GE plant in a low-exposure scenario in the context of the generalized ERA paradigm, building on discussions and case studies presented during a session at ISBGMO 12.     

Tests for establishing compatibility of an observed genotype distribution with Hardy-Weinberg equilibrium in the case of a biallelic locus

The classical chi(2)-procedure for the assessment of genetic equilibrium is tailored for establishing lack rather than goodness of fit of an observed genotype distribution to a model satisfying the Hardy-Weinberg law, and the same is true for the exact competitors to the large-sample procedure, which have been proposed in the biostatistical literature since the late 1930s. In this contribution, the methodology of statistical equivalence testing is adopted for the construction of tests for problems in which the assumption of approximate compatibility of the genotype distribution actually sampled with Hardy-Weinberg equilibrium (HWE) plays the role of the alternative hypothesis one aims to establish. The result of such a construction highly depends on the choice of a measure of distance to be used for defining an indifference zone containing those genotype distributions whose degree of disequilibrium shall be considered irrelevant. The first such measure proposed here is the Euclidean distance of the true parameter vector from that of a genotype distribution with identical allele frequencies being in strict HWE. The second measure is based on the (scalar) parameter of the distribution first introduced into the present context by Stevens (1938, Annals of Eugenics 8, 377-383). The first approach leads to a nonconditional test (which nevertheless can be carried out in a numerically exact way), the second to an exact conditional test shown to be uniformly most powerful unbiased (UMPU) for the associated pair of hypotheses. Both tests are compared in terms of the exact power attained against the class of those specific alternatives under which HWE is strictly satisfied.     

Assessing Equivalence Tests with Respect to their Expected p‐Value

Monte‐Carlo simulation methods are commonly used for assessing the performance of statistical tests under finite sample scenarios. They help us ascertain the nominal level for tests with approximate level, e.g. asymptotic tests. Additionally, a simulation can assess the quality of a test on the alternative. The latter can be used to compare new tests and established tests under certain assumptions in order to determinate a preferable test given characteristics of the data. The key problem for such investigations is the choice of a goodness criterion. We expand the expected p‐value as considered by Sackrowitz and Samuel‐Cahn (1999) to the context of univariate equivalence tests. This presents an effective tool to evaluate new purposes for equivalence testing because of its independence of the distribution of the test statistic under null‐hypothesis. It helps to avoid the often tedious search for the distribution under null‐hypothesis for test statistics which have no considerable advantage over yet available methods. To demonstrate the usefulness in biometry a comparison of established equivalence tests with a nonparametric approach is conducted in a simulation study for three distributional assumptions.     

A Confidence‐Limit‐Based Approach to the Assessment of Hardy–Weinberg Equilibrium

The classical χ²‐procedure for the assessment of Hardy–Weinberg equilibrium (HWE) is tailored for detecting violations of HWE. However, many applications in genetic epidemiology require approximate compatibility with HWE. In a previous contribution to the field (Wellek, S. (2004). Biometrics, 60 , 694–703), the methodology of statistical equivalence testing was exploited for the construction of tests for problems in which the assumption of approximate compatibility of a given genotype distribution with HWE plays the role of the alternative hypothesis one aims to establish. In this article, we propose a procedure serving the same purpose but relying on confidence limits rather than critical bounds of a significance test. Interval estimation relates to essentially the same parametric function that was previously chosen as the target parameter for constructing an exact conditional UMPU test for equivalence with a HWE conforming genotype distribution. This population parameter is shown to have a direct genetic interpretation as a measure of relative excess heterozygosity. Confidence limits are constructed using both asymptotic and exact methods. The new approach is illustrated by reanalyzing genotype distributions obtained from published genetic association studies, and detailed guidance for choosing the equivalence margin is provided. The methods have been implemented in freely available SAS macros.     

Using statistical equivalence testing in clinical biofeedback research

The technique of statistical equivalence testing is described and recommended for use in clinical biofeedback research. Equivalence testing is valuable in aiding the interpretation of negative results or statistically significant results where effects are small in clinical terms. The method is also useful for establishing the similarity of treatment groups at baseline or for showing that the effect of a potentially confounding variable is tolerably small. Finally, equivalence testing is recommended as a method for documenting the equivalence of biofeedback therapy to proven conventional medical therapies in clinical equivalence trials. Examples, drawn from published literature, are provided.     

Statistical inference and power analysis for direct and spillover effects in two-stage randomized experiments

Two-stage randomized experiments become an increasingly popular experimental design for causal inference when the outcome of one unit may be affected by the treatment assignments of other units in the same cluster. In this paper, we provide a methodological framework for general tools of statistical inference and power analysis for two-stage randomized experiments. Under the randomization-based framework, we consider the estimation of a new direct effect of interest as well as the average direct and spillover effects studied in the literature. We provide unbiased estimators of these causal quantities and their conservative variance estimators in a general setting. Using these results, we then develop hypothesis testing procedures and derive sample size formulas. We theoretically compare the two-stage randomized design with the completely randomized and cluster randomized designs, which represent two limiting designs. Finally, we conduct simulation studies to evaluate the empirical performance of our sample size formulas. For empirical illustration, the proposed methodology is applied to the randomized evaluation of the Indian National Health Insurance Program. An open-source software package is available for implementing the proposed methodology.     

Farm Scale Evaluations of spring-sown genetically modified herbicide-tolerant crops: a statistical assessment

Primary results from the Farm Scale Evaluations (FSEs) of spring-sown genetically modified herbicide-tolerant crops were published in 2003. We provide a statistical assessment of the results for count data, addressing issues of sample size (n), efficiency, power, statistical significance, variability and model selection. Treatment effects were consistent between rare and abundant species. Coefficients of variation averaged 73% but varied widely. High variability in vegetation indicators was usually offset by large n and treatment effects, whilst invertebrate indicators often had smaller n and lower variability; overall, achieved power was broadly consistent across indicators. Inferences about treatment effects were robust to model misspecification, justifying the statistical model adopted. As expected, increases in n would improve detectability of effects whilst, for example, halving n would have resulted in a loss of significant results of about the same order. 40% of the 531 published analyses had greater than 80% power to detect a 1.5-fold effect; reducing n by one-third would most likely halve the number of analyses meeting this criterion. Overall, the data collected vindicated the initial statistical power analysis and the planned replication. The FSEs provide a valuable database of variability and estimates of power under various sample size scenarios to aid planning of more efficient future studies.     

GEIGER: investigating evolutionary radiations

SUMMARY: GEIGER is a new software package, written in the R language, to describe evolutionary radiations. GEIGER can carry out simulations, parameter estimation and statistical hypothesis testing. Additionally, GEIGER's simulation algorithms can be used to analyze the statistical power of comparative approaches. AVAILABILITY: This open source software is written entirely in the R language and is freely available through the Comprehensive R Archive Network (CRAN) at http://cran.r-project.org/.     

Integrating coalescent and ecological niche modeling in comparative phylogeography

Understanding the factors that contribute to the formation of population genetic structure is a central goal of phylogeographic research, but achieving this goal can be complicated by the stochastic variance inherent to genetic processes. Statistical approaches to testing phylogeographic hypotheses accommodate this stochasticity by evaluating competing models of putative historical population structure, often by simulating null distributions of the expected variance. The effectiveness of these tests depends on the biological realism of the models. Information from the fossil record can aid in reconstructing the historical distributions of some taxa. However, for the majority of taxa, which lack sufficient fossils, paleodistributional modeling can provide valuable spatial-geographic data concerning ancestral distributions. Paleodistributional models are generated by projecting ecological niche models, which predict the current distribution of each species, onto a model of past climatic conditions. Here, we generate paleodistributional models describing the suitable habitat during the last glacial maximum for lineages from the mesic forests of the Pacific Northwest of North America, and use these models to generate alternative phylogeographic hypotheses. Coalescent simulations are then used to test these hypotheses to improve our understanding of the historical events that promoted the formation of population genetic structure in this ecosystem. Results from Pacific Northwest mesic forest organisms demonstrate the utility of these combined approaches. Paleodistribution models and population genetic structure are congruent across three amphibian lineages, suggesting that they have responded in a concerted manner to environmental change. Two other species, a willow and a water vole, despite being currently codistributed and having similar population genetic structure, were predicted by the paleodistributional model to have had markedly different distributions during the last glacial maximum. This suggests that congruent phylogeographic patterns can arise from incongruent ancestral distributions. Paleodistributional models introduce a much-needed spatial-geographic perspective to statistical phylogeography. In conjunction with coalescent models of population genetic structure, they have the potential to improve our understanding of the factors that promote population divergence and ultimately produce regional patterns of biodiversity.     

Spatio-temporal modelling and assessment of within-species phenological variability using thermal time methods

Phenological observations of flowering date, budding date or senescence provide very valuable time series. They hold out the prospect for relating plant growth to environmental and climatic factors and hence for engendering a better understanding of plant physiology under natural conditions. The statistical establishment of associations between time series of phenological data and climatic factors provides a means of aiding forecasts of the biological impacts of future climatic change. However, it must be kept in mind that plant growth and behaviour vary spatially as well as temporally. Environmental, climatic and genetic diversity can give rise to spatially structured variation on a range of scales. The variations extend from large-scale geographical (clinal) trends, through medium-scale population and sub-population fluctuations, to micro-scale differentiation among neighbouring plants, where spatially close individuals are found to be genetically more alike than those some distance apart. We developed spatio-temporal phenological models that allow observations from multiple locations to be analysed simultaneously. We applied the models to the first-flowering dates of Prunus padus and Tilia cordata from localities as far apart as Norway and the Caucasus. Our growing-degree-day approach yielded a good fit to the available phenological data and yet involved only a small number of model parameters. It indicated that plants should display different sensitivities to temperature change according to their geographical location and the time of year at which they flower. For spring-flowering plants, we found strong temperature sensitivities for islands and archipelagos with oceanic climates, and low sensitivities in the interiors of continents.     

Association Tests of Multiple Phenotypes: ATeMP

Joint analysis of multiple phenotypes has gained growing attention in genome-wide association studies (GWASs), especially for the analysis of multiple intermediate phenotypes which measure the same underlying complex human disorder. One of the multivariate methods, MultiPhen (O’ Reilly et al. 2012), employs the proportional odds model to regress a genotype on multiple phenotypes, hence ignoring the phenotypic distributions. Despite the flexibilities of MultiPhen, the properties and performance of MultiPhen are not well understood, especially when the phenotypic distributions are non-normal. In fact, it is well known in the statistical literature that the estimation is attenuated when the explanatory variables contain measurement errors. In this study, we first established an equivalence relationship between MultiPhen and the generalized Kendall tau association test, shedding light on why MultiPhen can perform well for joint association analysis of multiple phenotypes. Through the equivalence, we show that MultiPhen may lose power when the phenotypes are non-normal. To maintain the power, we propose two solutions (ATeMP-rn and ATeMP-or) to improve MultiPhen, and demonstrate their effectiveness through extensive simulation studies and a real case study from the Guangzhou Twin Eye Study.