Optimal Testing for Serial Correlation in a Large Number of Small Samples

In recent years, there has been an increased awareness of the potential one-sided nature of many testing problems in applied sciences. Usually, these testing problems can be reduced, either by conditioning on sufficient statistics or by invariant techniques. COX and SOLOMON (1988) considered testing the serial correlation coefficient of a stationary first order autoregressive process and concentrated on four independent samples, with each of size three. We outline a general method for testing the serial correlation coefficient, using locally best invariant, point optimal invariant and locally most mean powerful invariant test procedures. The first procedure optimizes power near the null hypothesis, the second optimizes it at a pre-determined point away from the null while the third optimizes the average curvature of the power hypersurface in the neighbourhood of the null hypothesis.     

Testing for cubic smoothing splines under dependent data

Summary In most research on smoothing splines the focus has been on estimation, while inference, especially hypothesis testing, has received less attention. By defining design matrices for fixed and random effects and the structure of the covariance matrices of random errors in an appropriate way, the cubic smoothing spline admits a mixed model formulation, which places this nonparametric smoother firmly in a parametric setting. Thus nonlinear curves can be included with random effects and random coefficients. The smoothing parameter is the ratio of the random‐coefficient and error variances and tests for linear regression reduce to tests for zero random‐coefficient variances. We propose an exact F‐test for the situation and investigate its performance in a real pine stem data set and by simulation experiments. Under certain conditions the suggested methods can also be applied when the data are dependent.     

Test Based on Independent,but Non-identically Distributed Random Variables

The binomial test is applied for the problem of testing a hypothesis based on a sample of independent, but non-identically distributed random variables. The used basic idea is that each random variable indicates the presence of the hypothesis. Hence each random variable is transformed such that the binomial test can be used as a simple procedure.     

Basing the Analysis of Comparative Bioavailability Trials on an Individualized Statistical Definition of Equivalence

The conventional definition of bioequivalence in terms of population means only, is criticized for lacking relevance to the individual subject. Both approaches to bioequivalence assessment proposed here for avoiding this shortcoming, focus on the probability of an event induced by the response of a randomly selected subject to two formulations of a given active agent. The first approach leads to converting the basic idea underlying the well-known 75-rule into an exact statistical procedure. The second approach is of a parametric nature. It reduces bioequivalence assessment to testing against the alternative hypothesis that the standardized expected value of a Gaussian distribution is contained in a short interval around zero. For this problem, an exact optimal solution is provided as well.     

On the finite sample distribution of the likelihood ratio statistic for testing heterogeneity in meta-analysis

In meta-analysis, hypothesis testing is one of the commonly used approaches for assessing whether heterogeneity exists in effects between studies. The literature concluded that the Q-statistic is clearly the best choice and criticized the performance of the likelihood ratio test in terms of the type I error control and power. However, all the criticism for the likelihood ratio test is based on the use of a mixture of two chi-square distributions with 0 and 1 degrees of freedom, which is justified only asymptotically. In this study, we develop a novel method to derive the finite sample distribution of the likelihood ratio test and restricted likelihood ratio test statistics for testing the zero variance component in the random effects model for meta-analysis. We also extend this result to the heterogeneity test when metaregression is applied. A numerical study shows that the proposed statistics have superior performance to the Q-statistic, especially when the number of studies collected for meta-analysis is small to moderate.     

Valid inference in random effects meta-analysis

The standard approach to inference for random effects meta-analysis relies on approximating the null distribution of a test statistic by a standard normal distribution. This approximation is asymptotic on k, the number of studies, and can be substantially in error in medical meta-analyses, which often have only a few studies. This paper proposes permutation and ad hoc methods for testing with the random effects model. Under the group permutation method, we randomly switch the treatment and control group labels in each trial. This idea is similar to using a permutation distribution for a community intervention trial where communities are randomized in pairs. The permutation method theoretically controls the type I error rate for typical meta-analyses scenarios. We also suggest two ad hoc procedures. Our first suggestion is to use a t-reference distribution with k-1 degrees of freedom rather than a standard normal distribution for the usual random effects test statistic. We also investigate the use of a simple t-statistic on the reported treatment effects.     

Testing random effects in linear mixed‐effects models with serially correlated errors

In linear mixed‐effects models, random effects are used to capture the heterogeneity and variability between individuals due to unmeasured covariates or unknown biological differences. Testing for the need of random effects is a nonstandard problem because it requires testing on the boundary of parameter space where the asymptotic chi‐squared distribution of the classical tests such as likelihood ratio and score tests is incorrect. In the literature several tests have been proposed to overcome this difficulty, however all of these tests rely on the restrictive assumption of i.i.d. measurement errors. The presence of correlated errors, which often happens in practice, makes testing random effects much more difficult. In this paper, we propose a permutation test for random effects in the presence of serially correlated errors. The proposed test not only avoids issues with the boundary of parameter space, but also can be used for testing multiple random effects and any subset of them. Our permutation procedure includes the permutation procedure in Drikvandi, Verbeke, Khodadadi, and Partovi Nia (2013) as a special case when errors are i.i.d., though the test statistics are different. We use simulations and a real data analysis to evaluate the performance of the proposed permutation test. We have found that random slopes for linear and quadratic time effects may not be significant when measurement errors are serially correlated.     

Testing homogeneity in Weibull-regression models

In survival studies with families or geographical units it may be of interest testing whether such groups are homogeneous for given explanatory variables. In this paper we consider score type tests for group homogeneity based on a mixing model in which the group effect is modelled as a random variable. As opposed to hazard-based frailty models, this model presents survival times that conditioned on the random effect, has an accelerated failure time representation. The test statistics requires only estimation of the conventional regression model without the random effect and does not require specifying the distribution of the random effect. The tests are derived for a Weibull regression model and in the uncensored situation, a closed form is obtained for the test statistic. A simulation study is used for comparing the power of the tests. The proposed tests are applied to real data sets with censored data.     

A simple permutation-type method for testing circular uniformity with correlated angular measurements

A simple method is provided for testing uniformity on the circle that allows dependence among repeated angular measurements on the same subject. Our null hypothesis is that the distribution of repeated angles is unaffected by rotation. This null can be evaluated with any test of uniformity by using a null reference distribution obtained by simulation, where each subject's vector of angles is rotated by a random amount. A new weighted version of the univariate Rayleigh test of circular uniformity is proposed.     

Most Powerful Permutation Invariant Tests for Relatedness Hypotheses Using Genotypic Data

The problem of inferring kinship structure among a sample of individuals using genetic markers is considered with the objective of developing hypothesis tests for genetic relatedness with nearly optimal properties. The class of tests considered are those that are constrained to be permutation invariant, which in this context defines tests whose properties do not depend on the labeling of the individuals. This is appropriate when all individuals are to be treated identically from a statistical point of view. The approach taken is to derive tests that are probably most powerful for a permutation invariant alternative hypothesis that is, in some sense, close to a null hypothesis of mutual independence. This is analagous to the locally most powerful test commonly used in parametric inference. Although the resulting test statistic is a U-statistic, normal approximation theory is found to be inapplicable because of high skewness. As an alternative it is found that a conditional procedure based on the most powerful test statistic can calculate accurate significance levels without much loss in power. Examples are given in which this type of test proves to be more powerful than a number of alternatives considered in the literature, including Queller and Goodknight's (1989) estimate of genetic relatedness, the average number of shared alleles (Blouin, 1996), and the number of feasible sibling triples (Almudevar and Field, 1999).     

Exact Tests for Comparing Two Paired Proportions with Incomplete Data

Various asymptotic test procedures have been developed previously for testing the equality of two binomial proportions with partially incomplete paired data. Test procedures that discard incomplete observations have been shown to be less powerful than those procedures that utilize all available observations. On the other hand, asymptotic test procedures that utilize all available observations may not be reliable in small‐sample problems or sparse data structures. In this article, unconditional exact test procedures are proposed for testing the equality of two paired binomial proportions with partially incomplete paired data under a random mechanism. The proposed unconditional exact test methods are illustrated with real data from a neurological study. Empirical studies are conducted to investigate the performance of these and other test procedures with respect to size and power. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Testing for Independence of Binary Responses

In the context of experiments involving visual inspection of random dot patterns the problem of testing the null hypothesis of independence of binary responses is considered. A flexible model for dependence between binary responses is proposed. Two tests, optimal under different versions of the model, are derived. These two tests turn out to involve the same computations as the Wilcoxon two sample test and the runs test respectively.     

Score Tests for Exploring Complex Models: Application to HIV Dynamics Models

In biostatistics, more and more complex models are being developed. This is particularly the case in system biology. Fitting complex models can be very time‐consuming, since many models often have to be explored. Among the possibilities are the introduction of explanatory variables and the determination of random effects. The particularity of this use of the score test is that the null hypothesis is not itself very simple; typically, some random effects may be present under the null hypothesis. Moreover, the information matrix cannot be computed, but only an approximation based on the score. This article examines this situation with the specific example of HIV dynamics models. We examine the score test statistics for testing the effect of explanatory variables and the variance of random effect in this complex situation. We study type I errors and the statistical powers of this score test statistics and we apply the score test approach to a real data set of HIV‐infected patients.     

Assessment of the prevalence of vCJD through testing tonsils and appendices for abnormal prion protein

The objective of this study was to determine the age group or groups which will provide the most information on the potential size of the vCJD epidemic in Great Britain via the sampling of tonsil and appendix material to detect the presence of abnormal prion protein (PrP(Sc)). A subsidiary aim was to determine the degree to which such an anonymous age-stratified testing programme will reduce current uncertainties in the size of the epidemic in future years. A cohort- and time-stratified model was used to generate epidemic scenarios consistent with the observed vCJD case incidence. These scenarios, together with data on the age distribution of tonsillectomies and appendectomies, were used to evaluate the optimal age group and calendar time for undertaking testing and to calculate the range of epidemic sizes consistent with different outcomes. The analyses suggested that the optimal five-year age group to test is 25-29 years, although a random sample of appendix tissue from all age groups is nearly as informative. A random sample of tonsil tissue from all age groups is less informative, but the information content is improved if sampling is restricted to tissues removed from those over ten years of age. Based on the assumption that the test is able to detect infection in the last 75% of the incubation period, zero detected infections in an initial random sample of 1000 tissues would suggest that the epidemic will be less than 870,000 cases. If infections are detected, then the model prediction suggests that both relatively small epidemics (800+ cases if one is detected or 8300+ if two are detected) and larger epidemics (21,000+ cases if three or more are detected) are possible. It was concluded that testing will be most informative if undertaken using appendix tissues or tonsil tissues removed from those over ten years of age. Large epidemics can only be excluded if a small number of infections are detected and the test is able to detect infection early in the incubation period.     

A group sequential type design for three‐arm non‐inferiority trials with binary endpoints

The three‐arm design with a test treatment, an active control and a placebo group is the gold standard design for non‐inferiority trials if it is ethically justifiable to expose patients to placebo. In this paper, we first use the closed testing principle to establish the hierarchical testing procedure for the multiple comparisons involved in the three‐arm design. For the effect preservation test we derive the explicit formula for the optimal allocation ratios. We propose a group sequential type design, which naturally accommodates the hierarchical testing procedure. Under this proposed design, Monte Carlo simulations are conducted to evaluate the performance of the sequential effect preservation test when the variance of the test statistic is estimated based on the restricted maximum likelihood estimators of the response rates under the null hypothesis. When there are uncertainties for the placebo response rate, the proposed design demonstrates better operating characteristics than the fixed sample design.     

A general framework for subgroup detection via one-step value difference estimation

Recent statistical methodology for precision medicine has focused on either identification of subgroups with enhanced treatment effects or estimating optimal treatment decision rules so that treatment is allocated in a way that maximizes, on average, predefined patient outcomes. Less attention has been given to subgroup testing, which involves evaluation of whether at least a subgroup of the population benefits from an investigative treatment, compared to some control or standard of care. In this work, we propose a general framework for testing for the existence of a subgroup with enhanced treatment effects based on the difference of the estimated value functions under an estimated optimal treatment regime and a fixed regime that assigns everyone to the same treatment. Our proposed test does not require specification of the parametric form of the subgroup and allows heterogeneous treatment effects within the subgroup. The test applies to cases when the outcome of interest is either a time-to-event or a (uncensored) scalar, and is valid at the exceptional law. To demonstrate the empirical performance of the proposed test, we study the type I error and power of the test statistics in simulations and also apply our test to data from a Phase III trial in patients with hematological malignancies.     

The Residual Likelihood Ratio Test for the Variance Ratio in a Linear Model with Two Variance Components

Residual maximum likelihood has proved to be a successful approach to the estimation of variance components. In this paper, its counterpart in testing, the residual likelihood ratio test, is applied to testing the ratio of two variance components. The test is compared with the Wald test and the locally most powerful invariant test.     

Testing the predictive performance of distribution models

Distribution models are used to predict the likelihood of occurrence or abundance of a species at locations where census data are not available. An integral part of modelling is the testing of model performance. We compared different schemes and measures for testing model performance using 79 species from the North American Breeding Bird Survey. The four testing schemes we compared featured increasing independence between test and training data: resubstitution, random data hold‐out and two spatially segregated data hold‐out designs. The different testing measures also addressed different levels of information content in the dependent variable: regression R² for absolute abundance, squared correlation coefficient r² for relative abundance and AUC/Somer’s D for presence/absence. We found that higher levels of independence between test and training data lead to lower assessments of prediction accuracy. Even for data collected independently, spatial autocorrelation leads to dependence between random hold‐out test data and training data, and thus to inflated measures of model performance. While there is a general awareness of the importance of autocorrelation to model building and hypothesis testing, its consequences via violation of independence between training and testing data have not been addressed systematically and comprehensively before. Furthermore, increasing information content (from correctly classifying presence/absence, to predicting relative abundance, to predicting absolute abundance) leads to decreasing predictive performance. The current tests for presence/absence distribution models are typically overly optimistic because a) the test and training data are not independent and b) the correct classification of presence/absence has a relatively low information content and thus capability to address ecological and conservation questions compared to a prediction of abundance. Meaningful evaluation of model performance requires testing on spatially independent data, if the intended application of the model is to predict into new geographic or climatic space, which arguably is the case for most applications of distribution models.     

Cross‐sectional human immunodeficiency virus incidence estimation accounting for heterogeneity across communities

Accurate estimation of human immunodeficiency virus (HIV) incidence rates is crucial for the monitoring of HIV epidemics, the evaluation of prevention programs, and the design of prevention studies. Traditional cohort approaches to measure HIV incidence require repeatedly testing large cohorts of HIV‐uninfected individuals with an HIV diagnostic test (eg, enzyme‐linked immunosorbent assay) for long periods of time to identify new infections, which can be prohibitively costly, time‐consuming, and subject to loss to follow‐up. Cross‐sectional approaches based on the usual HIV diagnostic test and biomarkers of recent infection offer important advantages over standard cohort approaches, in terms of time, cost, and attrition. Cross‐sectional samples usually consist of individuals from different communities. However, small sample sizes limit the ability to estimate community‐specific incidence and existing methods typically ignore heterogeneity in incidence across communities. We propose a permutation test for the null hypothesis of no heterogeneity in incidence rates across communities, develop a random‐effects model to account for this heterogeneity and to estimate community‐specific incidence, and provide one way to estimate the coefficient of variation. We evaluate the performance of the proposed methods through simulation studies and apply them to the data from the National Institute of Mental Health Project ACCEPT, a phase 3 randomized controlled HIV prevention trial in Sub‐Saharan Africa, to estimate the overall and community‐specific HIV incidence rates.     

One‐Sided Test to Assess Correlation in Linear Logistic Models using Estimating Equations

A score‐type test is proposed for testing the hypothesis of independent binary random variables against positive correlation in linear logistic models with sparse data and cluster specific covariates. The test is developed for univariate and multivariate one‐sided alternatives. The main advantage of using score test is that it requires estimation of the model only under the null hypothesis, that in this case corresponds to the binomial maximum likelihood fit. The score‐type test is developed from a class of estimating equations with block‐diagonal structure in which the coefficients of the linear logistic model are estimated simultaneously with the correlation. The simplicity of the score test is illustrated in two particular examples.