Testing Equality of Means and Simultaneous Confidence Intervals in Repeated Measures with Missing Data

In this paper, repeated measures with intraclass correlation model is considered when the observations are missing at random. An exact test for the equality of the mean components and simultaneous confidence intervals (Scheffé and Bonferroni inequality types) are given for linear contrasts of the mean components when the missing observations are of a monotone type. When the missing observations are not of the monotone type, the maximum likelihood estimates are obtained numerically by iterative methods given in Srivastava and Carter (1986). These estimators are then used to obtain asymptotic tests and confidence intervals for the equality of mean components and linear contrasts, respectively. An example is given to illustrate the method.     

Simultaneous small-sample multivariate Bernoulli confidence intervals

A technique based on the bootstrap is presented for assessing the simultaneous confidence level of k small-sample confidence intervals for multivariate Bernoulli marginal frequencies. The small-sample intervals used are those of Clopper and Pearson (1934, Biometrika 26, 404-413) and require iterative computation. To estimate the simultaneous confidence level, the multivariate Bernoulli vectors are resampled via the bootstrap and the Clopper-Pearson intervals recomputed on each pseudosample. The bootstrap estimate is then the proportion of times (computed via Monte Carlo) that all the k intervals computed by resampling contain the original sample frequencies. The technique is applied to single-sample HLA data.     

Sharp simultaneous confidence intervals for the means of selected populations with application to microarray data analysis

Summary Simultaneous inference for a large number, N, of parameters is a challenge. In some situations, such as microarray experiments, researchers are only interested in making inference for the K parameters corresponding to the K most extreme estimates. Hence it seems important to construct simultaneous confidence intervals for these K parameters. The naïve simultaneous confidence intervals for the K means (applied directly without taking into account the selection) have low coverage probabilities. We take an empirical Bayes approach (or an approach based on the random effect model) to construct simultaneous confidence intervals with good coverage probabilities. For N= 10,000 and K= 100, typical for microarray data, our confidence intervals could be 77% shorter than the naïve K‐dimensional simultaneous intervals.     

Simultaneous confidence intervals for comparing biodiversity indices estimated from overdispersed count data

Diversity indices might be used to assess the impact of treatments on the relative abundance patterns in species communities. When several treatments are to be compared, simultaneous confidence intervals for the differences of diversity indices between treatments may be used. The simultaneous confidence interval methods described until now are either constructed or validated under the assumption of the multinomial distribution for the abundance counts. Motivated by four example data sets with background in agricultural and marine ecology, we focus on the situation when available replications show that the count data exhibit extra‐multinomial variability. Based on simulated overdispersed count data, we compare previously proposed methods assuming multinomial distribution, a method assuming normal distribution for the replicated observations of the diversity indices and three different bootstrap methods to construct simultaneous confidence intervals for multiple differences of Simpson and Shannon diversity indices. The focus of the simulation study is on comparisons to a control group. The severe failure of asymptotic multinomial methods in overdispersed settings is illustrated. Among the bootstrap methods, the widely known Westfall–Young method performs best for the Simpson index, while for the Shannon index, two methods based on stratified bootstrap and summed count data are preferable. The methods application is illustrated for an example.     

Comparison of SANOVA Procedure of Simultaneous Testing Hypotheses to Others

In this paper a procedure SANOVA of simultaneous testing hypotheses is compared with others used in analysis of variance in a fixed linear model. The geometrical relation between SANOVA and Scheffé's confidence regions is discussed. It is shown that individual confidence intervals from SANOVA procedure are not longer than Scheffe's, Dunnett's and Tukey's ones. The cases, when they are the same are indicated. Theoretical considerations are illustrated by a practical example.     

Simultaneous inference for ratios of linear combinations of general linear model parameters

Consider a general linear model with p -dimensional parameter vector beta and i.i.d. normal errors. Let K(1), ..., K(k ), and L be linearly independent vectors of constants such that L(T)beta not equal 0. We describe exact simultaneous tests for hypotheses that Ki(T)beta/L(T)beta equal specified constants using one-sided and two-sided alternatives, and describe exact simultaneous confidence intervals for these ratios. In the case where the confidence set is a single bounded contiguous set, we describe what we claim are the best possible conservative simultaneous confidence intervals for these ratios - best in that they form the minimum k -dimensional hypercube enclosing the exact simultaneous confidence set. We show that in the case of k = 2, this "box" is defined by the minimum and maximum values for the two ratios in the simultaneous confidence set and that these values are obtained via one of two sources: either from the solutions to each of four systems of equations or at points along the boundary of the simultaneous confidence set where the correlation between two t variables is zero. We then verify that these intervals are narrower than those previously presented in the literature.     

Assessing Conformance with Benford’s Law: Goodness-Of-Fit Tests and Simultaneous Confidence Intervals

Benford’s Law is a probability distribution for the first significant digits of numbers, for example, the first significant digits of the numbers 871 and 0.22 are 8 and 2 respectively. The law is particularly remarkable because many types of data are considered to be consistent with Benford’s Law and scientists and investigators have applied it in diverse areas, for example, diagnostic tests for mathematical models in Biology, Genomics, Neuroscience, image analysis and fraud detection. In this article we present and compare statistically sound methods for assessing conformance of data with Benford’s Law, including discrete versions of Cramér-von Mises (CvM) statistical tests and simultaneous confidence intervals. We demonstrate that the common use of many binomial confidence intervals leads to rejection of Benford too often for truly Benford data. Based on our investigation, we recommend that the CvM statistic Ud2, Pearson’s chi-square statistic and 100(1 − α)% Goodman’s simultaneous confidence intervals be computed when assessing conformance with Benford’s Law. Visual inspection of the data with simultaneous confidence intervals is useful for understanding departures from Benford and the influence of sample size.     

A model for bacterial activity in the lung

A bivariate model is proposed for the analysis of data from experiments concerning bacterial activity in the mouse lung. The parameters of the model are discussed and estimated. Using the likelihood ratio statistic, simultaneous confidence intervals are constructed. The improvements over previously used models are then presented.     

Simultaneous confidence intervals from randomization tests with application in testing bioequivalence with multiple endpoints

We propose a method to construct simultaneous confidence intervals for a parameter vector from inverting a series of randomization tests (RT). The randomization tests are facilitated by an efficient multivariate Robbins–Monro procedure that takes the correlation information of all components into account. The estimation method does not require any distributional assumption of the population other than the existence of the second moments. The resulting simultaneous confidence intervals are not necessarily symmetric about the point estimate of the parameter vector but possess the property of equal tails in all dimensions. In particular, we present the constructing the mean vector of one population and the difference between two mean vectors of two populations. Extensive simulation is conducted to show numerical comparison with four methods. We illustrate the application of the proposed method to test bioequivalence with multiple endpoints on some real data.     

Obtaining Critical Values for Simultaneous Confidence Intervals and Multiple Testing

There are many situations where it is desired to make simultaneous tests or give simultaneous confidence intervals for linear combinations (contrasts) of population or treatment means. Somerville (1997, 1999) developed algorithms for calculating the critical values for a large class of simultaneous tests and simultaneous confidence intervals. Fortran 90 and SAS‐IML batch programs and interactive programs were developed. These programs calculate the critical values for 15 different simultaneous confidence interval procedures (and the corresponding simultaneous tests) and for arbitrary procedures where the user specifies a combination of one and two sided contrasts. The programs can also be used to obtain the constants for “step‐down” testing of multiple hypotheses. This paper gives examples of the use of the algorithms and programs and illustrates their versatility and generality. The designs need not be balanced, multiple covariates may be present and there may be many missing values. The use of multiple regression and dummy variables to obtain the required variance covariance matrix is illustrated. Under weak normality assumptions the methods are “exact” and make the use of approximate methods or “simulation” unnecessary.     

Determining the optimal flexion-extension axis of the elbow in vivo - a study of interobserver and intraobserver reliability

In the current study the interobserver and intraobserver reliability of a recently developed method to obtain the position and orientation vectors of the flexion-extension axis of the elbow in vivo is determined. The method uses the Flock of Birds six degrees-of-freedom electromagnetic tracking device. Ten subjects performed three trials comprising five flexion and extension cycles. The movements of the forearm with respect to the upper arm were recorded. Observer A measured two trials and observer B measured one trial. Optimal instantaneous helical axes were calculated in a humeral coordinate system for each trial. Intraclass correlation coefficients and 99% confidence intervals were computed to compare the three measurements. Zero was in the range of all the narrow confidence intervals, which is strong indication for resemblance. Interobserver intraclass correlation coefficients values for orientation vectors were good to excellent and intraobserver values were fair to good. The intraclass correlation coefficients values for position vectors were lower, probably due to the lack of variance between subjects. It is concluded that the method is reliable and can be used in certain clinical settings.     

No Longer Confidential: Estimating the Confidence of Individual Regression Predictions

Quantitative predictions in computational life sciences are often based on regression models. The advent of machine learning has led to highly accurate regression models that have gained widespread acceptance. While there are statistical methods available to estimate the global performance of regression models on a test or training dataset, it is often not clear how well this performance transfers to other datasets or how reliable an individual prediction is–a fact that often reduces a user’s trust into a computational method. In analogy to the concept of an experimental error, we sketch how estimators for individual prediction errors can be used to provide confidence intervals for individual predictions. Two novel statistical methods, named CONFINE and CONFIVE, can estimate the reliability of an individual prediction based on the local properties of nearby training data. The methods can be applied equally to linear and non-linear regression methods with very little computational overhead. We compare our confidence estimators with other existing confidence and applicability domain estimators on two biologically relevant problems (MHC–peptide binding prediction and quantitative structure-activity relationship (QSAR)). Our results suggest that the proposed confidence estimators perform comparable to or better than previously proposed estimation methods. Given a sufficient amount of training data, the estimators exhibit error estimates of high quality. In addition, we observed that the quality of estimated confidence intervals is predictable. We discuss how confidence estimation is influenced by noise, the number of features, and the dataset size. Estimating the confidence in individual prediction in terms of error intervals represents an important step from plain, non-informative predictions towards transparent and interpretable predictions that will help to improve the acceptance of computational methods in the biological community.     

Multiple Comparisons in Nonlinear Repeated Measurements

It is interest to compare functions of parameters in nonlinear models for repeated measurements. In a pharmacokinetic model, the maximum value of the model would be a nonlinear function of some unknown parameters. In this paper, simultaneous confidence intervals of functions of parameters in a nonlinear model for repeated mesurement data are considered to compare the populations.     

Confidence interval estimation of the intraclass correlation coefficient for binary outcome data

We obtain closed-form asymptotic variance formulae for three point estimators of the intraclass correlation coefficient that may be applied to binary outcome data arising in clusters of variable size. Our results include as special cases those that have previously appeared in the literature (Fleiss and Cuzick, 1979, Applied Psychological Measurement 3, 537-542; Bloch and Kraemer, 1989, Biometrics 45, 269-287; Altaye, Donner, and Klar, 2001, Biometrics 57, 584-588). Simulation results indicate that confidence intervals based on the estimator proposed by Fleiss and Cuzick provide coverage levels close to nominal over a wide range of parameter combinations. Two examples are presented.     

Confidence regions for treatment effects in subgroups in biomarker stratified designs

Subgroup analysis has important applications in the analysis of controlled clinical trials. Sometimes the result of the overall group fails to demonstrate that the new treatment is better than the control therapy, but for a subgroup of patients, the treatment benefit may exist; or sometimes, the new treatment is better for the overall group but not for a subgroup. Hence we are interested in constructing a simultaneous confidence interval for the difference of the treatment effects in a subgroup and the overall group. Subgroups are usually formed on the basis of a predictive biomarker such as age, sex, or some genetic marker. While, for example, age can be detected precisely, it is often only possible to detect the biomarker status with a certain probability. Because patients detected with a positive or negative biomarker may not be truly biomarker positive or negative, responses in the subgroups depend on the treatment therapy as well as on the sensitivity and specificity of the assay used in detecting the biomarkers. In this work, we show how (approximate) simultaneous confidence intervals and confidence ellipsoid for the treatment effects in subgroups can be found for biomarker stratified clinical trials using a normal framework with normally distributed or binary data. We show that these intervals maintain the nominal confidence level via simulations.     

Population size estimation in a two-list surveillance system with a discrete covariate

Summary .   Capture–recapture methods are widely adopted to estimate sizes of populations of public health interest using information from surveillance systems. For a two-list surveillance system with a discrete covariate, a population is divided into several subpopulations. A unified framework is proposed in which the logits of presence probabilities are decomposed into case effects and list effects. The estimators for the whole population and subpopulation sizes, their adjusted versions, and asymptotic standard errors admit closed-form expressions. Asymptotic and bootstrap individual and simultaneous confidence intervals are easily constructed. Conditional likelihood ratio tests are used to select one from three possible models. Real examples are investigated.     

A comparison of confidence interval methods for the intraclass correlation coefficient

Different methods of obtaining confidence intervals for the intraclass correlation coefficient rho in the unbalanced one-way random-effects model are investigated, focusing on applications to family studies. Methods based on simple modifications of formulas for the case of equal group sizes are found to provide adequate coverage at small to moderate values of rho. A method based on the large-sample standard error of the sample intraclass correlation, as derived by Smith (1956, Annals of Human Genetics 21, 363-373), is shown to provide consistently good coverage at all values of rho. A method proposed by Thomas and Hultquist (1978, Annals of Statistics 6, 582-587) also provides consistently good coverage, but generates mean interval widths substantially greater than those generated by Smith's method at values of rho likely to arise in practice.     

Functional regression analysis of fluorescence curves

Summary .  Fluorescence curves are useful for monitoring changes in photosynthesis activity. Various summary measures have been used to quantify differences among fluorescence curves corresponding to different treatments, but these approaches may forfeit valuable information. As each individual fluorescence curve is a functional observation, it is natural to consider a functional regression model. The proposed model consists of a nonparametric component capturing the general form of the curves and a semiparametric component describing the differences among treatments and allowing comparisons of treatments. Several graphical model-checking approaches are introduced. Both approximate, asymptotic confidence intervals as well as simulation-based confidence intervals are available. Analysis of data from a crop experiment using the proposed model shows that the salient features in the fluorescence curves are captured adequately. The proposed functional regression model is useful for analysis of high throughput fluorescence curve data from regular monitoring or screening of plant growth.     

Exact inference for growth curves with intraclass correlation structure

We consider repeated observations taken over time for each of several subjects. For example, one might consider the growth curve of a cohort of babies over time. We assume a simple linear growth curve model. Exact results based on sufficient statistics (exact tests of the null hypothesis that a coefficient is zero, or exact confidence intervals for coefficients) are not available to make inference on regression coefficients when an intraclass correlation structure is assumed. This paper will demonstrate that such exact inference is possible using generalized inference.     

A program to compare genetic differentiation statistics across loci using resampling of individuals and loci

Comparisons of genetic differentiation across populations based on different loci can provide insight into the evolutionary patterns acting on various regions of genomes. Here, we develop a program to statistically compare population genetic differentiation statistics (F(ST) or G'(ST) ) calculated from different loci. The program employs a routine that resamples either or both of individuals and loci and calculates a bootstrap confidence interval in the statistics. Resampling individuals is important when fewer than 25 individuals are sampled per population and when confidence intervals are required for individual loci. Resampling loci provides confidence intervals for sets of loci, such as a set presumed to be neutral, but can be anticonservative if fewer than 20 loci are analysed. We demonstrate the program using previously published data on the genetic differentiation at a major histocompatibility complex locus and at microsatellite loci across 10 populations of the guppy (Poecilia reticulata).