Multiple comparisons distortions of parameter estimates

In experiments involving many variables, investigators typically use multiple comparisons procedures to determine differences that are unlikely to be the result of chance. However, investigators rarely consider how the magnitude of the greatest observed effect sizes may have been subject to bias resulting from multiple testing. These questions of bias become important to the extent investigators focus on the magnitude of the observed effects. As an example, such bias can lead to problems in attempting to validate results, if a biased effect size is used to power a follow-up study. An associated important consequence is that confidence intervals constructed using standard distributions may be badly biased. A bootstrap approach is used to estimate and adjust for the bias in the effect sizes of those variables showing strongest differences. This bias is not always present; some principles showing what factors may lead to greater bias are given and a proof of the convergence of the bootstrap distribution is provided.     

Instrumental noise estimates stabilize and quantify endothelial cell micro-impedance barrier function parameter estimates

A novel transcellular micro-impedance biosensor, referred to as the electric cell-substrate impedance sensor or ECIS, has become increasingly applied to the study and quantification of endothelial cell physiology. In principle, frequency dependent impedance measurements obtained from this sensor can be used to estimate the cell–cell and cell–matrix impedance components of endothelial cell barrier function based on simple geometric models. Few studies, however, have examined the numerical optimization of these barrier function parameters and established their error bounds. This study, therefore, illustrates the implementation of a multi-response Levenberg–Marquardt algorithm that includes instrumental noise estimates and applies it to frequency dependent porcine pulmonary artery endothelial cell impedance measurements. The stability of cell–cell, cell–matrix and membrane impedance parameter estimates based on this approach is carefully examined, and several forms of parameter instability and refinement illustrated. Including frequency dependent noise variance estimates in the numerical optimization reduced the parameter value dependence on the frequency range of measured impedances. The increased stability provided by a multi-response non-linear fit over one-dimensional algorithms indicated that both real and imaginary data should be used in the parameter optimization. Error estimates based on single fits and Monte Carlo simulations showed that the model barrier parameters were often highly correlated with each other. Independently resolving the different parameters can, therefore, present a challenge to the experimentalist and demand the use of non-linear multivariate statistical methods when comparing different sets of parameters.     

Background frequencies for residue variability estimates: BLOSUM revisited

Background  

Shannon entropy applied to columns of multiple sequence alignments as a score of residue conservation has proven one of the most fruitful ideas in bioinformatics. This straightforward and intuitively appealing measure clearly shows the regions of a protein under increased evolutionary pressure, highlighting their functional importance. The inability of the column entropy to differentiate between residue types, however, limits its resolution power.     

Genetic parameter estimates for pre-weaning weight traits in Dorper sheep

Genetic parameters were estimated for birth-, 42-day, and 100-day (weaning) weight in the Dorper flock of the Glen Agricultural Institute in South Africa. Direct heritability estimates of 0.11, 0.28 and 0.20 and maternal heritability estimates of 0.10, 0.10 and 0.10 were obtained for body weights at birth, 42 and 100 days, respectively. The corresponding genetic correlation estimates between direct and maternal effects were 0.35, −0.63 and −0.58, respectively. Both direct and maternal genetic correlation estimates among the traits were of moderate to high magnitude and positive. It is concluded that the traits can be improved by selection with no serious antagonisms among traits studied.     

Identifying sites under positive selection with uncertain parameter estimates.

Codon-based substitution models are routinely used to measure selective pressures acting on protein-coding genes. To this effect, the nonsynonymous to synonymous rate ratio (dN/dS = omega) is estimated. The proportion of amino-acid sites potentially under positive selection, as indicated by omega > 1, is inferred by fitting a probability distribution where some sites are permitted to have omega > 1. These sites are then inferred by means of an empirical Bayes or by a Bayes empirical Bayes approach that, respectively, ignores or accounts for sampling errors in maximum-likelihood estimates of the distribution used to infer the proportion of sites with omega > 1. Here, we extend a previous full-Bayes approach to include models with high power and low false-positive rates when inferring sites under positive selection. We propose some heuristics to alleviate the computational burden, and show that (i) full Bayes can be superior to empirical Bayes when analyzing a small data set or small simulated data, (ii) full Bayes has only a small advantage over Bayes empirical Bayes with our small test data, and (iii) Bayesian methods appear relatively insensitive to mild misspecifications of the random process generating adaptive evolution in our simulations, but in practice can prove extremely sensitive to model specification. We suggest that the codon model used to detect amino acids under selection should be carefully selected, for instance using Akaike information criterion (AIC).     

Red herrings revisited: spatial autocorrelation and parameter estimation in geographical ecology

There have been numerous claims in the ecological literature that spatial autocorrelation in the residuals of ordinary least squares (OLS) regression models results in shifts in the partial coefficients, which bias the interpretation of factors influencing geographical patterns. We evaluate the validity of these claims using gridded species richness data for the birds of North America, South America, Europe, Africa, the ex‐USSR, and Australia. We used richness in 110×110 km cells and environmental predictor variables to generate OLS and simultaneous autoregressive (SAR) multiple regression models for each region. Spatial correlograms of the residuals from each OLS model were then used to identify the minimum distance between cells necessary to avoid short‐distance residual spatial autocorrelation in each data set. This distance was used to subsample cells to generate spatially independent data. The partial OLS coefficients estimated with the full dataset were then compared to the distributions of coefficients created with the subsamples. We found that OLS coefficients generated from data containing residual spatial autocorrelation were statistically indistinguishable from coefficients generated from the same data sets in which short‐distance spatial autocorrelation was not present in all 22 coefficients tested. Consistent with the statistical literature on this subject, we conclude that coefficients estimated from OLS regression are not seriously affected by the presence of spatial autocorrelation in gridded geographical data. Further, shifts in coefficients that occurred when using SAR tended to be correlated with levels of uncertainty in the OLS coefficients. Thus, shifts in the relative importance of the predictors between OLS and SAR models are expected when small‐scale patterns for these predictors create weaker and more unstable broad‐scale coefficients. Our results indicate both that OLS regression is unbiased and that differences between spatial and nonspatial regression models should be interpreted with an explicit awareness of spatial scale.     

Effects of censoring on parameter estimates and power in genetic modeling.

Genetic and environmental influences on variance in phenotypic traits may be estimated with normal theory Maximum Likelihood (ML). However, when the assumption of multivariate normality is not met, this method may result in biased parameter estimates and incorrect likelihood ratio tests. We simulated multivariate normal distributed twin data under the assumption of three different genetic models. Genetic model fitting was performed in six data sets: multivariate normal data, discrete uncensored data, censored data, square root transformed censored data, normal scores of censored data, and categorical data. Estimates were obtained with normal theory ML (data sets 1-5) and with categorical data analysis (data set 6). Statistical power was examined by fitting reduced models to the data. When fitting an ACE model to censored data, an unbiased estimate of the additive genetic effect was obtained. However, the common environmental effect was underestimated and the unique environmental effect was overestimated. Transformations did not remove this bias. When fitting an ADE model, the additive genetic effect was underestimated while the dominant and unique environmental effects were overestimated. In all models, the correct parameter estimates were recovered with categorical data analysis. However, with categorical data analysis, the statistical power decreased. The analysis of L-shaped distributed data with normal theory ML results in biased parameter estimates. Unbiased parameter estimates are obtained with categorical data analysis, but the power decreases.     

When does parameter drift decrease the uncertainty in extinction risk estimates?

Halley (2003) proposed that parameter drift decreases the uncertainty in long‐range extinction risk estimates, because drift mitigates the extreme sensitivity of estimated risk to estimated mean growth rate. However, parameter drift has a second, opposing effect: it increases the uncertainty in parameter estimates from a given data set. When both effects are taken into account, parameter drift can increase, sometimes substantially, the uncertainty in risk estimates. The net effect depends sensitively on the type of drift and on which model parameters must be estimated from observational data on the population at risk. In general, unless many parameters are estimated from independent data, parameter drift increases the uncertainty in extinction risk. These findings suggest that more mechanistic PVA models, using long‐term data on key environmental variables and experiments to quantify their demographic impacts, offer the best prospects for escaping the high data requirements when extinction risk is estimated from observational data.     

Genetic parameter estimates for pre-weaning performance and reproduction traits in Markhoz goats

Genetic parameters for growth, mortality and reproductive performances of Markhoz goats were estimated from data collected during 1993–2010 at Markhoz goat Performance Testing Station in Sanandaj, Iran. For kid performance traits 3763 records were available for birth weight (BW), 2931 for weaning weight (WW), average daily gain (ADG) and Kleiber ratio (KR) (approximated as ADW/WW^0.75) and 3032 for pre-weaning mortality (PWM). For doe reproductive performance traits there were 2920 records available for litter size at birth (LSB), litter size at weaning (LSW), total litter weight at birth (TLWB) and litter mean weight per kid born (LMWKB), and 2182 for total litter weight at weaned (TLWW) and litter mean weight per kid weaned (LMWKW). Genetic parameters were estimated with univariate and bivariate models using restricted maximum likelihood (REML) procedures. Random effects were explored by fitting additive direct genetic effects, maternal additive genetic effects, maternal permanent environmental effects, the covariance between direct and maternal genetic effects, and common litter effects in different models for pre-weaning traits of kids. Also, in addition to an animal model, sire and threshold models, using a logit link function, were used for analyses of PWM. Models for LSB, LSW, TLWB, TLWW, LMWKB, and LMWKW included direct additive genetic effects, permanent environmental effects due to the animal as well as service sire effects. Estimated direct heritabilities were moderate for pre-weaning traits (0.22 for BW, 0.16 for WW, 0.21 for ADG, and 0.27 for KR and 0.29 for PWM), and low for reproduction traits (0.01 for LSB, 0.01 for LSW, 0.02 for TLWB, 0.03 for TLWW, 0.07 for LMWKB, and 0.06 for LMWKW). The estimates for the maternal additive genetic variance ratios were lower than direct heritability for BW (0.07) and KR (0.04). The estimate for the maternal permanent environmental variance ratios (c²) varied from 0.01 for KR to 0.07 for WW and ADG. The magnitude of common litter variance ratios (l²) was more substantial for BW (0.46) than the PWM (0.19) and KR (0.16). The estimate for the permanent environmental variance due to the animal (c²) ranged from 0.03 for LMWKB to 0.07 for TLWB and LMWKW, whereas service sire effects (s²) ranged from 0.02 to 0.04. The correlation between direct and maternal genetic effects were negative and high for BW (?0.51) and KR (?0.62). The genetic correlations between pre-weaning growth traits were positive and moderate to strong, as were genetic correlations between reproductive traits. Between BW and PWM the correlation was ?0.35. Phenotypic and environmental correlations for all traits were generally lower than genetic correlations.     

Analysis of variance of parameter estimates: F tests and t tests.

The problem of comparing and pooling experimentally independent estimates of a parameter such as a Michaelis constant (K) has been treated as a simple analysis of variance of "within" and "between" set deviations from the fitted variable (v). As applied to assessing the reproducibility of multiple estimates of the same K, this is identical to the procedure of Duggleby (Anal. Biochem. 189, 84-87, 1990). However, the theory developed here shows that applying Duggleby's procedure to the comparison of two experiments (each consisting of multiple data sets) depends critically on the assumption of equal errors within and between the individual sets, i.e., Fvb vw = s2wv/s2bv is close to 1. Application of the method when this is not the case will underestimate the common error (s2rv), overestimate its associated degrees of freedom (vr = vb+vw), and may suggest apparently significant differences where there are none. The theory also shows that this situation is an instance of the Fisher-Behrens problem and shows how Welch's solution can be applied. This gives the between set error s2bv as the corrected estimate of the common error and the corrected degrees of freedom as a simple function of vb, vw, and Fvb vw. When the nine prephenate dehydratase data sets which originally showed three apparently significant differences were reanalyzed in this way, all the variations in K were found to be within the range of the experimental error.     

The distribution of genetic parameter estimates and confidence intervals from small disconnected diallels

The distributions of genetic variance components and their ratios (heritability and type-B genetic correlation) from 105 pairs of six-parent disconnected half-diallels of a breeding population of loblolly pine (Pinus taeda L.) were examined. A series of simulations based on these estimates were carried out to study the coverage accuracy of confidence intervals based on the usual t-method and several other alternative methods. Genetic variance estimates fluctuated greatly from one experiment to another. Both general combining ability variance (²_g) and specific combining ability variance (²_s) had a large positive skewness. For ²_g and ²_s, a skewness-adjusted t-method proposed by Boos and Hughes-Oliver (Am Stat 54:121–128, 2000) provided better upper endpoint confidence intervals than t-intervals, whereas they were similar for the lower endpoint. Bootstrap BCa-intervals (Efron and Tibshirani, An introduction to the bootstrap. Chapman & Hall, London 436 p, 1993) and Halls transformation methods (Zhou and Gao, Am Stat 54:100–104, 2000) had poor coverages. Coverage accuracy of Fiellers interval endpoint(J R Stat Soc Ser B 16:175–185, 1954) and t-interval endpoint were similar for both h² and r_B for sample sizes n10, but for n=30 the Fiellers method is much better.     

HIV model parameter estimates from interruption trial data including drug efficacy and reservoir dynamics

Mathematical models based on ordinary differential equations (ODE) have had significant impact on understanding HIV disease dynamics and optimizing patient treatment. A model that characterizes the essential disease dynamics can be used for prediction only if the model parameters are identifiable from clinical data. Most previous parameter identification studies for HIV have used sparsely sampled data from the decay phase following the introduction of therapy. In this paper, model parameters are identified from frequently sampled viral-load data taken from ten patients enrolled in the previously published AutoVac HAART interruption study, providing between 69 and 114 viral load measurements from 3-5 phases of viral decay and rebound for each patient. This dataset is considerably larger than those used in previously published parameter estimation studies. Furthermore, the measurements come from two separate experimental conditions, which allows for the direct estimation of drug efficacy and reservoir contribution rates, two parameters that cannot be identified from decay-phase data alone. A Markov-Chain Monte-Carlo method is used to estimate the model parameter values, with initial estimates obtained using nonlinear least-squares methods. The posterior distributions of the parameter estimates are reported and compared for all patients.     

Genetic and phenotypic parameter estimates of pelvic measurements and birth weight in beef heifers

Data on 1210 spring-born (1983 to 1988) yearling heifers were analyzed by paternal half-sib procedures to obtain genetic and phenotypic parameter estimates involving birth weight and pelvic measurements. Data included records on 629 Angus, 325 Simmental, and 256 Salers representing 93, 49, and 22 paternal half-sib sire groups, respectively. Heritabilities for birth weight (BW), pelvic height (PH), pelvic width (PW), and pelvic area (PA) for Angus were 0.30, 0.61, 0.28, and 0.43, respectively. Corresponding values for Simmental and Salers heifers were 0.14, 0.34, 0.44, 0.37, and 0.18, 0.02, 0.29, 0.15, respectively. Genetic correlations among pelvic measurements (PH-PW, PH-PA, PW-PA) were positive (0.25 to 1.03) except for the estimate of -0.07 for PH-PW in Simmentals. Genetic correlations between BW and the 3 pelvic measurements (BW-PH, BW-PW, BW-PA) were negative (-0.18 to -0.36) except for the estimates of 0.53 (BW-PW) and 0.26 (BW-PA) in Simmentals and 2.84 (BW-PH) and 0.39 (BW-PA) in Salers. Phenotypic correlations among pelvic measurements ranged from 0.16 to 0.80. Phenotypic correlations between birth weight and the 3 pelvic measurements were consistently lower (-0.02 to 0.09) than the genetic correlations.     

Effect of lactation length adjustment procedures on genetic parameter estimates for buffalo milk yield

The objectives of this study were to estimate the genetic parameters for milk yield unadjusted and adjusted for days in milk and, subsequently, to assess the influence of adjusting for days in milk on sire rank. Complete lactations from 90 or 150 days of lactation to 270 or 350 days in milk were considered in these analyses. Milk yield was adjusted for days in milk by multiplicative correction factors, or by including lactation length as a covariable in the model. Milk yields adjusted by different procedures were considered as different traits. Heritability estimates varied from 0.17 to 0.28. Genetic correlation estimates between milk yields unadjusted and adjusted for days in milk were greater than 0.82. Adjusting for days in milk affected the parameter estimates. Multiplicative correction factors produced the highest heritability estimates. More reliable breeding value estimates can be expected by including short length lactation records in the analyses and adjusting the milk yields for days in milk, regardless of the method used for the adjustment. High selection intensity coupled to the inclusion of short length lactations and adjustment with multiplicative factors can change the sire rank..     

Genetic parameter estimates and response to selection for weight and testicular traits in Nelore cattle

We estimated genetic parameters for various phases of body and testicular growth until 550 days of age in Nelore cattle, using Bayesian inference, including correlation values and error estimates. Weight and scrotal records of 54,182 Nelore animals originating from 18 farms participating in the Brazilian Nelore Breeding Program (PMGRN) were included. The following traits were measured: weight at standard ages of 120 (W120), 210 (W210), 365 (W365), 450 (W450), and 550 (W550) days; weight gain between 120/210 (WG1), 210/365 (WG2), 365/450 (WG3), 450/550 (WG4), 120/365 (WG5), 120/450 (WG6), 120/550 (WG7), 210/450 (WG8), 210/550 (WG9), and 365/550 (WG10) days of age; scrotal circumference at 365 (SC365), 450 (SC450) and 550 (SC550) days of age, and testicular growth between 365/450 (TG1), 450/550 (TG2) and 365/550 (TG3) days of age. The model included contemporary group (current farm, year and two-month period of birth, sex, and management group) and age of dam at calving, divided into classes as fixed effects. The model also included random effects for direct additive, maternal additive and maternal permanent environmental, and residual effects. The direct heritability estimates ranged from 0.23 to 0.39, 0.13 to 0.39 and 0.32 to 0.56 for weights at standard ages, weight gains and testicular measures, respectively. The genetic correlations between weights (0.69 to 0.94) and scrotal circumferences (0.91 to 0.97) measured at standard ages were higher than those between weight gain and testicular growth (0.18 to 0.97 and 0.36 to 0.77, respectively). The weights at standard ages responded more effectively to selection, and also gave strong correlations with the other traits.     

Application of jackknife procedures to inter-experiment comparisons of parameter estimates for the Michaelis-Menten equation.

The jackknife procedure is introduced as a means of making comparisons among Michaelis-Menten parameter estimates for six different experimental conditions. In addition to providing a solution to the general inter-experimental comparison problem, the jackknife procedure will provide valid parameter estimates even when some of the assumptions usually required for statistical analysis are violated, e.g., the random errors are not normally distributed and the variances are not homogeneous. Other recent variations of the jackknife have also been introduced and briefly investigated: (i) the linear jackknife, which is more efficient computationally, and (ii) the weighted jackknife, which reduces the influence of design points (substrate concentrations) that have an excessive influence on the precision of parameter estimates.