Goodness of Fit Tests for Proportional Hazards Regression Models

We present a test of goodness of fit for the proportional hazard regression model. The test is based on a score statistic for testing against local mixture alternatives. Contrary to the findings of several other authors, we detect a significant lack of fit in Freireich's leukemia data.     

Testing Differentiation in Diploid Populations

We examine the power of different exact tests of differentiation for diploid populations. Since there is not necessarily random mating within populations, the appropriate hypothesis to construct exact tests is that of independent sampling of genotypes. There are two categories of tests, F(ST)-estimator tests and goodness of fit tests. In this latter category, we distinguish ``allelic statistics', which account for the nature of alleles within genotypes, from ``genotypic statistics' that do not. We show that the power of F(ST)-estimator tests and of allelic goodness of fit tests are similar when sampling is balanced, and higher than the power of genotypic goodness of fit tests. When sampling is unbalanced, the most powerful tests are shown to belong to the allelic goodness of fit group.     

Assessing the goodness of fit of logistic regression models in large samples: A modification of the Hosmer-Lemeshow test

Evaluating the goodness of fit of logistic regression models is crucial to ensure the accuracy of the estimated probabilities. Unfortunately, such evaluation is problematic in large samples. Because the power of traditional goodness of fit tests increases with the sample size, practically irrelevant discrepancies between estimated and true probabilities are increasingly likely to cause the rejection of the hypothesis of perfect fit in larger and larger samples. This phenomenon has been widely documented for popular goodness of fit tests, such as the Hosmer-Lemeshow test. To address this limitation, we propose a modification of the Hosmer-Lemeshow approach. By standardizing the noncentrality parameter that characterizes the alternative distribution of the Hosmer-Lemeshow statistic, we introduce a parameter that measures the goodness of fit of a model but does not depend on the sample size. We provide the methodology to estimate this parameter and construct confidence intervals for it. Finally, we propose a formal statistical test to rigorously assess whether the fit of a model, albeit not perfect, is acceptable for practical purposes. The proposed method is compared in a simulation study with a competing modification of the Hosmer-Lemeshow test, based on repeated subsampling. We provide a step-by-step illustration of our method using a model for postneonatal mortality developed in a large cohort of more than 300 000 observations.     

An Improved Goodness of Fit Statistic for Probability Prediction Models

We consider the general case of probability prediction models having two or more outcomes and propose an adjusted χ² statistic which can be used to assess the goodness of fit of these models. We present a simulation study to show that our proposed statistic has an approximate χ² distribution under the null hypothesis. Two applications are provided to illustrate the use of the new statistic. The first application examines the fit of a logistic regression model using both the proposed statistic and the popular Hosmer-Lemeshow statistic and we compare and contrast these two methods. The second application evaluates the goodness of fit of a polychotomous regression model.     

A note on goodness of fit of a population to Hardy-Weinberg structure.

The test of goodness of fit of a population to Hardy-Weinberg equilibrium given by Levene [2] and Haldane [1] is valid within a widely accepted recipe for testing goodness of fit of a composite hypothesis. The nature of the result of Cannings and Edwards [3] is described. The result was shown to be quite different than they claimed and, although possible of some interest, not relevant to the testing of goodness of fit to Hardy-Weinberg structure.     

Modified GEE and Goodness of the Marginal Fit (GOMF) Test with Correlated Binary Responses for Contingency Tables

This paper addresses testing the goodness of fit of models for marginal probabilities estimated by generalized estimating equations. We develop a modified version of generalized estimating equation and a goodness‐of‐fit test based on the fitted marginal means. The test statistic is easy to compute and has a simple reference distribution. Its performance is evaluated asymptotically and in small samples. It is also compared to the deviance and Pearson X² statistics. Example applications are given. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Model averaging in the analysis of leukemia mortality among Japanese A-bomb survivors

Epidemiological studies often include numerous covariates, with a variety of possible approaches to control for confounding of the association of primary interest, as well as a variety of possible models for the exposure–response association of interest. Walsh and Kaiser (Radiat Environ Biophys 50:21–35, 2011) advocate a weighted averaging of the models, where the weights are a function of overall model goodness of fit and degrees of freedom. They apply this method to analyses of radiation–leukemia mortality associations among Japanese A-bomb survivors. We caution against such an approach, noting that the proposed model averaging approach prioritizes the inclusion of covariates that are strong predictors of the outcome, but which may be irrelevant as confounders of the association of interest, and penalizes adjustment for covariates that are confounders of the association of interest, but may contribute little to overall model goodness of fit. We offer a simple illustration of how this approach can lead to biased results. The proposed model averaging approach may also be suboptimal as way to handle competing model forms for an exposure–response association of interest, given adjustment for the same set of confounders; alternative approaches, such as hierarchical regression, may provide a more useful way to stabilize risk estimates in this setting.     

S-Sample Smooth Goodness of Fit Testing: Rederivation and Monte Carlo Assessment

S-sample smooth goodness of fit tests may be constructed using components from one sample goodness of fit testing. Each sample could be assessed for consistency with a target distribution using these components, although that is not our objective here. Contrasts in the components may be used to assess consistency of the samples with each other. If all the samples are consistent, we could then conveniently perform a one-sample goodness of fit test for the target distribution. If the samples are not consistent, an LSD-type analysis can be performed on the one-sample components to identify where the differences between occur. This approach gives a detailed and informative scrutiny of the data.     

Measuring size distribution in highly heterogeneous systems with fluorescence correlation spectroscopy

Fluorescence correlation spectroscopy (FCS) is a sensitive and widely used technique for measuring diffusion. FCS data are conventionally modeled with a finite number of diffusing components and fit with a least-square fitting algorithm. This approach is inadequate for analyzing data obtained from highly heterogeneous systems. We introduce a Maximum Entropy Method based fitting routine (MEMFCS) that analyzes FCS data in terms of a quasicontinuous distribution of diffusing components, and also guarantees a maximally wide distribution that is consistent with the data. We verify that for a homogeneous specimen (green fluorescent protein in dilute aqueous solution), both MEMFCS and conventional fitting yield similar results. Further, we incorporate an appropriate goodness of fit criterion in MEMFCS. We show that for errors estimated from a large number of repeated measurements, the reduced chi(2) value in MEMFCS analysis does approach unity. We find that the theoretical prediction for errors in FCS experiments overestimates the actual error, but can be empirically modified to serve as a guide for estimating the goodness of the fit where reliable error estimates are unavailable. Finally, we compare the performance of MEMFCS with that of a conventional fitting routine for analyzing simulated data describing a highly heterogeneous distribution containing 41 diffusing species. Both methods fit the data well. However, the conventional fit fails to reproduce the essential features of the input distribution, whereas MEMFCS yields a distribution close to the actual input.     

Time series analysis of hybrid neurophysiological data and application of mutual information

Multivariate time series data of which some components are continuous time series and the rest are point processes are called hybrid data. Such data sets routinely arise while working with neuroscience data, EEG and spike trains would perhaps be the most obvious example. In this paper, we discuss the modeling of a hybrid time series, with the continuous component being the physiological tremors in the distal phalanx of the middle finger, and motor unit firings in the middle finger portion of the extensor digitorum communis (EDC) muscle. We employ a model for the two components based on Auto-regressive Moving Average (ARMA) type models. Another major issue to arise in the modeling of such data is to assess the goodness of fit. We suggest a visual procedure based on mutual information towards assessing the dependence pattern of hybrid data. The goodness of fit is also verified by standard model fitting diagnostic techniques for univariate data.     

Testing goodness of fit for stochastic models of carcinogenesis.

This paper considers the utility of statistical goodness of fit testing in the context of mechanistic models of carcinogenesis. Two stochastic models of carcinogenesis were tested with several sets of experimental and epidemiological data using a formal goodness of fit test specially designed to accommodate censored observations: these were the two-stage model allowing for clonal expansion of initiated cells and its simpler version with gamma distributed promotion time. The results of this application, supplemented by visual examination of local likelihood kernel estimates of the hazard function and the corresponding model-based estimates, show that mechanistic models of carcinogenesis provide a good fit to the data in the majority of cases under study.     

Goodness of Fit Tests with Misclassified Data Based on φ‐Divergences

The well known χ² goodness of fit test for a multinomial distribution is generally biased when observations are subject to misclassification. In this paper, based on a double sampling scheme, the family of φ‐divergence test statistics is introduced for testing goodness of fit under misclassification of the data. The case of binomial data is discussed and an illustrative example is also given.     

Fit and Frailties in Proportional Hazards Regression

We exploit a conjectured equivalence between proportional hazards models with frailties and a particular subclass of non proportional hazards models, specifically those with declining effects, to address the question of fit. A goodness of fit test of the proportional hazards assumption against an alternative of declining regression effect is equivalent to a test for the presence of frailties. Such tests are now widely available in standard software. Although a number of tests of the proportional hazards assumption have been developed there is no test that directly formulates the alternative in terms of a non‐specified monotonic decline in regression effect and that enables a quantification of this in terms of a simple index. The index we obtain lies between zero and one such that, for any given set of covariates, values of the index close to one indicate that the fit cannot essentially be improved by allowing the possibility of regression effects to decline. Values closer to zero and away from one indicate that the fit can be improved by relaxing the proportional hazards constraint in this particular direction. (© 2004 WILEY‐VCH Verlag GmbH & Co. KGaA, Weinheim)     

Estimation of the Spirometric Residual Volume (RV) by a Regression Built from Gower Distances

We estimate the Residual Volume, a spirometric parameter, by use of four continuous and four categorical variables. The estimation is done using distance-based regression, which allows to construct the predicting regression equation from mixed-type explanatory variables. The additionally introduced categorical variables improve essentially the goodness of fit of the regression equation.     

Sound representation methods for spectro-temporal receptive field estimation

The spectro-temporal receptive field (STRF) of an auditory neuron describes the linear relationship between the sound stimulus in a time-frequency representation and the neural response. Time-frequency representations of a sound in turn require a nonlinear operation on the sound pressure waveform and many different forms for this non-linear transformation are possible. Here, we systematically investigated the effects of four factors in the non-linear step in the STRF model: the choice of logarithmic or linear filter frequency spacing, the time-frequency scale, stimulus amplitude compression and adaptive gain control. We quantified the goodness of fit of these different STRF models on data obtained from auditory neurons in the songbird midbrain and forebrain. We found that adaptive gain control and the correct stimulus amplitude compression scheme are paramount to correctly modelling neurons. The time-frequency scale and frequency spacing also affected the goodness of fit of the model but to a lesser extent and the optimal values were stimulus dependant. Action Editor: Israel Nelken     

种子植物区系属大小与种多度分布格局的相似性

对属大小的分析是植物区系研究的重要组成,属大小的分布格局可以在一定程度上反映植物区系的类群多样性与系统发生多样性。在植物区系研究中,平均属大小指标与植物区系包含的物种数有关,使不同植物区系的研究结果难以进行整合分析。对中国8个地区种子植物区系属大小的分布格局进行研究,对数级数模型取得了良好的拟合结果,拟合优度R²＞0.99,属大小表现出与种—多度相似的分布格局。稀疏分析和相关分析结果表明：抽样大小较小时,平均属大小在稀疏分析前后的相关性不显著;而对数级数模型中α指数与稀疏分析前表现出稳定的相关性。同时,对于相同的抽样大小α指数与平均属大小之间存在显著的相关性。因此,α指数可以作为稳定的指标来描述植物区系的丰富程度。     

Goodness of Fit of Probability Distributions for Sightings as Species Approach Extinction

Estimating the probability that a species is extinct and the timing of extinctions is useful in biological fields ranging from paleoecology to conservation biology. Various statistical methods have been introduced to infer the time of extinction and extinction probability from a series of individual sightings. There is little evidence, however, as to which of these models provide adequate fit to actual sighting records. We use L-moment diagrams and probability plot correlation coefficient (PPCC) hypothesis tests to evaluate the goodness of fit of various probabilistic models to sighting data collected for a set of North American and Hawaiian bird populations that have either gone extinct, or are suspected of having gone extinct, during the past 150 years. For our data, the uniform, truncated exponential, and generalized Pareto models performed moderately well, but the Weibull model performed poorly. Of the acceptable models, the uniform distribution performed best based on PPCC goodness of fit comparisons and sequential Bonferroni-type tests. Further analyses using field significance tests suggest that although the uniform distribution is the best of those considered, additional work remains to evaluate the truncated exponential model more fully. The methods we present here provide a framework for evaluating subsequent models.     

Testing goodness of fit of a uniform truncation model

Several goodness-of-fit tests of a lifetime distribution have been suggested in the literature; many take into account censoring and/or truncation of event times. In some contexts, a goodness-of-fit test for the truncation distribution is of interest. In particular, better estimates of the lifetime distribution can be obtained when knowledge of the truncation law is exploited. In cross-sectional sampling, for example, there are theoretical justifications for the assumption of a uniform truncation distribution, and several studies have used it to improve the efficiency of their survival estimates. The duality of lifetime and truncation in the absence of censoring enables methods for testing goodness of fit of the lifetime distribution to be used for testing goodness of fit of the truncation distribution. However, under random censoring, this duality does not hold and different tests are required. In this article, we introduce several goodness-of-fit tests for the truncation distribution and investigate their performance in the presence of censored event times using simulation. We demonstrate the use of our tests on two data sets.     

Parameter estimation and goodness-of-fit in log binomial regression

An estimate of the risk, adjusted for confounders, can be obtained from a fitted logistic regression model, but it substantially over-estimates when the outcome is not rare. The log binomial model, binomial errors and log link, is increasingly being used for this purpose. However this model's performance, goodness of fit tests and case-wise diagnostics have not been studied. Extensive simulations are used to compare the performance of the log binomial, a logistic regression based method proposed by Schouten et al. (1993) and a Poisson regression approach proposed by Zou (2004) and Carter, Lipsitz, and Tilley (2005). Log binomial regression resulted in "failure" rates (non-convergence, out-of-bounds predicted probabilities) as high as 59%. Estimates by the method of Schouten et al. (1993) produced fitted log binomial probabilities greater than unity in up to 19% of samples to which a log binomial model had been successfully fit and in up to 78% of samples when the log binomial model fit failed. Similar percentages were observed for the Poisson regression approach. Coefficient and standard error estimates from the three models were similar. Rejection rates for goodness of fit tests for log binomial fit were around 5%. Power of goodness of fit tests was modest when an incorrect logistic regression model was fit. Examples demonstrate the use of the methods. Uncritical use of the log binomial regression model is not recommended.