Determining the number of clusters using the weighted gap statistic

Estimating the number of clusters in a data set is a crucial step in cluster analysis. In this article, motivated by the gap method (Tibshirani, Walther, and Hastie, 2001, Journal of the Royal Statistical Society B63, 411-423), we propose the weighted gap and the difference of difference-weighted (DD-weighted) gap methods for estimating the number of clusters in data using the weighted within-clusters sum of errors: a measure of the within-clusters homogeneity. In addition, we propose a "multilayer" clustering approach, which is shown to be more accurate than the original gap method, particularly in detecting the nested cluster structure of the data. The methods are applicable when the input data contain continuous measurements and can be used with any clustering method. Simulation studies and real data are investigated and compared among these proposed methods as well as with the original gap method.     

Testing for differences in survival with delayed ascertainment

During the interim stages of most large-scale clinical trials, knowledge that a patient is alive or dead is usually not up-to-date. This is due to the pattern of patient visits to hospitals as well as the administrative set-up used by the study to obtain information on vital status. On a two-armed study, if the process of ascertaining vital status is not the same in both treatment groups, then the standard method of testing based on the logrank statistic may not be applicable. Instead, an ad hoc modification to the logrank test, which artificially truncates follow-up prior to the time of analysis, is often used. These approaches have not been formally addressed in the literature. In the early stages of a clinical trial, severe bias or loss of power may result. For this situation, we propose a class of test statistics that extends the usual class of U statistics. Asymptotic normality is derived by reformulating the statistics in terms of counting processes and employing the theory of U statistics along with martingale techniques. For early interim analyses, a numerical study indicates that the new tests can be more powerful than the current practice when differential ascertainment is present. To illustrate the potential loss of information when lagging follow-up to control for ascertainment delays, we reanalyze an AIDS clinical trial with the truncated logrank and the new statistics.     

Measuring Pairwise Agreement Among Many Observers. II. Some Improvements and Additions

Weighted kappa was defined as a measure of pairwise interobserver agreement for the case where the observers judging one subject are not necessarily the same as those judging another subject. In this paper improved formulas for the large sample variance of the weighted kappa statistic are derived, a new definition of interclass kappa coefficients is suggested, and the intraclass correlation coefficient is shown to be a special case of weighted kappa.     

A Note on Binary Contingency Tables

This paper considers contingency tables in which the marginal frequencies for one variable are all 1. This could occur with two-category binary data or when a continuous variable is treated in categorical fashion. Some results concerning the expectation of goodness-of-fit statistics are reported. In particular it is noted that the expectation of the Pearson statistic is independent of the model being fitted.     

Testing equality of survival functions based on both paired and unpaired censored data

We introduce two test procedures for comparing two survival distributions on the basis of randomly right-censored data consisting of both paired and unpaired observations. Our procedures are based on generalizations of a pooled rank test statistic previously proposed for uncensored data. One generalization adapts the Prentice-Wilcoxon score, while the other adapts the Akritas score. The use of these particular scoring systems in pooled rank tests with randomly right-censored paired data has been advocated by several researchers. Our test procedures utilize the permutation distributions of the test statistics based on a novel manner of permuting the scores. Permutation versions of tests for right-censored paired data and for two independent right-censored samples that use the proposed scoring systems are obtained as special cases of our test procedures. Simulation results show that our test procedures have high power for detecting scale and location shifts in exponential and log-logistic distributions for the survival times. We also demonstrate the advantages of our test procedures in terms of utilizing randomly occurring unpaired observations that are discarded in test procedures for paired data. The tests are applied to skin graft data previously reported elsewhere.     

Testing against umbrella or tree orderings for binomial proportions with an adaptation of an insect resistance case

Alternative hypotheses for order restrictions, such as umbrella or inverse umbrella (a.k.a tree) orderings, have been studied extensively in the literature, although less so when the studied response for each individual is the presence or absence of the event of interest. Two families of test statistics for solving the problem of testing against an umbrella or a tree ordering when the responses are binomial proportions are studied in this work and their asymptotic distributions are derived. A simulation study is conducted to compare the empirical power of some members of the derived families of test statistics with competing approaches. The methodology developed here was driven by an applied problem arising in stored products research where despite universal mortality in the case of doses of 1000 ppm of the insecticide phosphine, unexpected survival was noted at higher doses.