Confidence intervals for uncommon but dramatic responses to treatment

A small literature discusses locally most powerful rank tests when only a fraction of treated subjects respond to treatment. The ranks used in these tests are very different from conventional ranks, being relatively flat for low responses and then rising steeply, and the associated tests are much more powerful than conventional rank tests when, indeed, only a small fraction of treated subjects exhibit dramatic responses. Because the tests are derived from considerations of local power, they do not yield a plausible family of models for effect, and therefore they do not yield confidence intervals for the magnitude of effect formed by inverting the tests. There is a similarity between these tests and another family of tests, originally motivated by different considerations involving peak performance in small subsets. Exploiting this similarity, a method for obtaining confidence statements is proposed. In the case of observational studies, sensitivity to unobserved bias from nonrandom assignment of treatments is also examined. Two examples are used as illustrations: (i) a study of smoking during pregnancy and its effects on birth weight, in which smokers are matched to six controls, and (ii) a matched pair study of damage to DNA among workers at aluminum production plants.