A New Extension of the Binomial Error Model for Responses to Items of Varying Difficulty in Educational Testing and Attitude Surveys

We put forward a new item response model which is an extension of the binomial error model first introduced by Keats and Lord. Like the binomial error model, the basic latent variable can be interpreted as a probability of responding in a certain way to an arbitrarily specified item. For a set of dichotomous items, this model gives predictions that are similar to other single parameter IRT models (such as the Rasch model) but has certain advantages in more complex cases. The first is that in specifying a flexible two-parameter Beta distribution for the latent variable, it is easy to formulate models for randomized experiments in which there is no reason to believe that either the latent variable or its distribution vary over randomly composed experimental groups. Second, the elementary response function is such that extensions to more complex cases (e.g., polychotomous responses, unfolding scales) are straightforward. Third, the probability metric of the latent trait allows tractable extensions to cover a wide variety of stochastic response processes.     

Biases and Power for Groups Comparison on Subjective Health Measurements

Subjective health measurements are increasingly used in clinical research, particularly for patient groups comparisons. Two main types of analytical strategies can be used for such data: so-called classical test theory (CTT), relying on observed scores and models coming from Item Response Theory (IRT) relying on a response model relating the items responses to a latent parameter, often called latent trait. Whether IRT or CTT would be the most appropriate method to compare two independent groups of patients on a patient reported outcomes measurement remains unknown and was investigated using simulations. For CTT-based analyses, groups comparison was performed using t-test on the scores. For IRT-based analyses, several methods were compared, according to whether the Rasch model was considered with random effects or with fixed effects, and the group effect was included as a covariate or not. Individual latent traits values were estimated using either a deterministic method or by stochastic approaches. Latent traits were then compared with a t-test. Finally, a two-steps method was performed to compare the latent trait distributions, and a Wald test was performed to test the group effect in the Rasch model including group covariates. The only unbiased IRT-based method was the group covariate Wald’s test, performed on the random effects Rasch model. This model displayed the highest observed power, which was similar to the power using the score t-test. These results need to be extended to the case frequently encountered in practice where data are missing and possibly informative.     

Cooperation driven by mutations in multi-person Prisoner's Dilemma

The n-person Prisoner's Dilemma is a widely used model for populations where individuals interact in groups. The evolutionary stability of populations has been analysed in the literature for the case where mutations in the population may be considered as isolated events. For this case, and assuming simple trigger strategies and many iterations per game, we analyse the rate of convergence to the evolutionarily stable populations. We find that for some values of the payoff parameters of the Prisoner's Dilemma this rate is so low that the assumption, that mutations in the population are infrequent on that time-scale, is unreasonable. Furthermore, the problem is compounded as the group size is increased. In order to address this issue, we derive a deterministic approximation of the evolutionary dynamics with explicit, stochastic mutation processes, valid when the population size is large. We then analyse how the evolutionary dynamics depends on the following factors: mutation rate, group size, the value of the payoff parameters, and the structure of the initial population. In order to carry out the simulations for groups of more than just a few individuals, we derive an efficient way of calculating the fitness values. We find that when the mutation rate per individual and generation is very low, the dynamics is characterized by populations which are evolutionarily stable. As the mutation rate is increased, other fixed points with a higher degree of cooperation become stable. For some values of the payoff parameters, the system is characterized by (apparently) stable limit cycles dominated by cooperative behaviour. The parameter regions corresponding to high degree of cooperation grow in size with the mutation rate, and in number with the group size. For some parameter values, we find more than one stable fixed point, corresponding to different structures of the initial population.     

Linear Logistic Latent Class Analysis

In the present paper the linear logistic extension of latent class analysis is described. Thereby it is assumed that the item latent probabilities as well as the class sizes can be attributed to some explanatory variables. The basic equations of the model state the decomposition of the log-odds of the item latent probabilities and of the class sizes into weighted sums of basic parameters representing the effects of the predictor variables. Further, the maximum likelihood equations for these effect parameters and statistical tests for goodness-of-fit are given. Finally, an example illustrates the practical application of the model and the interpretation of the model parameters.     

Joint inference of the distribution of fitness effects of deleterious mutations and population demography based on nucleotide polymorphism frequencies

The distribution of fitness effects of new mutations (DFE) is important for addressing several questions in genetics, including the nature of quantitative variation and the evolutionary fate of small populations. Properties of the DFE can be inferred by comparing the distributions of the frequencies of segregating nucleotide polymorphisms at selected and neutral sites in a population sample, but demographic changes alter the spectrum of allele frequencies at both neutral and selected sites, so can bias estimates of the DFE if not accounted for. We have developed a maximum-likelihood approach, based on the expected allele-frequency distribution generated by transition matrix methods, to estimate parameters of the DFE while simultaneously estimating parameters of a demographic model that allows a population size change at some time in the past. We tested the method using simulations and found that it accurately recovers simulated parameter values, even if the simulated demography differs substantially from that assumed in our analysis. We use our method to estimate parameters of the DFE for amino acid-changing mutations in humans and Drosophila melanogaster. For a model of unconditionally deleterious mutations, with effects sampled from a gamma distribution, the mean estimate for the distribution shape parameter is approximately 0.2 for human populations, which implies that the DFE is strongly leptokurtic. For Drosophila populations, we estimate that the shape parameter is approximately 0.35. Differences in the shape of the distribution and the mean selection coefficient between humans and Drosophila result in significantly more strongly deleterious mutations in Drosophila than in humans, and, conversely, nearly neutral mutations are significantly less frequent.     

Selection of the Better of Two Treatments when They May Not be Equivalent

A procedure for selecting the better of two treatments which allows for the possibility of non-selection when the treatments appear to be equivalent (that is, similar) is presented. The proposed procedure is a modification of the indifference zone approach. It is assumed that the treatments are compared with respect to a continuous response variable, which has a normal or a two-parameter exponential distribution. For the normal distribution, each of the parameters is considered as the ranking parameter. For the two-parameter exponential distribution, the guarantee time (location parameter) is the ranking parameter. The values of the estimates of the ranking parameters and the observed distance between these estimates are used in this selection procedure.     

Performance of a Receptive Language Test among Young Children in Madagascar

Language tests developed and validated in one country may lose their desired properties when translated for use in another, possibly resulting in misleading estimates of ability. Using Item Response Theory (IRT) methodology, we assess the performance of a test of receptive vocabulary, the U.S.-validated Peabody Picture Vocabulary Test-Third Edition (PPVT-III), when translated, adapted, and administered to children 3 to 10 years of age in Madagascar (N = 1372), in the local language (Malagasy). Though Malagasy is considered a single language, there are numerous dialects spoken in Madagascar. Our findings were that test scores were positively correlated with age and indicators of socio-economic status. However, over half (57/96) of items evidenced unexpected response variation and/or bias by local dialect spoken. We also encountered measurement error and reduced differentiation among person abilities when we used the publishers’ recommended stopping rules, largely because we lost the original item ordering by difficulty when we translated test items into Malagasy. Our results suggest that bias and testing inefficiency introduced from the translation of the PPVT can be significantly reduced with the use of methods based on IRT at both the pre-testing and analysis stages. We explore and discuss implications for cross-cultural comparisons of internationally recognized tests, such as the PPVT.     

A Probit Latent Class Model with General Correlation Structures for Evaluating Accuracy of Diagnostic Tests

Summary Traditional latent class modeling has been widely applied to assess the accuracy of dichotomous diagnostic tests. These models, however, assume that the tests are independent conditional on the true disease status, which is rarely valid in practice. Alternative models using probit analysis have been proposed to incorporate dependence among tests, but these models consider restricted correlation structures. In this article, we propose a probit latent class model that allows a general correlation structure. When combined with some helpful diagnostics, this model provides a more flexible framework from which to evaluate the correlation structure and model fit. Our model encompasses several other PLC models but uses a parameter‐expanded Monte Carlo EM algorithm to obtain the maximum‐likelihood estimates. The parameter‐expanded EM algorithm was designed to accelerate the convergence rate of the EM algorithm by expanding the complete‐data model to include a larger set of parameters and it ensures a simple solution in fitting the PLC model. We demonstrate our estimation and model selection methods using a simulation study and two published medical studies.     

Power and Sample Size Determination for the Group Comparison of Patient-Reported Outcomes with Rasch Family Models

Background

Patient-reported outcomes (PRO) that comprise all self-reported measures by the patient are important as endpoint in clinical trials and epidemiological studies. Models from the Item Response Theory (IRT) are increasingly used to analyze these particular outcomes that bring into play a latent variable as these outcomes cannot be directly observed. Preliminary developments have been proposed for sample size and power determination for the comparison of PRO in cross-sectional studies comparing two groups of patients when an IRT model is intended to be used for analysis. The objective of this work was to validate these developments in a large number of situations reflecting real-life studies.

Methodology

The method to determine the power relies on the characteristics of the latent trait and of the questionnaire (distribution of the items), the difference between the latent variable mean in each group and the variance of this difference estimated using Cramer-Rao bound. Different scenarios were considered to evaluate the impact of the characteristics of the questionnaire and of the variance of the latent trait on performances of the Cramer-Rao method. The power obtained using Cramer-Rao method was compared to simulations.

Principal Findings

Powers achieved with the Cramer-Rao method were close to powers obtained from simulations when the questionnaire was suitable for the studied population. Nevertheless, we have shown an underestimation of power with the Cramer-Rao method when the questionnaire was less suitable for the population. Besides, the Cramer-Rao method stays valid whatever the values of the variance of the latent trait.

Conclusions

The Cramer-Rao method is adequate to determine the power of a test of group effect at design stage for two-group comparison studies including patient-reported outcomes in health sciences. At the design stage, the questionnaire used to measure the intended PRO should be carefully chosen in relation to the studied population.     

Power and Sample Size Determination in the Rasch Model: Evaluation of the Robustness of a Numerical Method to Non-Normality of the Latent Trait

Patient-reported outcomes (PRO) have gained importance in clinical and epidemiological research and aim at assessing quality of life, anxiety or fatigue for instance. Item Response Theory (IRT) models are increasingly used to validate and analyse PRO. Such models relate observed variables to a latent variable (unobservable variable) which is commonly assumed to be normally distributed. A priori sample size determination is important to obtain adequately powered studies to determine clinically important changes in PRO. In previous developments, the Raschpower method has been proposed for the determination of the power of the test of group effect for the comparison of PRO in cross-sectional studies with an IRT model, the Rasch model. The objective of this work was to evaluate the robustness of this method (which assumes a normal distribution for the latent variable) to violations of distributional assumption. The statistical power of the test of group effect was estimated by the empirical rejection rate in data sets simulated using a non-normally distributed latent variable. It was compared to the power obtained with the Raschpower method. In both cases, the data were analyzed using a latent regression Rasch model including a binary covariate for group effect. For all situations, both methods gave comparable results whatever the deviations from the model assumptions. Given the results, the Raschpower method seems to be robust to the non-normality of the latent trait for determining the power of the test of group effect.     

Multiple comparisons distortions of parameter estimates

In experiments involving many variables, investigators typically use multiple comparisons procedures to determine differences that are unlikely to be the result of chance. However, investigators rarely consider how the magnitude of the greatest observed effect sizes may have been subject to bias resulting from multiple testing. These questions of bias become important to the extent investigators focus on the magnitude of the observed effects. As an example, such bias can lead to problems in attempting to validate results, if a biased effect size is used to power a follow-up study. An associated important consequence is that confidence intervals constructed using standard distributions may be badly biased. A bootstrap approach is used to estimate and adjust for the bias in the effect sizes of those variables showing strongest differences. This bias is not always present; some principles showing what factors may lead to greater bias are given and a proof of the convergence of the bootstrap distribution is provided.     

Efficiency of indirect selection at selection equilibrium

Summary Efficiency of indirect selection compared with that of direct selection to increase the mean value of some trait has been usually studied by considering a single generation of indirect and direct responses to selection only. However, under continued selection, genetic variances and covariances, and therefore expected genetic responses, change each generation due to linkage disequilibrium. With directional and truncation selection, genetic parameters asymptote to limiting values after several generations. The efficiency of indirect selection is examined in this limiting situation. The ratio of correlated response to direct response for the trait to improve in the limit is compared with the ratio after the first generation of selection. For all initial parameter values for which indirect selection is more efficient than direct selection, relative efficiency of indirect selection is smaller in the limit than in the first generation. For some parameter values, indirect selection is more efficient than direct selection in the first generation, but less efficient in the limit. Expressions for minimum values of the initial genetic correlation and heritability of the alternative trait required for indirect selection to be preferred in the limit are derived. These values are higher when limiting responses are used instead of single generation responses. The loss in relative efficiency of indirect selection from changes in genetic parameters due to selection should be taken into account when applications of indirect selection are considered.     

Asymptotic behavior of the scaled mutation rate estimators

Important aspects of population evolution have been investigated using nucleotide sequences. Under the neutral Wright–Fisher model, the scaled mutation rate represents twice the average number of new mutations per generations and it is one of the key parameters in population genetics. In this study, we present various methods of estimation of this parameter, analytical studies of their asymptotic behavior as well as comparisons of the distribution's behavior of these estimators through simulations. As knowledge of the genealogy is needed to estimate the maximum likelihood estimator (MLE), an application with real data is also presented, using jackknife to correct the bias of the MLE, which can be generated by the estimation of the tree. We proved analytically that the Waterson's estimator and the MLE are asymptotically equivalent with the same rate of convergence to normality. Furthermore, we showed that the MLE has a better rate of convergence than Waterson's estimator for values of the parameter greater than one and this relationship is reversed when the parameter is less than one.     

Beyond the proportional frailty model: Bayesian estimation of individual heterogeneity on mortality parameters

Today, we know that demographic rates can be greatly influenced by differences among individuals in their capacity to survive and reproduce. These intrinsic differences, commonly known as individual heterogeneity, can rarely be measured and are thus treated as latent variables when modeling mortality. Finite mixture models and mixed effects models have been proposed as alternative approaches for inference on individual heterogeneity in mortality. However, in general models assume that individual heterogeneity influences mortality proportionally, which limits the possibility to test hypotheses on the effect of individual heterogeneity on other aspects of mortality such as ageing rates. Here, we propose a Bayesian model that builds upon the mixture models previously developed, but that facilitates making inferences on the effect of individual heterogeneity on mortality parameters other than the baseline mortality. As an illustration, we apply this framework to the Gompertz–Makeham mortality model, commonly used in human and wildlife studies, by assuming that the Gompertz rate parameter is affected by individual heterogeneity. We provide results of a simulation study where we show that the model appropriately retrieves the parameters used for simulation, even for low variances in the heterogeneous parameter. We then apply the model to a dataset on captive chimpanzees and on a cohort life table of 1751 Swedish men, and show how model selection against a null model (i.e., without heterogeneity) can be carried out.     

Penalized item response theory models: application to epigenetic alterations in bladder cancer

Increasingly used in health-related applications, latent variable models provide an appealing framework for handling high-dimensional exposure and response data. Item response theory (IRT) models, which have gained widespread popularity, were originally developed for use in the context of educational testing, where extremely large sample sizes permitted the estimation of a moderate-to-large number of parameters. In the context of public health applications, smaller sample sizes preclude large parameter spaces. Therefore, we propose a penalized likelihood approach to reduce mean square error and improve numerical stability. We present a continuous family of models, indexed by a tuning parameter, that range between the Rasch model and the IRT model. The tuning parameter is selected by cross validation or approximations such as Akaike Information Criterion. While our approach can be placed easily in a Bayesian context, we find that our frequentist approach is more computationally efficient. We demonstrate our methodology on a study of methylation silencing of gene expression in bladder tumors. We obtain similar results using both frequentist and Bayesian approaches, although the frequentist approach is less computationally demanding. In particular, we find high correlation of methylation silencing among 16 loci in bladder tumors, that methylation is associated with smoking and also with patient survival.     

Bias correction in the hierarchical likelihood approach to the analysis of multivariate survival data

Frailty models are useful for measuring unobserved heterogeneity in risk of failures across clusters, providing cluster-specific risk prediction. In a frailty model, the latent frailties shared by members within a cluster are assumed to act multiplicatively on the hazard function. In order to obtain parameter and frailty variate estimates, we consider the hierarchical likelihood (H-likelihood) approach (Ha, Lee and Song, 2001. Hierarchical-likelihood approach for frailty models. Biometrika 88, 233-243) in which the latent frailties are treated as "parameters" and estimated jointly with other parameters of interest. We find that the H-likelihood estimators perform well when the censoring rate is low, however, they are substantially biased when the censoring rate is moderate to high. In this paper, we propose a simple and easy-to-implement bias correction method for the H-likelihood estimators under a shared frailty model. We also extend the method to a multivariate frailty model, which incorporates complex dependence structure within clusters. We conduct an extensive simulation study and show that the proposed approach performs very well for censoring rates as high as 80%. We also illustrate the method with a breast cancer data set. Since the H-likelihood is the same as the penalized likelihood function, the proposed bias correction method is also applicable to the penalized likelihood estimators.     

A two-stage model for concurrent sequences

Schneider and Davison [Schneider, S.M., Davison, M., 2005. Demarcated response sequences and the generalised matching law. Behav. Proc. 70, 51–61] showed that the generalised matching law applied to concurrent free-operant two-response sequences. When sufficient temporal spacing is required between the responses, however, neither the response-level nor the sequence-level forms of the generalised matching law provide good fits. An alternative “two-stage sensitivity” model with fewer free parameters features two types of sensitivity to the reinforcement contingencies on sequences. When temporal spacing between the responses is long, the “response distribution sensitivity” parameter describes sensitivity only of the individual responses to the sequence-level contingencies. At a threshold level, this sensitivity reaches a maximum. When spacing is shorter than threshold, the “response order sensitivity” parameter reflects a new sensitivity to the order of the responses within sequences. As this sensitivity approaches its maximum, sequence matching is achieved. For both stages, a changeover parameter describes bias against sequences that require changeovers between the two responses. The model fit data ranging from near-response matching with long minimum inter-response times (IRTs) to sequence matching with no minimum IRTs, using two species and a variety of sequence reinforcer distributions. Rats differed from pigeons in achieving sequence matching only at a nonzero minimum IRT. In a comparison based on pigeon data with no minimum IRT, the two-stage sensitivity model was more efficient than the generalised matching law according to the Akaike criterion. The logic of the model suggests a new way of understanding the mechanisms underlying behavioural units.     

Searching for convergence in phylogenetic Markov chain Monte Carlo

Markov chain Monte Carlo (MCMC) is a methodology that is gaining widespread use in the phylogenetics community and is central to phylogenetic software packages such as MrBayes. An important issue for users of MCMC methods is how to select appropriate values for adjustable parameters such as the length of the Markov chain or chains, the sampling density, the proposal mechanism, and, if Metropolis-coupled MCMC is being used, the number of heated chains and their temperatures. Although some parameter settings have been examined in detail in the literature, others are frequently chosen with more regard to computational time or personal experience with other data sets. Such choices may lead to inadequate sampling of tree space or an inefficient use of computational resources. We performed a detailed study of convergence and mixing for 70 randomly selected, putatively orthologous protein sets with different sizes and taxonomic compositions. Replicated runs from multiple random starting points permit a more rigorous assessment of convergence, and we developed two novel statistics, delta and epsilon, for this purpose. Although likelihood values invariably stabilized quickly, adequate sampling of the posterior distribution of tree topologies took considerably longer. Our results suggest that multimodality is common for data sets with 30 or more taxa and that this results in slow convergence and mixing. However, we also found that the pragmatic approach of combining data from several short, replicated runs into a "metachain" to estimate bipartition posterior probabilities provided good approximations, and that such estimates were no worse in approximating a reference posterior distribution than those obtained using a single long run of the same length as the metachain. Precision appears to be best when heated Markov chains have low temperatures, whereas chains with high temperatures appear to sample trees with high posterior probabilities only rarely.     

Reconstructing Local Population Dynamics in Noisy Metapopulations—The Role of Random Catastrophes and Allee Effects

Reconstructing the dynamics of populations is complicated by the different types of stochasticity experienced by populations, in particular if some forms of stochasticity introduce bias in parameter estimation in addition to error. Identification of systematic biases is critical when determining whether the intrinsic dynamics of populations are stable or unstable and whether or not populations exhibit an Allee effect, i.e., a minimum size below which deterministic extinction should follow. Using a simulation model that allows for Allee effects and a range of intrinsic dynamics, we investigated how three types of stochasticity—demographic, environmental, and random catastrophes— affect our ability to reconstruct the intrinsic dynamics of populations. Demographic stochasticity aside, which is only problematic in small populations, we find that environmental stochasticity—positive and negative environmental fluctuations—caused increased error in parameter estimation, but bias was rarely problematic, except at the highest levels of noise. Random catastrophes, events causing large-scale mortality and likely to be more common than usually recognized, caused immediate bias in parameter estimates, in particular when Allee effects were large. In the latter case, population stability was predicted when endogenous dynamics were actually unstable and the minimum viable population size was overestimated in populations with small or non-existent Allee effects. Catastrophes also generally increased extinction risk, in particular when endogenous Allee effects were large. We propose a method for identifying data points likely resulting from catastrophic events when such events have not been recorded. Using social spider colonies (Anelosimus spp.) as models for populations, we show that after known or suspected catastrophes are accounted for, reconstructed growth parameters are consistent with intrinsic dynamical instability and substantial Allee effects. Our results are applicable to metapopulation or time series data and are relevant for predicting extinction in conservation applications or the management of invasive species.     

Relationship between Item Responses of Negative Affect Items and the Distribution of the Sum of the Item Scores in the General Population

BackgroundSeveral studies have shown that total depressive symptom scores in the general population approximate an exponential pattern, except for the lower end of the distribution. The Center for Epidemiologic Studies Depression Scale (CES-D) consists of 20 items, each of which may take on four scores: “rarely,” “some,” “occasionally,” and “most of the time.” Recently, we reported that the item responses for 16 negative affect items commonly exhibit exponential patterns, except for the level of “rarely,” leading us to hypothesize that the item responses at the level of “rarely” may be related to the non-exponential pattern typical of the lower end of the distribution. To verify this hypothesis, we investigated how the item responses contribute to the distribution of the sum of the item scores.MethodsData collected from 21,040 subjects who had completed the CES-D questionnaire as part of a Japanese national survey were analyzed. To assess the item responses of negative affect items, we used a parameter r, which denotes the ratio of “rarely” to “some” in each item response. The distributions of the sum of negative affect items in various combinations were analyzed using log-normal scales and curve fitting.ResultsThe sum of the item scores approximated an exponential pattern regardless of the combination of items, whereas, at the lower end of the distributions, there was a clear divergence between the actual data and the predicted exponential pattern. At the lower end of the distributions, the sum of the item scores with high values of r exhibited higher scores compared to those predicted from the exponential pattern, whereas the sum of the item scores with low values of r exhibited lower scores compared to those predicted.ConclusionsThe distributional pattern of the sum of the item scores could be predicted from the item responses of such items.