Test‐compatible confidence intervals for adaptive two‐stage single‐arm designs with binary endpoint

Inference after two‐stage single‐arm designs with binary endpoint is challenging due to the nonunique ordering of the sampling space in multistage designs. We illustrate the problem of specifying test‐compatible confidence intervals for designs with nonconstant second‐stage sample size and present two approaches that guarantee confidence intervals consistent with the test decision. Firstly, we extend the well‐known Clopper–Pearson approach of inverting a family of two‐sided hypothesis tests from the group‐sequential case to designs with fully adaptive sample size. Test compatibility is achieved by using a sample space ordering that is derived from a test‐compatible estimator. The resulting confidence intervals tend to be conservative but assure the nominal coverage probability. In order to assess the possibility of further improving these confidence intervals, we pursue a direct optimization approach minimizing the mean width of the confidence intervals. While the latter approach produces more stable coverage probabilities, it is also slightly anti‐conservative and yields only negligible improvements in mean width. We conclude that the Clopper–Pearson‐type confidence intervals based on a test‐compatible estimator are the best choice if the nominal coverage probability is not to be undershot and compatibility of test decision and confidence interval is to be preserved.     

Confidence Intervals and Tests of Hypotheses on Variance Components in an Unbalanced Two-fold Nested Design

Confidence intervals and tests of hypotheses on variance components are required in studies that employ a random effects design. The unbalanced random two-fold nested design is considered in this paper and confidence intervals are proposed for the variance components σ2/A and σ2/B. Computer simulation is used to show that even in very unbalanced designs, these intervals generally maintain the stated confidence coefficient. The hypothesis test for σ2/A based on the lower bound of the recommended confidence interval is shown to be better than previously proposed approximate tests.     

Contrasting Diversity Values: Statistical Inferences Based on Overlapping Confidence Intervals

Ecologists often contrast diversity (species richness and abundances) using tests for comparing means or indices. However, many popular software applications do not support performing standard inferential statistics for estimates of species richness and/or density. In this study we simulated the behavior of asymmetric log-normal confidence intervals and determined an interval level that mimics statistical tests with P(α) = 0.05 when confidence intervals from two distributions do not overlap. Our results show that 84% confidence intervals robustly mimic 0.05 statistical tests for asymmetric confidence intervals, as has been demonstrated for symmetric ones in the past. Finally, we provide detailed user-guides for calculating 84% confidence intervals in two of the most robust and highly-used freeware related to diversity measurements for wildlife (i.e., EstimateS, Distance).     

In-silico wear prediction for knee replacements—methodology and corroboration

The capability to predict in-vivo wear of knee replacements is a valuable pre-clinical analysis tool for implant designers. Traditionally, time-consuming experimental tests provided the principal means of investigating wear. Today, computational models offer an alternative. However, the validity of these models has not been demonstrated across a range of designs and test conditions, and several different formulas are in contention for estimating wear rates, limiting confidence in the predictive power of these in-silico models.This study collates and retrospectively simulates a wide range of experimental wear tests using fast rigid-body computational models with extant wear prediction algorithms, to assess the performance of current in-silico wear prediction tools.The number of tests corroborated gives a broader, more general assessment of the performance of these wear-prediction tools, and provides better estimates of the wear ‘constants’ used in computational models. High-speed rigid-body modelling allows a range of alternative algorithms to be evaluated. Whilst most cross-shear (CS)-based models perform comparably, the ‘A/A+B’ wear model appears to offer the best predictive power amongst existing wear algorithms. However, the range and variability of experimental data leaves considerable uncertainty in the results. More experimental data with reduced variability and more detailed reporting of studies will be necessary to corroborate these models with greater confidence. With simulation times reduced to only a few minutes, these models are ideally suited to large-volume ‘design of experiment’ or probabilistic studies (which are essential if pre-clinical assessment tools are to begin addressing the degree of variation observed clinically and in explanted components).     

Empirical study of seven data mining algorithms on different characteristics of datasets for biomedical classification applications

Background

Various kinds of data mining algorithms are continuously raised with the development of related disciplines. The applicable scopes and their performances of these algorithms are different. Hence, finding a suitable algorithm for a dataset is becoming an important emphasis for biomedical researchers to solve practical problems promptly.

Methods

In this paper, seven kinds of sophisticated active algorithms, namely, C4.5, support vector machine, AdaBoost, k-nearest neighbor, naïve Bayes, random forest, and logistic regression, were selected as the research objects. The seven algorithms were applied to the 12 top-click UCI public datasets with the task of classification, and their performances were compared through induction and analysis. The sample size, number of attributes, number of missing values, and the sample size of each class, correlation coefficients between variables, class entropy of task variable, and the ratio of the sample size of the largest class to the least class were calculated to character the 12 research datasets.

Results

The two ensemble algorithms reach high accuracy of classification on most datasets. Moreover, random forest performs better than AdaBoost on the unbalanced dataset of the multi-class task. Simple algorithms, such as the naïve Bayes and logistic regression model are suitable for a small dataset with high correlation between the task and other non-task attribute variables. K-nearest neighbor and C4.5 decision tree algorithms perform well on binary- and multi-class task datasets. Support vector machine is more adept on the balanced small dataset of the binary-class task.

Conclusions

No algorithm can maintain the best performance in all datasets. The applicability of the seven data mining algorithms on the datasets with different characteristics was summarized to provide a reference for biomedical researchers or beginners in different fields.

     

Sharp simultaneous confidence intervals for the means of selected populations with application to microarray data analysis

Summary Simultaneous inference for a large number, N, of parameters is a challenge. In some situations, such as microarray experiments, researchers are only interested in making inference for the K parameters corresponding to the K most extreme estimates. Hence it seems important to construct simultaneous confidence intervals for these K parameters. The naïve simultaneous confidence intervals for the K means (applied directly without taking into account the selection) have low coverage probabilities. We take an empirical Bayes approach (or an approach based on the random effect model) to construct simultaneous confidence intervals with good coverage probabilities. For N= 10,000 and K= 100, typical for microarray data, our confidence intervals could be 77% shorter than the naïve K‐dimensional simultaneous intervals.     

Effects of cluster sampling on epidemiologic analysis in population-based case-control studies

We consider population-based case-control designs in which controls are selected by one of three cluster sampling plans from the entire population at risk. The effects of cluster sampling on classical epidemiologic procedures are investigated, and appropriately modified procedures are developed. In particular, modified procedures for testing the homogeneity of odds ratios across strata, and for estimating and testing a common odds ratio are presented. Simulations that use the data from the 1970 Health Interview Survey as a population suggest that classical procedures may be fairly robust in the presence of cluster sampling. A more extreme example based on a mixed multinomial model clearly demonstrates that the classical Mantel-Haenszel (1959, Journal of the National Cancer Institute 22, 719-748) and Woolf-Haldane tests of no exposure effect may have sizes exceeding nominal levels and confidence intervals with less than nominal coverage under an alternative hypothesis. Classical estimates of odds ratios may also be biased with non-self-weighting cluster samples. The modified procedures we propose remedy these defects.     

Confidence Intervals for Concentration and Brightness from Fluorescence Fluctuation Measurements

The theory of photon count histogram (PCH) analysis describes the distribution of fluorescence fluctuation amplitudes due to populations of fluorophores diffusing through a focused laser beam and provides a rigorous framework through which the brightnesses and concentrations of the fluorophores can be determined. In practice, however, the brightnesses and concentrations of only a few components can be identified. Brightnesses and concentrations are determined by a nonlinear least-squares fit of a theoretical model to the experimental PCH derived from a record of fluorescence intensity fluctuations. The χ² hypersurface in the neighborhood of the optimum parameter set can have varying degrees of curvature, due to the intrinsic curvature of the model, the specific parameter values of the system under study, and the relative noise in the data. Because of this varying curvature, parameters estimated from the least-squares analysis have varying degrees of uncertainty associated with them. There are several methods for assigning confidence intervals to the parameters, but these methods have different efficacies for PCH data. Here, we evaluate several approaches to confidence interval estimation for PCH data, including asymptotic standard error, likelihood joint-confidence region, likelihood confidence intervals, skew-corrected and accelerated bootstrap (BCa), and Monte Carlo residual resampling methods. We study these with a model two-dimensional membrane system for simplicity, but the principles are applicable as well to fluorophores diffusing in three-dimensional solution. Using simulated fluorescence fluctuation data, we find the BCa method to be particularly well-suited for estimating confidence intervals in PCH analysis, and several other methods to be less so. Using the BCa method and additional simulated fluctuation data, we find that confidence intervals can be reduced dramatically for a specific non-Gaussian beam profile.     

Mini-review: effect sizes and meta-analysis for antifouling research

Abstract

It is widely recognised that findings from experimental studies should be replicated before their conclusions are accepted as definitive. In many research areas, synthesis of results from multiple studies is carried out via systematic review and meta-analysis. Some fields are also moving away from null hypothesis significance testing, which uses p values to identify ‘significant’ effects, towards an estimation approach concerned with effect sizes and confidence intervals. This review argues that these techniques are underused in biofouling and antifouling (AF) research and discusses potential benefits of their adoption. They enable comparison of test surfaces even when these are not tested simultaneously, and allow results from repeated tests on the same surfaces to be combined. They also enable the use of published data to explore effects of different variables on the functioning of AF surfaces. AF researchers should consider using these approaches and reporting results in ways that facilitate future research syntheses.     

Bootstrap methods for estimating spatial synchrony of fluctuating populations

We describe and examine methods for estimating spatial correlations used in population ecology. We base our analyses on a hypothetical example of a species that has been censured at 30 different locations for 20 years. We assume that the population fluctuations can be described by a simple linear model on logarithmic scale. Stochastic simulations is utilized to check how seven different ways of resampling perform when the goal is to find nominal 95% confidence intervals for the spatial correlation in growth rates at given distances. It turns out that resampling of locations performs badly, with true coverage level as low as 30–40%, especially for small correlations at long distances. Resampling of timepoints performs much better, with coverage varying from 80 to 90%, depending on the strength of density regulation and whether the spatial correlation is estimated for the response variable or for the error terms in the model. Assuming that the underlying model is known, the best results are achieved for parametric bootstrapping, a result that strongly emphasize the importance of defining and estimating a proper population model when studying spatial processes.     

Biometry has to improve the quality of decision making both in biomedical research and in clinical practice

The leading principles of biometrical design, evaluation and decision making in biomedicine can be summarized as follows: (1) formulation of the problem to be solved and of the specific question(s) to be answered; (2) definition of the population in study, ways of sampling and manoeuvres (treatments); (3) determining the kind and number of variables (time intervals of measurement, investigated biomedical parameters); (4) performing the pilot study if necessary (to arrange randomized blocks, to form hypotheses); (5) formulation of zero (alternative) hypothesis, setting the values of alpha(beta) risk; (6) ordering the sample sizes; (7) testing the type of statistical distribution and the homoscedasticity of the obtained data; (8) calculation of the point and interval (confidence, tolerance) estimates; (9) performing the chosen tests, e.g. on the significance of differences or correlations; and (10) the conclusions for science and practice, with eventual return to point (1) on a higher level of knowledge.     

Empirical nonparametric bootstrap strategies in quantitative trait loci mapping: conditioning on the genetic model.

Several nonparametric bootstrap methods are tested to obtain better confidence intervals for the quantitative trait loci (QTL) positions, i.e., with minimal width and unbiased coverage probability. Two selective resampling schemes are proposed as a means of conditioning the bootstrap on the number of genetic factors in our model inferred from the original data. The selection is based on criteria related to the estimated number of genetic factors, and only the retained bootstrapped samples will contribute a value to the empirically estimated distribution of the QTL position estimate. These schemes are compared with a nonselective scheme across a range of simple configurations of one QTL on a one-chromosome genome. In particular, the effect of the chromosome length and the relative position of the QTL are examined for a given experimental power, which determines the confidence interval size. With the test protocol used, it appears that the selective resampling schemes are either unbiased or least biased when the QTL is situated near the middle of the chromosome. When the QTL is closer to one end, the likelihood curve of its position along the chromosome becomes truncated, and the nonselective scheme then performs better inasmuch as the percentage of estimated confidence intervals that actually contain the real QTL''s position is closer to expectation. The nonselective method, however, produces larger confidence intervals. Hence, we advocate use of the selective methods, regardless of the QTL position along the chromosome (to reduce confidence interval sizes), but we leave the problem open as to how the method should be altered to take into account the bias of the original estimate of the QTL''s position.     

Use of resampling to assess optimal subgroup composition for estimating genetic parameters from progeny trials

Estimations of genetic parameters of wood traits based on reduced sample populations are widely reported in the literature, but few investigations have considered the consequences of these small populations on the precision of parameter estimates. The purpose of this study was to determine an optimal strategy for sampling subgroups, by varying either the number of families or the number of individuals (trees) per family, and by verifying the accuracy of certain genetic parameters (across-trials analysis). To achieve this, simulations were conducted using random resampling without replacement (k?=?1,000/pair of varying factors) on datasets containing 10-year total height of two coniferous species (Larix laricina and Picea mariana), as well as pilodyn measurements of wood density evaluated on a 26-year-old population of P. mariana. SAS® 9.2 Macro Language and Procedures were used to estimate confidence intervals of several genetic parameters with different reduced samplings. Simulation results show that reducing the number of trees per family per site had more impact on the magnitude and precision of genetic parameter estimates than reducing the number of families, especially for half-sib heritability and type B genetic correlations for height and wood density. A priori determination of an optimal subsampling strategy to evaluate the accuracy of genetic parameters should become common practice before assessing wood traits, in tree breeding studies or when planning juvenile retrospective progeny trials for forest tree species.     

Drop-the-losers design: normal case

Drop-the-losers designs are statistical designs which have two stages of a trial separated by a data based decision. In the first stage k experimental treatments and a control are administered. During a transition period, the empirically best experimental treatment is selected for continuation into the second phase, along with the control. At the study's end, inference focuses on the comparison of the selected treatment with the control using both stages' data. Traditional methods used to make inferences based on both stages' data can yield tests with higher than advertised levels of significance and confidence intervals with lower than advertised confidence. For normally distributed data, methods are provided to correct these deficiencies, providing confidence intervals with accurate levels of confidence. Drop-the-losers designs are particularly applicable to biopharmaceutical clinical trials where they can allow Phase II and Phase III clinical trials to be conducted under a single protocol with the use of all available data.     

Pathogen threat and intergroup prejudice using the minimal group paradigm: Evidence from a registered report

Infection by parasites, bacteria, and other microorganisms has been a powerful selection pressure faced by humans and other species. Consequently, avoiding pathogens has played an important role in human evolution and continues to play a role in contemporary social psychological processes. The current research tested the hypothesis that pathogen avoidance promotes intergroup prejudice. Whereas previous tests relied on existing cultural groups, which can conflate outgroup status with pre-existing group stereotypes about disease or geographic variability in pathogen prevalence, the current experiments assessed intergroup bias using a minimal group paradigm. Based on preliminary evidence (Study 1, N = 207, undergraduate students) that experimentally priming pathogen avoidance motivation promoted negativity toward a minimal outgroup, we conducted a Registered Report (Study 2, N = 1339 online participants) to replicate and extend those findings. Our primary hypothesis was that an experimental manipulation of pathogen threat (relative to two control conditions) would produce greater intergroup prejudice (negativity toward the nominal outgroup relative to the nominal ingroup). This hypothesis was not supported in the larger, registered second study. Exploratory analyses provided some evidence for interactions between experimental priming of pathogen threat and individual differences in pathogen disgust, but the interactive pattern differed across the two experiments. Findings call into question the hypothesis that, in the absence of cultural stereotypes, situationally activated pathogen avoidance promotes intergroup prejudice.     

Combining literature text mining with microarray data: advances for system biology modeling

A huge amount of important biomedical information is hidden in the bulk of research articles in biomedical fields. At the same time, the publication of databases of biological information and of experimental datasets generated by high-throughput methods is in great expansion, and a wealth of annotated gene databases, chemical, genomic (including microarray datasets), clinical and other types of data repositories are now available on the Web. Thus a current challenge of bioinformatics is to develop targeted methods and tools that integrate scientific literature, biological databases and experimental data for reducing the time of database curation and for accessing evidence, either in the literature or in the datasets, useful for the analysis at hand. Under this scenario, this article reviews the knowledge discovery systems that fuse information from the literature, gathered by text mining, with microarray data for enriching the lists of down and upregulated genes with elements for biological understanding and for generating and validating new biological hypothesis. Finally, an easy to use and freely accessible tool, GeneWizard, that exploits text mining and microarray data fusion for supporting researchers in discovering gene-disease relationships is described.     

Pair-switching rerandomization

Rerandomization discards assignments with covariates unbalanced in the treatment and control groups to improve estimation and inference efficiency. However, the acceptance-rejection sampling method used in rerandomization is computationally inefficient. As a result, it is time-consuming for rerandomization to draw numerous independent assignments, which are necessary for performing Fisher randomization tests and constructing randomization-based confidence intervals. To address this problem, we propose a pair-switching rerandomization (PSRR) method to draw balanced assignments efficiently. We obtain the unbiasedness and variance reduction of the difference-in-means estimator and show that the Fisher randomization tests are valid under PSRR. Moreover, we propose an exact approach to invert Fisher randomization tests to confidence intervals, which is faster than the existing methods. In addition, our method is applicable to both nonsequentially and sequentially randomized experiments. We conduct comprehensive simulation studies to compare the finite-sample performance of the proposed method with that of classical rerandomization. Simulation results indicate that PSRR leads to comparable power of Fisher randomization tests and is 3–23 times faster than classical rerandomization. Finally, we apply the PSRR method to analyze two clinical trial datasets, both of which demonstrate the advantages of our method.     

Confidence Intervals for Morphology-Based Cladistic Trees: A Platyrrhine Phylogeny Test Case

I estimate confidence intervals for phylogenetic trees based on bootstrap resampling while calculating special coefficients of similarity. I treat each successive cladistic dichotomy as a null hypothesis for sampling from a universe of cranial and postcranial synapomorphically-based similarities that includes the next lower similarity. Successive dichotomies that are not at significantly different similarity levels are collapsed into polytomies. Following a trial application to equid cladistic traits employed in Felsenstein's introduction (Felsenstein, J. (1985). Evolution 39: 783–791), I apply the methods to New World monkey relationships using morphological character sets. Unresolvable polytomies among platyrrhine subfamilies are the rule when these methods are applied.     

Categorical data analysis in experimental biology

The categorical data set is an important data class in experimental biology and contains data separable into several mutually exclusive categories. Unlike measurement of a continuous variable, categorical data cannot be analyzed with methods such as the Student's t-test. Thus, these data require a different method of analysis to aid in interpretation. In this article, we will review issues related to categorical data, such as how to plot them in a graph, how to integrate results from different experiments, how to calculate the error bar/region, and how to perform significance tests. In addition, we illustrate analysis of categorical data using experimental results from developmental biology and virology studies.     

Inference for reclassification statistics under nested and non-nested models for biomarker evaluation

Abstract

The Net Reclassification Improvement (NRI) and the Integrated Discrimination Improvement (IDI) are used to evaluate the diagnostic accuracy improvement for biomarkers in a wide range of applications. Most applications for these reclassification metrics are confined to nested model comparison. We emphasize the important extensions of these metrics to the non-nested comparison. Non-nested models are important in practice, in particular, in high-dimensional data analysis and in sophisticated semiparametric modeling. We demonstrate that the assessment of accuracy improvement may follow the familiar NRI and IDI evaluation. While the statistical properties of the estimators for NRI and IDI have been well studied in the nested setting, one cannot always rely on these asymptotic results to implement the inference procedure for practical data, especially for testing the null hypothesis of no improvement, and these properties have not been established for the non-nested setting. We propose a generic bootstrap re-sampling procedure for the construction of confidence intervals and hypothesis tests. Extensive simulations and real biomedical data examples illustrate the applicability of the proposed inference methods for both nested and non-nested models.