Classification based upon gene expression data: bias and precision of error rates

MOTIVATION: Gene expression data offer a large number of potentially useful predictors for the classification of tissue samples into classes, such as diseased and non-diseased. The predictive error rate of classifiers can be estimated using methods such as cross-validation. We have investigated issues of interpretation and potential bias in the reporting of error rate estimates. The issues considered here are optimization and selection biases, sampling effects, measures of misclassification rate, baseline error rates, two-level external cross-validation and a novel proposal for detection of bias using the permutation mean. RESULTS: Reporting an optimal estimated error rate incurs an optimization bias. Downward bias of 3-5% was found in an existing study of classification based on gene expression data and may be endemic in similar studies. Using a simulated non-informative dataset and two example datasets from existing studies, we show how bias can be detected through the use of label permutations and avoided using two-level external cross-validation. Some studies avoid optimization bias by using single-level cross-validation and a test set, but error rates can be more accurately estimated via two-level cross-validation. In addition to estimating the simple overall error rate, we recommend reporting class error rates plus where possible the conditional risk incorporating prior class probabilities and a misclassification cost matrix. We also describe baseline error rates derived from three trivial classifiers which ignore the predictors. AVAILABILITY: R code which implements two-level external cross-validation with the PAMR package, experiment code, dataset details and additional figures are freely available for non-commercial use from http://www.maths.qut.edu.au/profiles/wood/permr.jsp     

Simultaneous Discovery,Estimation and Prediction Analysis of Complex Traits Using a Bayesian Mixture Model

Gene discovery, estimation of heritability captured by SNP arrays, inference on genetic architecture and prediction analyses of complex traits are usually performed using different statistical models and methods, leading to inefficiency and loss of power. Here we use a Bayesian mixture model that simultaneously allows variant discovery, estimation of genetic variance explained by all variants and prediction of unobserved phenotypes in new samples. We apply the method to simulated data of quantitative traits and Welcome Trust Case Control Consortium (WTCCC) data on disease and show that it provides accurate estimates of SNP-based heritability, produces unbiased estimators of risk in new samples, and that it can estimate genetic architecture by partitioning variation across hundreds to thousands of SNPs. We estimated that, depending on the trait, 2,633 to 9,411 SNPs explain all of the SNP-based heritability in the WTCCC diseases. The majority of those SNPs (>96%) had small effects, confirming a substantial polygenic component to common diseases. The proportion of the SNP-based variance explained by large effects (each SNP explaining 1% of the variance) varied markedly between diseases, ranging from almost zero for bipolar disorder to 72% for type 1 diabetes. Prediction analyses demonstrate that for diseases with major loci, such as type 1 diabetes and rheumatoid arthritis, Bayesian methods outperform profile scoring or mixed model approaches.     

A core system for the implementation of task sets

When performing tasks, humans are thought to adopt task sets that configure moment-to-moment data processing. Recently developed mixed blocked/event-related designs allow task set-related signals to be extracted in fMRI experiments, including activity related to cues that signal the beginning of a task block, "set-maintenance" activity sustained for the duration of a task block, and event-related signals for different trial types. Data were conjointly analyzed from mixed design experiments using ten different tasks and 183 subjects. Dorsal anterior cingulate cortex/medial superior frontal cortex (dACC/msFC) and bilateral anterior insula/frontal operculum (aI/fO) showed reliable start-cue and sustained activations across all or nearly all tasks. These regions also carried the most reliable error-related signals in a subset of tasks, suggesting that the regions form a "core" task-set system. Prefrontal regions commonly related to task control carried task-set signals in a smaller subset of tasks and lacked convergence across signal types.     

Whole genome approaches to quantitative genetics

Apart from parent-offspring pairs and clones, relative pairs vary in the proportion of the genome that they share identical by descent. In the past, quantitative geneticists have used the expected value of sharing genes by descent to estimate genetic parameters and predict breeding values. With the possibility to genotype individuals for many markers across the genome it is now possible to empirically estimate the actual relationship between relatives. We review some of the theory underlying the variation in genetic identity, show applications to estimating genetic variance for height in humans and discuss other applications.     

Collagen-induced arthritis in common marmosets: a new nonhuman primate model for chronic arthritis

Introduction  

There is an ever-increasing need for animal models to evaluate efficacy and safety of new therapeutics in the field of rheumatoid arthritis (RA). Particularly for the early preclinical evaluation of human-specific biologicals targeting the progressive phase of the disease, there is a need for relevant animal models. In response to this requirement we set out to develop a model of collagen-induced arthritis (CIA) in a small-sized nonhuman primate species (300 to 400 g at adult age); that is, the common marmoset (Callithrix jacchus).     

Aspects of early arthritis. What determines the evolution of early undifferentiated arthritis and rheumatoid arthritis? An update from the Norfolk Arthritis Register

Over 3500 patients with recent onset inflammatory polyarthritis (IP) have been recruited by the Norfolk Arthritis Register (NOAR) since 1990. Longitudinal data from this cohort have been used to examine the prevalence and predictors of remission, functional disability, radiological outcome, cardiovascular mortality and co-morbidity and the development of non-Hodgkin's lymphoma. Rheumatoid factor titre, high baseline C-reactive protein and high baseline HAQ score are all predictors of a poor outcome. There is a strong association between possession of the shared epitope and the development of erosions. Patients who satisfy the American College of Rheumatology criteria for rheumatoid arthritis (RA) have a worse prognosis than those who do not. However, it appears that these patients are a poorly defined subset of all those with IP rather than having an entirely separate disease entity. New statistical techniques offer exciting possibilities for using longitudinal datasets such as NOAR to explore the long-term effects of treatment in IP and RA.     

Linkage disequilibrium in the domesticated pig

This study investigated the extent of linkage disequilibrium (LD) in two genomic regions (on chromosomes 4 and 7) in five populations of domesticated pigs. LD was measured with D' and tested for significance with the Fisher exact test. Effects of genetic (linkage) distance, chromosome, population, and their interactions on D' were tested both through a linear model analysis of covariance and by a theoretical nonlinear model. The overall result was that (1) the distance explained most of the variability of D', (2) the effect of chromosome was significant, and (3) the effect of population was significant. The significance of the chromosome effect may have resulted from selection and the significance of the population effect illustrates the effects of population structures and effective population sizes on LD. These results suggest that mapping methods based on LD may be valuable even with only moderately dense marker spacing in pigs.     

Parental assignment in fish using microsatellite genetic markers with finite numbers of parents and offspring

Deterministic predictions for the proportion of offspring assigned to different numbers of parent-pairs are developed in order to investigate the power of microsatellite loci for parental assignment in fish species. Comparisons with stochastic simulation results show that predictions based on exclusion probabilities are accurate, provided that the number of parents involved in the crosses is large. Accounting for sampling of parents gave very accurate predictions for a small number of parents and a single biallelic locus. For large numbers of loci or large numbers of alleles per locus stochastic simulations are, however, the only available method to predict the power of assignment of a particular set of loci when the number of parents is small. Nine 5-allele loci or six 10-allele loci with equifrequent alleles, are sufficient for assigning, with certainty, parents to 99% of the fish resulting from either 100 or 400 crosses. Results simulating a set of highly polymorphic microsatellites developed for Atlantic salmon show that the four most informative loci are sufficient to assign at least 99% of the offspring to the correct pair with 100 crosses involving 100 males and 100 females. An additional locus is required for correctly assigning 99% of the offspring when the 100 crosses are produced with 10 males and 10 females.     

On the mapping of quantitative trait loci at marker and non-marker locations

Previous studies have noted that the estimated positions of a large proportion of mapped quantitative trait loci (QTLs) coincide with marker locations and have suggested that this indicates a bias in the mapping methodology. In this study we predict the expected proportion of QTLs with positions estimated to be at the location of a marker and further examine the problem using simulated data. The results show that the higher proportion of putative QTLs estimated to be at marker positions compared with non-marker positions is an expected consequence of the estimation methods. The study initially focused on a single interval with no QTLs and was extended to include multiple intervals and QTLs of large effect. Further, the study demonstrated that the larger proportion of estimated QTL positions at the location of markers was not unique to linear regression mapping. Maximum likelihood produced similar results, although the accumulation of positional estimates at outermost markers was reduced when regions outside the linkage group were also considered. The bias towards marker positions is greatest under the null hypothesis of no QTLs or when QTL effects are small. This study discusses the impact the findings could have on the calculation of thresholds and confidence intervals produced by bootstrap methods.     

Mapping of quantitative trait loci on porcine chromosome 4

A F₂ population derived from a cross between European Large White and Chinese Meishan pigs was established in order to study the genetic basis of breed differences for growth and fat traits. Chromosome 4 was chosen for initial study as previous work had revealed quantitative trait loci (QTLs) on this chromosome affected growth and fat traits in a Wild Boar × Large White cross. Individuals in the F₂ population were typed for nine markers spanning a region of approximately 124 c m . We found evidence for QTLs affecting growth between weaning and the end of test (additive effect: 43·4 g/day) and fat depth measured in the mid-back position (additive effect: 1·82 mm). There was no evidence of interactions between the QTLs and sex, grandparents or F₁ sires, suggesting that the detected QTLs were fixed for alternative alleles in the Meishan and Large White breeds. Comparison of locations suggests that these QTLs could be the same as those found in the Wild Boar × Large White cross.