Examining the practical limits of batch effect-correction algorithms: When should you care about batch effects?

Batch effects are technical sources of variation and can confound analysis. While many performance ranking exercises have been conducted to establish the best batch effect-correction algorithm (BECA), we hold the viewpoint that the notion of best is context-dependent. Moreover, alternative questions beyond the simplistic notion of “best” are also interesting: are BECAs robust against various degrees of confounding and if so, what is the limit? Using two different methods for simulating class (phenotype) and batch effects and taking various representative datasets across both genomics (RNA-Seq) and proteomics platforms, we demonstrate that under situations where sample classes and batch factors are moderately confounded, most BECAs are remarkably robust and only weakly affected by upstream normalization procedures. This observation is consistently supported across the multitude of test datasets. BECAs do have limits: When sample classes and batch factors are strongly confounded, BECA performance declines, with variable performance in precision, recall and also batch correction. We also report that while conventional normalization methods have minimal impact on batch effect correction, they do not affect downstream statistical feature selection, and in strongly confounded scenarios, may even outperform BECAs. In other words, removing batch effects is no guarantee of optimal functional analysis. Overall, this study suggests that simplistic performance ranking exercises are quite trivial, and all BECAs are compromises in some context or another.     

In the eye (and ears) of the beholder: Receiver psychology and human signal design

Although the study of signals has been part of human behavioral ecology since the field's inception,¹ only recently has signaling theory become important to the evolutionary study of human behavior and culture.² Signaling theory's rise to prominence has been propelled mainly by applications of costly signaling theory,³ which has shed light on a wide variety of human behaviors ranging from hunting⁴ to religion.^5,6 Costly signaling rests on the idea that wasteful but highly visible traits and behaviors can be explained as honest indicators of underlying qualities that are otherwise difficult to detect. For example, a laborious hunting technique may serve as a display of skill on the part of the hunter, who may then be favorably perceived by potential mates and allies.⁴ The costs of the activity ensure that the signal is honest, since unskilled hunters will not be able to perform as well. Despite the usefulness of this perspective, many such studies begin by documenting a costly behavior that is then explained with reference to costly signaling theory. Because such behaviors are easy to detect, they may be overemphasized in the literature.⁷ Moreover, costly signaling theory by itself can explain neither all signals nor all aspects of signal design. In this review, we argue that a focus on the role that the psychology of the intended receiver plays in signal design can expand the scope of signaling theory as a promising avenue to explain human behavior.     

Plotting receiver operating characteristic and precision–recall curves from presence and background data

1. The receiver operating characteristic (ROC) and precision–recall (PR) plots have been widely used to evaluate the performance of species distribution models. Plotting the ROC/PR curves requires a traditional test set with both presence and absence data (namely PA approach), but species absence data are usually not available in reality. Plotting the ROC/PR curves from presence‐only data while treating background data as pseudo absence data (namely PO approach) may provide misleading results.
2. In this study, we propose a new approach to calibrate the ROC/PR curves from presence and background data with user‐provided information on a constant c, namely PB approach. Here, c defines the probability that species occurrence is detected (labeled), and an estimate of c can also be derived from the PB‐based ROC/PR plots given that a model with good ability of discrimination is available. We used five virtual species and a real aerial photography to test the effectiveness of the proposed PB‐based ROC/PR plots. Different models (or classifiers) were trained from presence and background data with various sample sizes. The ROC/PR curves plotted by PA approach were used to benchmark the curves plotted by PO and PB approaches.
3. Experimental results show that the curves and areas under curves by PB approach are more similar to that by PA approach as compared with PO approach. The PB‐based ROC/PR plots also provide highly accurate estimations of c in our experiment.
4. We conclude that the proposed PB‐based ROC/PR plots can provide valuable complements to the existing model assessment methods, and they also provide an additional way to estimate the constant c (or species prevalence) from presence and background data.

     

Riboflavin as an independent and accurate biomarker for adherence in a randomized double-blind and placebo-controlled clinical trial

Background: Medication adherence is critical for success of clinical trials.

Objective: To assess oral riboflavin is an adherence marker.

Methods: Riboflavin was incorporated into active treatment and placebo pills for a clinical trial lasting for 2 years.

Results: The accuracy (area under the receiver operating curve) of urinary riboflavin was 0.91 as a binary classifier of adherence, and was similar or better than for two active study ingredients daidzein (0.92) and genistein (0.87) (all p?<?0.0001). Decreased adherence over time was similar in the two study groups.

Conclusion: Riboflavin is an accurate and useful biomarker for study pill ingestion.     

Crystal structure of master biofilm regulator CsgD regulatory domain reveals an atypical receiver domain

The master regulator CsgD switches planktonic growth to biofilm formation by activating synthesis of curli fimbriae and cellulose in Enterobacteriaceae. CsgD was classified to be the LuxR response regulatory family, while its cognate sensor histidine kinase has not been identified yet. CsgD consists of a C‐terminal DNA binding domain and an N‐terminal regulatory domain that provokes the upstream signal transduction to further modulate its function. We provide the crystal structure of Salmonella Typhimurium CsgD regulatory domain, which reveals an atypical β5α5 response regulatory receiver domain folding with the α2 helix representing as a disorder loop compared to the LuxR/FixJ canonical response regulator, and the structure indicated a noteworthy α5 helix similar to the non‐canonical master regulator VpsT receiver domain α6. CsgD regulatory domain assembles with two dimerization interfaces mainly through α1 and α5, which has shown similarity to the c‐di‐GMP independent and stabilized dimerization interface of VpsT from Vibrio cholerae respectively. The potential phosphorylation site D59 is directly involved in the interaction of interfaces I and mutagenesis studies indicated that both dimerization interfaces could be crucial for CsgD activity. The structure reveals important molecular details for the dimerization assembly of CsgD and will shed new insight into its regulation mechanism.     

Cultural complexity and demography: The case of folktales

We investigate the relationship between cultural complexity and population size in a non-technological cultural domain for which we have suitable quantitative records: folktales. We define three levels of complexity for folk narratives: the number of tale types, the number of narrative motifs, and, finally, the number of traits in variants of the same type, for two well-known tales for which we have data from previous studies. We found a positive relationship between number of tale types and population size, a negative relationship for the number of narrative motifs, and no relationship for the number of traits. The absence of a consistent relationship between population size and complexity in folktales provides a novel perspective on the current debates in cultural evolution. We propose that the link between cultural complexity and demography could be domain dependent: in some domains (e.g. technology) this link is important, whereas in others, such as folktales, complex traditions can be easily maintained in small populations as well as large ones, as they may appeal to universal cognitive biases.     

Predator exposure alters female mate choice in the green swordtail

Female green swordtails, Xiphophorus helleri, show a matingpreference for males with brightly colored, elongated swords.This preference is thought to be due to a preexisting receiverbias favoring longer sworded males. In this study, we examinedvariation in the expression of this sword bias in females. Specifically,we tested the hypothesis that an increase in perceived predationrisk will decrease female response to males with longer swords.We used a video playback experiment to evaluate female choicebetween two recordings of a displaying male that differed onlyin sword length. We scored responses of females to these recordingsimmediately before and after they had been exposed to a videorecording of a predation event between a cichlid and a malepossessing a long sword. We found that prior to exposure tothis predation event, females preferred the male with the longersword. However, after exposure to the predator, females alteredtheir mating response, preferring the male with the sword removed.Exposure to the predator also caused an increase in the frequencywith which females moved from potential mating positions toa neutral zone. The results presented here suggest that thefemale preference for males with longer swords can be modulatedbased on the perceived risk of predation.     

Near Homogeneity of PR2-Bias Fingerprints in the Human Genome and Their Implications in Phylogenetic Analyses

Genes of a multicellular organism are heterogeneous in the G+C content, which is particularly true in the third codon position. The extent of deviation from intra-strand equality rule of A = T and G = C (Parity Rule 2, or PR2) is specific for individual amino acids and has been expressed as the PR2-bias fingerprint. Previous results suggested that the PR2-bias fingerprints tend to be similar among the genes of an organism, and the fingerprint of the organism is specific for different taxa, reflecting phylogenetic relationships of organisms. In this study, using coding sequences of a large number of human genes, we examined the intragenomic heterogeneity of their PR2-bias fingerprints in relation to the G+C content of the third codon position (P ₃ ). Result shows that the PR2-bias fingerprint is similar in the wide range of the G+C content at the third codon position (0.30–0.80). This range covers approximately 89% of the genes, and further analysis of the high G+C range (0.80–1.00), where genes with normal PR2-bias fingerprints and those with anomalous fingerprints are mixed, shows that the total of 95% of genes have the similar finger prints. The result indicates that the PR2-bias fingerprint is a unique property of an organism and represents the overall characteristics of the genome. Combined with the previous results that the evolutionary change of the PR2-bias fingerprint is a slow process, PR2-bias fingerprints may be used for the phylogenetic analyses to supplement and augment the conventional methods that use the differences of the sequences of orthologous proteins and nucleic acids. Potential advantages and disadvantages of the PR2-bias fingerprint analysis are discussed. Received: 21 December 2000 / Accepted: 16 February 2001