Re-analysis of data and its integration

To understand a biological process it is clear that a single approach will not be sufficient, just like a single measurement on a protein--such as its expression level--does not describe protein function. Using reference sets of proteins as benchmarks different approaches can be scaled and integrated. Here, we demonstrate the power of data re-analysis and integration by applying it in a case study to data from deletion phenotype screens and mRNA expression profiling.     

Benchmarking protein classification algorithms via supervised cross-validation

Development and testing of protein classification algorithms are hampered by the fact that the protein universe is characterized by groups vastly different in the number of members, in average protein size, similarity within group, etc. Datasets based on traditional cross-validation (k-fold, leave-one-out, etc.) may not give reliable estimates on how an algorithm will generalize to novel, distantly related subtypes of the known protein classes. Supervised cross-validation, i.e., selection of test and train sets according to the known subtypes within a database has been successfully used earlier in conjunction with the SCOP database. Our goal was to extend this principle to other databases and to design standardized benchmark datasets for protein classification. Hierarchical classification trees of protein categories provide a simple and general framework for designing supervised cross-validation strategies for protein classification. Benchmark datasets can be designed at various levels of the concept hierarchy using a simple graph-theoretic distance. A combination of supervised and random sampling was selected to construct reduced size model datasets, suitable for algorithm comparison. Over 3000 new classification tasks were added to our recently established protein classification benchmark collection that currently includes protein sequence (including protein domains and entire proteins), protein structure and reading frame DNA sequence data. We carried out an extensive evaluation based on various machine-learning algorithms such as nearest neighbor, support vector machines, artificial neural networks, random forests and logistic regression, used in conjunction with comparison algorithms, BLAST, Smith-Waterman, Needleman-Wunsch, as well as 3D comparison methods DALI and PRIDE. The resulting datasets provide lower, and in our opinion more realistic estimates of the classifier performance than do random cross-validation schemes. A combination of supervised and random sampling was used to construct model datasets, suitable for algorithm comparison.

The datasets are available at http://hydra.icgeb.trieste.it/benchmark.     

Second litter syndrome in Iberian pig breed: factors influencing the performance

Second litter syndrome (SLS) consists of a loss of prolificacy in the second parity (P2), when a sow presents the same or lower results for litter size than in the first parity (P1). This syndrome has been reported for modern prolific breeds but has not been studied for rustic breeds. The objectives of this study are to determine how and to what degree Iberian sows (a low productivity breed recently raised on intensive farms) are affected by SLS; to establish a target and reference levels; and to assess the factors influencing the performance. Analysed data correspond to 66 Spanish farms with a total of 126 140 Iberian sows. The average Iberian sow prolificacy in P1 was 8.91 total born (TB) and 8.47 born alive (BA) piglets, whereas in P2, it decreased by ?0.05 TB and ?0.01 BA piglets, suggesting some general incidence of SLS. At the sow level, 56.63% did not improve prolificacy in terms of BA piglets in P2, and 16.98% had a clear decrease in prolificacy, losing ≥3 BA piglets in P2. Within herds, a mean of 57.75% of sows showed SLS, with an evident decrease in the number of BA piglets in P2. The plausible target for the Iberian farm’s prolificacy comes from the quartile of farms with the lowest percentage of SLS sows within the farms with the highest prolificacy between P1 and P2 (mean of 8.77 BA). So, in this subset of farms (N = 17), 47.3% of sows improved their prolificacy in P2 (i.e. did not show SLS). Hence, half the sows could be expected to show SLS even on farms with a good performance. Finally, this study brings out the main factors reducing P2 prolificacy through SLS in the Iberian breed: later age at first farrowing, long first lactation length, medium weaning to conception interval and large litter size in P1. In conclusion, improving the reproductive performance of Iberian farms requires reducing the percentage of sows with SLS, paying special attention to those risk factors. The knowledge derived from this study can provide references for comparing and establishing objectives of performance on Iberian sow farms which can be used for other robust breeds.     

Recent advances in mass spectrometry-based computational metabolomics

The computational metabolomics field brings together computer scientists, bioinformaticians, chemists, clinicians, and biologists to maximize the impact of metabolomics across a wide array of scientific and medical disciplines. The field continues to expand as modern instrumentation produces datasets with increasing complexity, resolution, and sensitivity. These datasets must be processed, annotated, modeled, and interpreted to enable biological insight. Techniques for visualization, integration (within or between omics), and interpretation of metabolomics data have evolved along with innovation in the databases and knowledge resources required to aid understanding. In this review, we highlight recent advances in the field and reflect on opportunities and innovations in response to the most pressing challenges. This review was compiled from discussions from the 2022 Dagstuhl seminar entitled “Computational Metabolomics: From Spectra to Knowledge”.     

Setting reference conditions for mesohaline and oligohaline macroinvertebrate communities sensu WFD: Helping to define achievable scenarios in basin management plans

The Water Framework Directive requires all Member States to achieve good ecological quality status for all waters (e.g., transitional waters). For that purpose, Member States must assess water bodies based on information on the Biological Quality Elements listed for each of them (e.g., benthic macroinvertebrates). However, the production of such a quality status classification (high, good, moderate, poor, bad) requires high reference conditions (associated with the absence of, or very low, human pressure) against which the deviation of the samples to be assessed can be measured. In transitional waters, upper stretches have seldom been included in monitoring activities, resulting in very little knowledge of mesohaline and oligohaline areas, which means further difficulty when defining the required reference conditions for these zones.Regarding the benthic macroinvertebrates, large datasets from the mesohaline and oligohaline stretches of the Mondego estuary (four seasons, five years, environmental parameters, density and biomass data) were used to estimate high reference condition values. In terms of environmental conditions, summer was identified as the most stable season and the most suitable for defining reference conditions for selected ecological indicators. For each indicator, the multivariate linear model expressing the best correlation with measured environmental parameters was selected. These models were used afterwards, by replacing the environmental parameters in those equations with their high reference values, to calculate the reference condition for each ecological indicator.Generally, macrobenthic communities within each stretch changed over the years, being mainly influenced by salinity and sediment organic matter. In both stretches, only a few taxa occurred and two species (the amphipod Corophium multisetosum and the bivalve Corbicula fluminea) were clearly dominant. Diversity values (for Margalef, Shannon and ES₅₀ – Hurlbert indices) were low in both stretches, although higher in the mesohaline, and for the most part the ecological condition was low (AMBI – AZTI Marine Biotic Index, MEDDOC – Mediterranean Occidental index, BENTIX biotic index, BO2A – Benthic Opportunistic Annelida Amphipod index). On the whole, the RC estimated for each index followed the same trend, being different for each stretch and below those found for lower sections of the estuary in other surveys.     

Exploration of freely available web-interfaces for comparative homology modelling of microbial proteins

Aim: This study was conducted to find the best suited freely available software for modelling of proteins by taking a few sample proteins. The proteins used were small to big in size with available crystal structures for the purpose of benchmarking. Key players like Phyre2, Swiss-Model, CPHmodels-3.0, Homer, (PS)2, (PS)²-V², Modweb were used for the comparison and model generation. Results: Benchmarking process was done for four proteins, Icl, InhA, and KatG of Mycobacterium tuberculosis and RpoB of Thermus Thermophilus to get the most suited software. Parameters compared during analysis gave relatively better values for Phyre2 and Swiss-Model. Conclusion: This comparative study gave the information that Phyre2 and Swiss-Model make good models of small and large proteins as compared to other screened software. Other software was also good but is often not very efficient in providing full-length and properly folded structure.     

Evaluating outcomes in planning: Indicators and reference values for Swiss landscapes

Evaluation of achievement of set targets is a necessary step in landscape planning in order to learn from the past, reassess implemented measures and enhance trust in public managers and institutions. Though it is commonly accepted that indicators play a major role in such evaluations, so far no accepted framework for evaluating planning outcomes exists. Furthermore, the selection of appropriate indicators and reference values to effectively assess conditions of landscapes and determine whether observed developments can be considered positive or negative remains challenging. Our study contributes to much-needed research on this topic with a proposed evaluation framework built on goals, indicators and reference values. We analyzed the landscape section of eight Swiss cantonal comprehensive plans to specifically address (1) whether currently tracked indicators suffice to evaluate landscape-planning goals; (2) what a minimal set of landscape indicators for regional planning might look like; and (3) how the ratified value approach could be operationalized to develop reference values for landscape indicators. All eight plans have a similar hierarchical goal system with six major landscape goals, up to 18 themes and 21–33 subordinate goals. The studied cantons track from 29 to 84 indicators. We found a considerable imbalance in the ratio between subordinate goals and indicators, with comparatively few indicators being tracked to assess visual and recreational landscape quality. Our proposed minimal indicator set is well balanced since it lists 5–7 indicators for each theme. The general procedure for modeling reference values is based on the assumption that the protection status of a landscape is a proxy for high societal appreciation of a place. Consequently, indicator values for these areas would reflect reference values (ratified values). We illustrate the procedure with the exemplary indicator impervious surface. The proposed indicators and maps are powerful tools for outcome evaluation and also facilitate benchmarking, i.e. interregional comparisons of landscape qualities, which could be very useful for landscape planning in Europe.     

Dataset size and composition impact the reliability of performance benchmarks for peptide-MHC binding predictions

Background

It is important to accurately determine the performance of peptide:MHC binding predictions, as this enables users to compare and choose between different prediction methods and provides estimates of the expected error rate. Two common approaches to determine prediction performance are cross-validation, in which all available data are iteratively split into training and testing data, and the use of blind sets generated separately from the data used to construct the predictive method. In the present study, we have compared cross-validated prediction performances generated on our last benchmark dataset from 2009 with prediction performances generated on data subsequently added to the Immune Epitope Database (IEDB) which served as a blind set.

Results

We found that cross-validated performances systematically overestimated performance on the blind set. This was found not to be due to the presence of similar peptides in the cross-validation dataset. Rather, we found that small size and low sequence/affinity diversity of either training or blind datasets were associated with large differences in cross-validated vs. blind prediction performances. We use these findings to derive quantitative rules of how large and diverse datasets need to be to provide generalizable performance estimates.

Conclusion

It has long been known that cross-validated prediction performance estimates often overestimate performance on independently generated blind set data. We here identify and quantify the specific factors contributing to this effect for MHC-I binding predictions. An increasing number of peptides for which MHC binding affinities are measured experimentally have been selected based on binding predictions and thus are less diverse than historic datasets sampling the entire sequence and affinity space, making them more difficult benchmark data sets. This has to be taken into account when comparing performance metrics between different benchmarks, and when deriving error estimates for predictions based on benchmark performance.

Electronic supplementary material

The online version of this article (doi:10.1186/1471-2105-15-241) contains supplementary material, which is available to authorized users.