The Assessment of Science: The Relative Merits of Post-Publication Review,the Impact Factor,and the Number of Citations

The assessment of scientific publications is an integral part of the scientific process. Here we investigate three methods of assessing the merit of a scientific paper: subjective post-publication peer review, the number of citations gained by a paper, and the impact factor of the journal in which the article was published. We investigate these methods using two datasets in which subjective post-publication assessments of scientific publications have been made by experts. We find that there are moderate, but statistically significant, correlations between assessor scores, when two assessors have rated the same paper, and between assessor score and the number of citations a paper accrues. However, we show that assessor score depends strongly on the journal in which the paper is published, and that assessors tend to over-rate papers published in journals with high impact factors. If we control for this bias, we find that the correlation between assessor scores and between assessor score and the number of citations is weak, suggesting that scientists have little ability to judge either the intrinsic merit of a paper or its likely impact. We also show that the number of citations a paper receives is an extremely error-prone measure of scientific merit. Finally, we argue that the impact factor is likely to be a poor measure of merit, since it depends on subjective assessment. We conclude that the three measures of scientific merit considered here are poor; in particular subjective assessments are an error-prone, biased, and expensive method by which to assess merit. We argue that the impact factor may be the most satisfactory of the methods we have considered, since it is a form of pre-publication review. However, we emphasise that it is likely to be a very error-prone measure of merit that is qualitative, not quantitative.

Author summary

Subjective assessments of the merit and likely impact of scientific publications are routinely made by scientists during their own research, and as part of promotion, appointment, and government committees. Using two large datasets in which scientists have made qualitative assessments of scientific merit, we show that scientists are poor at judging scientific merit and the likely impact of a paper, and that their judgment is strongly influenced by the journal in which the paper is published. We also demonstrate that the number of citations a paper accumulates is a poor measure of merit and we argue that although it is likely to be poor, the impact factor, of the journal in which a paper is published, may be the best measure of scientific merit currently available.     

Intake of Farmed Atlantic Salmon Fed Soybean Oil Increases Insulin Resistance and Hepatic Lipid Accumulation in Mice

Background

To ensure sustainable aquaculture, fish derived raw materials are replaced by vegetable ingredients. Fatty acid composition and contaminant status of farmed Atlantic salmon (Salmo salar L.) are affected by the use of plant ingredients and a spillover effect on consumers is thus expected. Here we aimed to compare the effects of intake of Atlantic salmon fed fish oil (FO) with intake of Atlantic salmon fed a high proportion of vegetable oils (VOs) on development of insulin resistance and obesity in mice.

Methodology/principal findings

Atlantic salmon were fed diets where FO was partly (80%) replaced with three different VOs; rapeseed oil (RO), olive oil (OO) or soy bean oil (SO). Fillets from Atlantic salmon were subsequently used to prepare Western diets (WD) for a mouse feeding trial. Partial replacement of FO with VOs reduced the levels of polychlorinated biphenyls (PCB) and dichloro-diphenyl-tricloroethanes (DDT) with more than 50% in salmon fillets, in WDs containing the fillets, and in white adipose tissue from mice consuming the WDs. Replacement with VOs, SO in particular, lowered the n−3 polyunsaturated fatty acid (PUFA) content and increased n−6 PUFA levels in the salmon fillets, in the prepared WDs, and in red blood cells collected from mice consuming the WDs. Replacing FO with VO did not influence obesity development in the mice, but replacement of FO with RO improved glucose tolerance. Compared with WD-FO fed mice, feeding mice WD-SO containing lower PCB and DDT levels but high levels of linoleic acid (LA), exaggerated insulin resistance and increased accumulation of fat in the liver.

Conclusion/Significance

Replacement of FO with VOs in aqua feed for farmed salmon had markedly different spillover effects on metabolism in mice. Our results suggest that the content of LA in VOs may be a matter of concern that warrants further investigation.     

Plant and Fungal Diversity in Gut Microbiota as Revealed by Molecular and Culture Investigations

Background

Few studies describing eukaryotic communities in the human gut microbiota have been published. The objective of this study was to investigate comprehensively the repertoire of plant and fungal species in the gut microbiota of an obese patient.

Methodology/Principal Findings

A stool specimen was collected from a 27-year-old Caucasian woman with a body mass index of 48.9 who was living in Marseille, France. Plant and fungal species were identified using a PCR-based method incorporating 25 primer pairs specific for each eukaryotic phylum and universal eukaryotic primers targeting 18S rRNA, internal transcribed spacer (ITS) and a chloroplast gene. The PCR products amplified using these primers were cloned and sequenced. Three different culture media were used to isolate fungi, and these cultured fungi were further identified by ITS sequencing. A total of 37 eukaryotic species were identified, including a Diatoms (Blastocystis sp.) species, 18 plant species from the Streptophyta phylum and 18 fungal species from the Ascomycota, Basidiomycota and Chytridiocomycota phyla. Cultures yielded 16 fungal species, while PCR-sequencing identified 7 fungal species. Of these 7 species of fungi, 5 were also identified by culture. Twenty-one eukaryotic species were discovered for the first time in human gut microbiota, including 8 fungi (Aspergillus flavipes, Beauveria bassiana, Isaria farinosa, Penicillium brevicompactum, Penicillium dipodomyicola, Penicillium camemberti, Climacocystis sp. and Malassezia restricta). Many fungal species apparently originated from food, as did 11 plant species. However, four plant species (Atractylodes japonica, Fibraurea tinctoria, Angelica anomala, Mitella nuda) are used as medicinal plants.

Conclusions/Significance

Investigating the eukaryotic components of gut microbiota may help us to understand their role in human health.     

Global and 3D Spatial Assessment of Neuroinflammation in Rodent Models of Multiple Sclerosis

Multiple Sclerosis (MS) is a progressive autoimmune inflammatory and demyelinating disease of the central nervous system (CNS). T cells play a key role in the progression of neuroinflammation in MS and also in the experimental autoimmune encephalomyelitis (EAE) animal models for the disease. A technology for quantitative and 3 dimensional (3D) spatial assessment of inflammation in this and other CNS inflammatory conditions is much needed. Here we present a procedure for 3D spatial assessment and global quantification of the development of neuroinflammation based on Optical Projection Tomography (OPT). Applying this approach to the analysis of rodent models of MS, we provide global quantitative data of the major inflammatory component as a function of the clinical course. Our data demonstrates a strong correlation between the development and progression of neuroinflammation and clinical disease in several mouse and a rat model of MS refining the information regarding the spatial dynamics of the inflammatory component in EAE. This method provides a powerful tool to investigate the effect of environmental and genetic forces and for assessing the therapeutic effects of drug therapy in animal models of MS and other neuroinflammatory/neurodegenerative disorders.     

Overnight Resting of PBMC Changes Functional Signatures of Antigen Specific T- Cell Responses: Impact for Immune Monitoring within Clinical Trials

Polyfunctional CD4 or CD8 T cells are proposed to represent a correlate of immune control for persistent viruses as well as for vaccine mediated protection against infection. A well-suited methodology to study complex functional phenotypes of antiviral T cells is the combined staining of intracellular cytokines and phenotypic marker expression using polychromatic flow cytometry. In this study we analyzed the effect of an overnight resting period at 37°C on the quantity and functionality of HIV-1, EBV, CMV, HBV and HCV specific CD4 and CD8 T-cell responses in a cohort of 21 individuals. We quantified total antigen specific T cells by multimer staining and used 10-color intracellular cytokine staining (ICS) to determine IFNγ, TNFα, IL2 and MIP1β production. After an overnight resting significantly higher numbers of functionally active T cells were detectable by ICS for all tested antigen specificities, whereas the total number of antigen specific T cells determined by multimer staining remained unchanged. Overnight resting shifted the quality of T-cell responses towards polyfunctionality and increased antigen sensitivity of T cells. Our data suggest that the observed effect is mediated by T cells rather than by antigen presenting cells. We conclude that overnight resting of PBMC prior to ex vivo analysis of antiviral T-cell responses represents an efficient method to increase sensitivity of ICS-based methods and has a prominent impact on the functional phenotype of T cells.     

Selection of Osteoprogenitors from the Jaw Periosteum by a Specific Animal-Free Culture Medium

The goal of our research work is to establish mesenchymal osteoprogenitors derived from human jaw periosteum for tissue engineering applications in oral and maxillofacial surgery. For future autologous and/or allogeneic transplantations, some issues must be addressed. On the one hand, animal-free culture conditions have yet to be established. On the other hand, attempts should be undertaken to shorten the in vitro culturing process efficiently. The aim of the present study is to compare and analyze the phenotype of osteoprogenitors from the jaw periosteum under normal FCS-containing and animal-free culture conditions. Therefore, we analyzed the proliferation rates of MesenCult-XF medium (MC-) in comparison to DMEM-cultured JPCs. Whereas jaw periosteal cells (JPCs) show relatively slow proliferation rates and a fibroblastoid shape under DMEM culture conditions, MC-cultured JPCs diminished their cell size significantly and proliferated rapidly. By live-monitoring measurements of adhesion and proliferation, we made an interesting observation: whereas the proliferation of the MSCA-1⁺ subpopulation and the unseparated cell fraction were favored by the animal-free culture medium, the proliferation of the MSCA-1^- subpopulation seemed to be repressed under these conditions. The alkaline phosphatase expression pattern showed similar results under both culture conditions. Comparison of the mineralization capacity revealed an earlier formation of calcium-phosphate precipitates under MC culture conditions; however, the mineralization capacity of the DMEM-cultured cells seemed to be higher. We conclude that the tested animal-free medium is suitable for the in vitro expansion and even for the specific selection of osteoprogenitor cells derived from the jaw periosteum.