Measuring the Outcome of Biomedical Research: A Systematic Literature Review

Background

There is an increasing need to evaluate the production and impact of medical research produced by institutions. Many indicators exist, yet we do not have enough information about their relevance. The objective of this systematic review was (1) to identify all the indicators that could be used to measure the output and outcome of medical research carried out in institutions and (2) enlist their methodology, use, positive and negative points.

Methodology

We have searched 3 databases (Pubmed, Scopus, Web of Science) using the following keywords: [Research outcome* OR research output* OR bibliometric* OR scientometric* OR scientific production] AND [indicator* OR index* OR evaluation OR metrics]. We included articles presenting, discussing or evaluating indicators measuring the scientific production of an institution. The search was conducted by two independent authors using a standardised data extraction form. For each indicator we extracted its definition, calculation, its rationale and its positive and negative points. In order to reduce bias, data extraction and analysis was performed by two independent authors.

Findings

We included 76 articles. A total of 57 indicators were identified. We have classified those indicators into 6 categories: 9 indicators of research activity, 24 indicators of scientific production and impact, 5 indicators of collaboration, 7 indicators of industrial production, 4 indicators of dissemination, 8 indicators of health service impact. The most widely discussed and described is the h-index with 31 articles discussing it.

Discussion

The majority of indicators found are bibliometric indicators of scientific production and impact. Several indicators have been developed to improve the h-index. This indicator has also inspired the creation of two indicators to measure industrial production and collaboration. Several articles propose indicators measuring research impact without detailing a methodology for calculating them. Many bibliometric indicators identified have been created but have not been used or further discussed.     

Coupling between taxonomic and functional diversity in protistan coastal communities

The study of protistan functional diversity is crucial to understand the dynamics of oceanic ecological processes. We combined the metabarcoding data of various coastal ecosystems and a newly developed trait-based approach to study the link between taxonomic and functional diversity across marine protistan communities of different size-classes. Environmental DNA was extracted and the V4 18S rDNA genomic region was amplified and sequenced. In parallel, we tried to annotate the operational taxonomic units (OTUs) from our metabarcoding dataset to 30 biological traits using published and accessible information on protists. We then developed a method to study trait correlations across protists (i.e. trade-offs) in order to build the best functional groups. Based on the annotated OTUs and our functional groups, we demonstrated that the functional diversity of marine protist communities varied in parallel with their taxonomic diversity. The coupling between functional and taxonomic diversity was conserved across different protist size classes. However, the smallest size-fraction was characterized by wider taxonomic and functional groups diversity, corroborating the idea that nanoplankton and picoplankton are part of a more stable ecological background on which larger protists and metazoans might develop.     

Virulence factors of the human opportunistic pathogen Serratia marcescens identified by in vivo screening

The human opportunistic pathogen Serratia marcescens is a bacterium with a broad host range, and represents a growing problem for public health. Serratia marcescens kills Caenorhabditis elegans after colonizing the nematode's intestine. We used C.elegans to screen a bank of transposon-induced S.marcescens mutants and isolated 23 clones with an attenuated virulence. Nine of the selected bacterial clones also showed a reduced virulence in an insect model of infection. Of these, three exhibited a reduced cytotoxicity in vitro, and among them one was also markedly attenuated in its virulence in a murine lung infection model. For 21 of the 23 mutants, the transposon insertion site was identified. This revealed that among the genes necessary for full in vivo virulence are those that function in lipopolysaccharide (LPS) biosynthesis, iron uptake and hemolysin production. Using this system we also identified novel conserved virulence factors required for Pseudomonas aeruginosa pathogenicity. This study extends the utility of C.elegans as an in vivo model for the study of bacterial virulence and advances the molecular understanding of S.marcescens pathogenicity.     

Characterization of a radish introgression carrying the Ogura fertility restorer gene Rfo in rapeseed,using the Arabidopsis genome sequence and radish genetic mapping

The radish Rfo gene restores male fertility in radish or rapeseed plants carrying Ogura cytoplasmic male-sterility. This system was first discovered in radish and was transferred to rapeseed for the production of F1 hybrid seeds. We aimed to identify the region of the Arabidopsis genome syntenic to the Rfo locus and to characterize the radish introgression in restored rapeseed. We used two methods: amplified consensus genetic markers (ACGMs) in restored rapeseed plants and construction of a precise genetic map around the Rfo gene in a segregating radish population. The use of ACGMs made it possible to detect radish orthologs of Arabidopsis genes in the restored rapeseed genome. We identified radish genes, linked to Rfo in rapeseed and whose orthologs in Arabidopsis are carried by chromosomes 1, 4 and 5. This indicates several breaks in colinearity between radish and Arabidopsis genomes in this region. We determined the positions of markers relative to each other and to the Rfo gene, using the progeny of a rapeseed plant with unstable meiotic transmission of the radish introgression. This enabled us to produce a schematic diagram of the radish introgression in rapeseed. Markers which could be mapped both on radish and restored rapeseed indicate that at least 50 cM of the radish genome is integrated in restored rapeseed. Using markers closely linked to the Rfo gene in rapeseed and radish, we identified a contig spanning six bacterial artificial chromosome (BAC) clones on Arabidopsis chromosome 1, which is likely to carry the orthologous Rfo gene.Electronic Supplementary Material Supplementary material is available in the online version of this article at Communicated by H. C. BeckerS. Giancola and S. Marhadour contributed equally to this work     

Identification and characterization of adhesive factors of Clostridium difficile involved in adhesion to human colonic enterocyte-like Caco-2 and mucus-secreting HT29 cells in culture

Experiments reported in this communication showed that the highly toxinogenic Cd 79685, Cd 4784, and Wilkins Clostridium difficile strains and the moderately toxinogenic FD strain grown in the presence of blood adhere to polarized monolayers of two cultured human intestinal cell lines: the human colonic epithelial Caco-2 cells and the human mucus-secreting HT29-MTX cells. Scanning electron microscopy revealed that the bacteria interacted with well-defined apical microvilli of differentiated Caco-2 cells and that the bacteria strongly bind to the mucus layer that entirely covers the surface of the HT29-MTX cells. The binding of C. difficile to Caco-2 cells developed in parallel with the differentiation features of the Caco-2 cells, suggesting that the protein(s) which constitute C. difficile-binding sites are differentiation-related brush border protein(s). To better define this interaction, we tentatively characterized the mechanism(s) of adhesion of C. difficile with adherence assays. It was shown that heating of C. difficile grown in the presence of blood enhanced the bacterial interaction with the brush border of the enterocyte-like Caco-2 cells and the human mucus-secreting HT29-MTX cells. A labile surface-associated component was involved in C. difficile adhesion since washes of C. difficile grown in the presence of blood without heat shock decreased adhesion. After heating, washes of C. difficile grown in the presence of blood did not modify adhesion. Analysis of surface-associated proteins of C. difficile subjected to different culture conditions was con-ducted. After growth of C. difficile Cd 79685, Cd 4784, FD and Wilkins strains in the presence of blood and heating, two predominant SDS-extractable proteins with molecular masses of 12 and 27 kDa were observed and two other proteins with masses of 48 and 31 kDa disappeared. Direct involvement of the 12 and 27 kDa surface-associated proteins in the adhe-sion of C. difficile strains was demonstrated by using rat polycolonal antibodies pAb 12 and pAb 27 directed against the 12 and 27kDa proteins. Indeed, adhesion to Caco-2 cell monoiayers of C. difficiie strains grown in the presence of blood, without or with heat-shock, was blocked. Taken together, our results suggest that C. difficiie may utilize blood components as adhesins to adhere to human intestinal cultured cells.     

Effect of enhanced UV-B radiation on pollen quantity,quality, and seed yield in Brassica rapa (Brassicaceae)

We conducted three experiments to examine the influence of ultraviolet-B radiation (UV-B; 280–320 nm) exposure on reproduction in Brassica rapa (Brassicaceae). Plants were grown in a greenhouse under three biologically effective UV-B levels that simulated either an ambient stratospheric ozone level (control), 16% (“low enhanced”), or 32% (“high enhanced”) ozone depletion levels at Morgantown, WV, USA in mid-March. In the first experiment, we examined whether UV-B level during plant growth influenced in vivo pollen production and viability, and flower production. Pollen production and viability per flower were reduced by ≈50% under both enhanced UV-B levels relative to ambient controls. While plants under high-enhanced UV-B produced over 40% more flowers than plants under the two lower UV-B treatments, whole-plant production of viable pollen was reduced under high-enhanced UV-B to 17% of that of ambient controls. Whole-plant production of viable pollen was reduced under low-enhanced UV-B to 34% of ambient controls. In the second experiment, we collected pollen from plants under the three UV-B levels and examined whether source-plant UV-B exposure influenced in vitro pollen germination and viability. Pollen from plants under both enhanced-UV-B treatments had initially lower germination and viability than pollen from the ambient level. After in vitro exposure to the high-enhanced UV-B levels for 6 h, viability of the pollen from plants grown under ambient UV-B was reduced from 65 to 18%. In contrast, viability of the pollen from plants grown under both enhanced UV-B treatments was reduced to a much lesser extent: only from ≈43 to 22%. Thus, ambient source-plant pollen was more sensitive to enhanced UV-B exposure. In the third experiment, we used pollen collected from source plants under the three UV-B levels to fertilize plants growing under ambient-UV-B levels, and assessed subsequent seed production and germination. Seed abortion rates were higher in plants pollinated with pollen from the enhanced UV-B treatments, than from ambient UV-B. Despite this, seed yield (number and mass) per plant was similar, regardless of the UV-B exposure of their pollen source. Our findings demonstrate that enhanced UV-B levels associated with springtime ozone depletion events have the capacity to substantially reduce viable pollen production, and could ultimately reduce reproductive success of B. rapa.     

A sensitive fluorigenic substrate for selective in vitro and in vivo assay of leukotriene A4 hydrolase activity

Leukotriene A4 hydrolase (LTA4H) is a bifunctional zinc-dependent metalloprotease bearing both an epoxide hydrolase, producing the pro-inflammatory LTB4 leukotriene, and an aminopeptidase activity, whose physiological relevance has long been ignored. Distinct substrates are commonly used for each activity, although none is completely satisfactory; LTA4, substrate for the hydrolase activity, is unstable and inactivates the enzyme, whereas aminoacids β-naphthylamide and para-nitroanilide, used as aminopeptidase substrates, are poor and nonselective. Based on the three-dimensional structure of LTA4H, we describe a new, specific, and high-affinity fluorigenic substrate, PL553 [l-(4-benzoyl)phenylalanyl-β-naphthylamide], with both in vitro and in vivo applications. PL553 possesses a catalytic efficiency (k_cat/K_m) of 3.8 ± 0.5 × 10⁴ M⁻¹ s⁻¹ using human recombinant LTA4H and a limit of detection and quantification of less than 1 to 2 ng. The PL553 assay was validated by measuring the inhibitory potency of known LTA4H inhibitors and used to characterize new specific amino-phosphinic inhibitors. The LTA4H inhibition measured with PL553 in mouse tissues, after intravenous administration of inhibitors, was also correlated with a reduction in LTB4 levels. This authenticates the assay as the first allowing the easy measurement of endogenous LTA4H activity and in vitro specific screening of new LTA4H inhibitors.     

Quantitative IR microscopy and spectromics open the way to 3D digital pathology

Currently, only mass‐spectrometry (MS) microscopy brings a quantitative analysis of chemical contents of tissue samples in 3D. Here, the reconstruction of a 3D quantitative chemical images of a biological tissue by FTIR spectro‐microscopy is reported. An automated curve‐fitting method is developed to extract all intense absorption bands constituting IR spectra. This innovation benefits from three critical features: (1) the correction of raw IR spectra to make them quantitatively comparable; (2) the automated and iterative data treatment allowing to transfer the IR‐absorption spectrum into a IR‐band spectrum; (3) the reconstruction of an 3D IR‐band matrix (x, y, z for voxel position and a 4^th dimension with all IR‐band parameters). Spectromics, which is a new method for exploiting spectral data for tissue metadata reconstruction, is proposed to further translate the related chemical information in 3D, as biochemical and anatomical tissue parameters. An example is given with oxidative stress distribution and the reconstruction of blood vessels in tissues. The requirements of IR microscopy instrumentation to propose 3D digital histology as a clinical routine technology is briefly discussed.

     

Elevated concentrations of naturally occurring heavy metals inversely correlate with reproductive output and body mass of the Kagu Rhynochetos jubatus

To assess the effects of naturally occurring heavy metals on wild birds, we compared reproductive success and heavy metal contents in feathers of Kagu Rhynochetos jubatus living on ultramafic (rich in heavy metals) soil with those of Kagu living on non‐ultramafic soil. From 2003 to 2016, we monitored breeding of 19 Kagu families by radiotracking and video‐monitoring, and collected rump down feathers from 69 wild Kagu. The metal concentrations in Kagu feathers correlated with the concentrations in the soil. The mean numbers of eggs laid and fledglings per year of Kagu families on non‐ultramafic soil were about four times higher, and home‐ranges three times smaller, than those of Kagu on ultramafic soil. Mass of eggs and the proportion of eggs that developed to fledglings were similar in the two areas, whereas the mass of adult Kagu on non‐ultramafic soil was nearly 10% higher than that of adult Kagu living on ultramafic soil. The impact of naturally occurring heavy metals on Kagu breeding productivity and body mass appears to act through their effects on food supply rather than being caused directly by metal toxicity. The results imply that conservation of Kagu might be more effective in non‐ultramafic areas, as populations can recover much faster on these soils and Kagu can then recolonize and bolster populations in ultramafic areas.