Measuring the Outcome of Biomedical Research: A Systematic Literature Review

Background

There is an increasing need to evaluate the production and impact of medical research produced by institutions. Many indicators exist, yet we do not have enough information about their relevance. The objective of this systematic review was (1) to identify all the indicators that could be used to measure the output and outcome of medical research carried out in institutions and (2) enlist their methodology, use, positive and negative points.

Methodology

We have searched 3 databases (Pubmed, Scopus, Web of Science) using the following keywords: [Research outcome* OR research output* OR bibliometric* OR scientometric* OR scientific production] AND [indicator* OR index* OR evaluation OR metrics]. We included articles presenting, discussing or evaluating indicators measuring the scientific production of an institution. The search was conducted by two independent authors using a standardised data extraction form. For each indicator we extracted its definition, calculation, its rationale and its positive and negative points. In order to reduce bias, data extraction and analysis was performed by two independent authors.

Findings

We included 76 articles. A total of 57 indicators were identified. We have classified those indicators into 6 categories: 9 indicators of research activity, 24 indicators of scientific production and impact, 5 indicators of collaboration, 7 indicators of industrial production, 4 indicators of dissemination, 8 indicators of health service impact. The most widely discussed and described is the h-index with 31 articles discussing it.

Discussion

The majority of indicators found are bibliometric indicators of scientific production and impact. Several indicators have been developed to improve the h-index. This indicator has also inspired the creation of two indicators to measure industrial production and collaboration. Several articles propose indicators measuring research impact without detailing a methodology for calculating them. Many bibliometric indicators identified have been created but have not been used or further discussed.     

Gene silencing activity of siRNA polyplexes based on thiolated N,N,N-trimethylated chitosan

N,N,N-Trimethylated chitosan (TMC) is a biodegradable polymer emerging as a promising nonviral vector for nucleic acid and protein delivery. In the present study, we investigated whether the introduction of thiol groups in TMC enhances the extracellular stability of the complexes based on this polymer and promotes the intracellular release of siRNA. The gene silencing activity and the cellular cytotoxicity of polyplexes based on thiolated TMC were compared with those based on the nonthiolated counterpart and the regularly used lipidic transfection agent Lipofectamine. Incubation of H1299 human lung cancer cells expressing firefly luciferase with siRNA/thiolated TMC polyplexes resulted in 60-80% gene silencing activity, whereas complexes based on nonthiolated TMC showed less silencing (40%). The silencing activity of the complexes based on Lipofectamine 2000 was about 60-70%. Importantly, the TMC-SH polyplexes retained their silencing activity in the presence of hyaluronic acid, while nonthiolated TMC polyplexes hardly showed any silencing activity, demonstrating their stability against competing anionic macromolecules. Under the experimental conditions tested, the cytotoxicity of the thiolated and nonthiolated siRNA complexes was lower than those based on Lipofectamine. Given the good extracellular stability and good silencing activity, it is concluded that polyplexes based on TMC-SH are attractive systems for further in vivo evaluations.     

Ethacrynic acid analogues lacking the α,β-unsaturated carbonyl unit—Potential anti-metastatic drugs

A series of ethacrynic acid analogues, lacking the α,β-unsaturated carbonyl unit, was synthesized and subsequently evaluated for their ability to inhibit the migration of human breast cancer cells, MCF-7/AZ. Several of the analogues were already active in the low micromolar range, whereas ethacrynic acid itself shows no potential to inhibit the migration of these cancer cells. Preliminary studies show that the presence of one or more methoxy groups at the phenyl ring of ethacrynic acid is important in order for the ethacrynic acid analogues to demonstrate an inhibitory effect on the migration.     

Refuge from predation, the benefit of living in an extreme acidic environment?

Organisms living in extreme habitats require costly adaptations to cope with these conditions. Among the suggested potential benefits that trade off these costs is refuge from predation. To study these interactions in extreme environments, samples were taken in the cave Cueva de Villa Luz, Tabasco, Mexico, where more than 32 subterranean springs, some H(2)S rich, rise from the floor. Hydrogen sulfide gas plus oxygen is absorbed by freshwater, and oxidation forms concentrated sulfuric acid. Snottites, whitish hollow mucous tubes, hang from the ceiling of the cave. Fluid drops from these snottites were recorded as having pH values of 0-3. We report the discovery of a new species of nematode that thrives in the highly acidic environment of the snottite. Micro CT scan of snottites reveals a complex interaction between the acidic snottite, nematodes, and abundant nematode-eating mites. The nematode adaptation to low pH probably protects them against mite predation, for which nematodes are most likely the most important source of carbon in this sulfur-driven ecosystem.     

Plant–soil feedback of native and range-expanding plant species is insensitive to temperature

Temperature change affects many aboveground and belowground ecosystem processes. Here we investigate the effect of a 5°C temperature increase on plant–soil feedback. We compare plant species from a temperate climate region with immigrant plants that originate from warmer regions and have recently shifted their range polewards. We tested whether the magnitude of plant–soil feedback is affected by ambient temperature and whether the effect of temperature differs between these groups of plant species. Six European/Eurasian plant species that recently colonized the Netherlands (non-natives), and six related species (natives) from the Netherlands were selected. Plant–soil feedback of these species was determined by comparing performance in conspecific and heterospecific soils. In order to test the effect of temperature on these plant–soil feedback interactions, the experiments were performed at two greenhouse temperatures of 20/15°C and 25/20°C, respectively. Inoculation with unconditioned soil had the same effect on natives and non-natives. However, the effect of conspecific conditioned soil was negative compared to heterospecific soil for natives, but was positive for non-natives. In both cases, plant–soil interactions were not affected by temperature. Therefore, we conclude that the temperature component of climate change does not affect the direction, or strength of plant–soil feedback, neither for native nor for non-native plant species. However, as the non-natives have a more positive soil feedback than natives, climate warming may introduce new plant species in temperate regions that have less soil-borne control of abundance.     

Microbial metabolomics: past,present and future methodologies

Microbial metabolomics has received much attention in recent years mainly because it supports and complements a wide range of microbial research areas from new drug discovery efforts to metabolic engineering. Broadly, the term metabolomics refers to the comprehensive (qualitative and quantitative) analysis of the complete set of all low molecular weight metabolites present in and around growing cells at a given time during their growth or production cycle. This review focuses on the past, current and future development of various experimental protocols in the rapid developing area of metabolomics in the ongoing quest to reliably quantify microbial metabolites formed under defined physiological conditions. These developments range from rapid sample collection, instant quenching of microbial metabolic activity, extraction of the relevant intracellular metabolites as well as quantification of these metabolites using enzyme based and or modern high tech hyphenated analytical protocols, mainly chromatographic techniques coupled to mass spectrometry (LC-MSⁿ, GC-MSⁿ, CE-MSⁿ), where n indicates the number of tandem mass spectrometry, and nuclear magnetic resonance spectroscopy (NMR).     

On the chlorophyll a fluorescence yield in chloroplasts upon excitation with twin turnover flashes (TTF) and high frequency flash trains

Chlorophyll fluorescence is routinely taken as a quantifiable measure of the redox state of the primary quinone acceptor Q_A of PSII. The variable fluorescence in thylakoids increases in a single turnover flash (STF) from its low dark level F _o towards a maximum F _m^STF when Q_A becomes reduced. We found, using twin single turnover flashes (TTFs) that the fluorescence increase induced by the first twin-partner is followed by a 20–30% increase when the second partner is applied within 20–100 μs after the first one. The amplitude of the twin response shows a period-of-four oscillation associated with the 4-step oxidation of water in the Kok cycle (S states) and originates from two different trapped states with a life time of 0.2–0.4 and 2–5 ms, respectively. The oscillation is supplemented with a binary oscillation associated with the two-electron gate mechanism at the PSII acceptor side. The F(t) response in high frequency flash trains (1–4 kHz) shows (i) in the first 3–4 flashes a transient overshoot 20–30% above the F _m^STF = 3*F _o level reached in the 1st flash with a partial decline towards a dip D in the next 2–3 ms, independent of the flash frequency, and (ii) a frequency independent rise to F _m = 5*F _o in the 3–60 ms time range. The initial overshoot is interpreted to be due to electron trapping in the S₀ fraction with Q_B-nonreducing centers and the dip to the subsequent recovery accompanying the reoxidation of the double reduced acceptor pair in these RCs after trapping. The rise after the overshoot is, in agreement with earlier findings, interpreted to indicate a photo-electrochemical control of the chlorophyll fluorescence yield of PSII. It is anticipated that the double exciton and electron trapping property of PSII is advantageous for the plant. It serves to alleviate the depression of electron transport in single reduced Q_B-nonreducing RCs, associated with electrochemically coupled proton transport, by an increased electron trapping efficiency in these centers.     

Macroporous monolithic gels, cryogels, with immobilized phages from phage-display library as a new platform for fast development of affinity adsorbent capable of target capture from crude feeds

Selected phage clones expressing a peptide with high binding affinity for recombinant human lactoferrin or von Willebrand factor (vWF) were covalently coupled to macroporous poly(dimethylacrylamide) monolithic column. Large pore size (10-100 microm) of macroporous poly(dimethylacrylamide) makes it possible to couple long (1 microm) phage particles as ligands without any risk of blocking the monolithic column. The macroporous monolithic columns were successfully used for the direct affinity capture of target proteins from particulate containing feeds like milk containing casein micelles and fat globules (1-10 microm in size) or even whole blood containing blood cells (up to 20 microm in size). The newly developed platform based on selected bacteriophages immobilized within macropores of the monolithic cryogels presents a convenient alternative to antibodies for fast and selective development of the specific adsorbent.