Measuring the Outcome of Biomedical Research: A Systematic Literature Review

Background

There is an increasing need to evaluate the production and impact of medical research produced by institutions. Many indicators exist, yet we do not have enough information about their relevance. The objective of this systematic review was (1) to identify all the indicators that could be used to measure the output and outcome of medical research carried out in institutions and (2) enlist their methodology, use, positive and negative points.

Methodology

We have searched 3 databases (Pubmed, Scopus, Web of Science) using the following keywords: [Research outcome* OR research output* OR bibliometric* OR scientometric* OR scientific production] AND [indicator* OR index* OR evaluation OR metrics]. We included articles presenting, discussing or evaluating indicators measuring the scientific production of an institution. The search was conducted by two independent authors using a standardised data extraction form. For each indicator we extracted its definition, calculation, its rationale and its positive and negative points. In order to reduce bias, data extraction and analysis was performed by two independent authors.

Findings

We included 76 articles. A total of 57 indicators were identified. We have classified those indicators into 6 categories: 9 indicators of research activity, 24 indicators of scientific production and impact, 5 indicators of collaboration, 7 indicators of industrial production, 4 indicators of dissemination, 8 indicators of health service impact. The most widely discussed and described is the h-index with 31 articles discussing it.

Discussion

The majority of indicators found are bibliometric indicators of scientific production and impact. Several indicators have been developed to improve the h-index. This indicator has also inspired the creation of two indicators to measure industrial production and collaboration. Several articles propose indicators measuring research impact without detailing a methodology for calculating them. Many bibliometric indicators identified have been created but have not been used or further discussed.     

IL-4 from Th2-type cells suppresses induction of delayed-type hypersensitivity elicited shortly after immunization

The pure delayed-type hypersensitivity reaction obtained in 4-day ovalbumin-sensitized mice after antigen challenge in the footpad was abrogated by transfer of in vitro expanded, antigen-specific lymphoblasts derived from ovalbumin-hyperimmunized donors (high antibody producers), 12 h before immunization. This effect was specific inasmuch as Trypanosoma cruzi-specific blasts derived from Tc-Ag-hyperimmunized mice did not inhibit delayed-type hypersensitivity in ovalbumin-immunized recipients. The ovalbumin-specific blasts displayed a Th2 cytokine profile, secreting IL-4 and IL-10 upon restimulation in vitro with ovalbumin, but not IFN-gamma or IL-2. In addition, recipients of such cells produced much more IgG1 and IgE antibodies. When the frequency of T-cell blasts was enriched among these cells, transfer of four million cells was enough to prevent the induction of delayed-type hypersensitivity. Neutralization of IL-4 alone just before cell transfer not only restored the delayed-type hyper-sensitivity reaction, but also maintained it in a plateau for at least 72 h after challenge. Recipients treated in this way also showed a shift back towards a Th1 phenotype, indicated by the increase in IL-2, IFN-gamma and IL-12 synthesis. No synergistic action was observed when IL-4 and IL-10 were concomitantly neutralized. These results indicate that activation of Ag-specific Th2 cells early in the course of the immune response to a protein antigen provides an immunological environment rich in IL-4, thus leading to the inhibition of cell-mediated immunity.     

6-Dimethyl amino purine and 2-amino purine inhibit the induction of expresion of milk protein genes by prolactin

Two protein kinase-inhibitors, 6-dimethyl amino purine and 2-amino purine inhibited induction of -casein synthesis by prolactin when added to the culture medium of rabbit mammary explant and cells. The accumulation of the mRNA for _s1- and -caseins and for whey acidic protein did not take place in the presence of the inhibitors whereas -actin mRNA concentration was not altered. In the same experimental conditions, H7, an inhibitor of protein kinase C and, to a lower extent, of protein kinase A did not prevent prolactin from acting. These data suggest for the first time that specific protein kinases are involved in the transduction of the prolactin signal to milk protein genes.Abbreviations 6-DMAP 6-dimethyl amino purine - 2-AP 2-amino purine - H7 1-(5 isoquinoline sulfonyl)-2-methyl-piperazine dihydrochloride - WAP whey acidic protein - PRL prolactin     

Quantitative collection of milk and active recombinant proteins from the mammary glands of transgenic mice

Summary

Transgenic mice expressing foreign genes specifically in their mammary glands have been obtained by several groups in the world. The mouse is generally considered as a good reference animal to evaluate the efficiency of gene constructs to be used in larger mammals for the preparation of the corresponding recombinant proteins at an industrial scale. The method described here shows that mammary glands from lactating mice separated from their pups for one day spontaneously released 1.5 ml milk when stored at O'C. The proteins of milk obtained by this method were essentially similar to those obtained after milking. Human growth hormone (hGH) gene under the control of the rabbit whey acidic (WAP) gene promoter was expressed at a high level in the milk of transgenic mice (4 mg/ml milk in the mice examined here). hGH was present in milk obtained after milking or after the incubation of the mammary glands at O'C. In both cases, the hormone was present in essentially similar concentration, undegraded and biologically active (as judged by its prolactin‐like activity). The method depicted here is very simple and can be applied easily to many mice. Its major limitation is that it implies the breeding and the sacrifice of a relatively large number of animals. One gram of crude recombinant protein can be virtually obtained in this way with about 200 lactating mice from their milk containing the proteins at the concentration of 3‐4 mg/ml. The milk of transgenic mice can therefore be considered as a practical source of recombinant proteins for biochemical and pharmaceutical studies.     

IL-4 and IL-10 are essential for immunosuppression induced by high molecular weight proteins from Ascaris suum

The extract from Ascaris suum worms (Asc) impairs Th1 and Th2 responses to a non-related antigen, i.e. ovalbumin (OVA). Its suppressive capacity is due to high molecular weight components present in a gel filtration fraction (PI). This fraction is able to elicit IL-4 and IL-10 secretion. Interestingly enough, it induces anti-PI non-anaphylactic IgG1 synthesis through the action of IL-12/IFN-gamma. Here, we investigated the down-regulation of the immune response to OVA by PI in IL-12, IFN-gamma, IL-4 or IL-10 C57BL/6 knockout mice immunized with OVA+PI in adjuvant. OVA-induced delayed-type hypersensitivity (DTH) reactions, secretion of IL-2 and IFN-gamma, and IgG1, IgG2c and IgE antibody production were suppressed by PI in wild-type mice, as well as in IL-12- or IFN-gamma-deficient mice. In contrast, PI had no effect on anti-OVA IgE production and DTH, and induced only a partial suppression of IgG1 and IFN-gamma in IL-10(-/-) mice. The experiments also showed that IL-4 was involved in the PI-induced suppression of IgG2c antibodies and IL-2 secretion. Finally, down-regulation of IFN-gamma was not seen in mice lacking both IL-4 and IL-10, i.e. IL-4(-/-) mice treated with anti-IL-10 antibodies before immunization. These results exclude the participation of IL-12 and IFN-gamma in PI-induced immunosuppression, and highlight the essential role of IL-10 in the suppression of OVA-specific Th2-related parameters, as well as the cooperation between IL-10 and IL-4 in the suppression of Th1-related parameters.     

PAS-1, a protein affinity purified from Ascaris suum worms, maintains the ability to modulate the immune response to a bystander antigen

Helminth infections and parasite components have potent immunomodulatory effects on a host's immune system. In the present study, we investigated the effect of PAS-1, a protein component of Ascaris suum adult worms recognized by a monoclonal antibody (MAIP-1), on humoral and cell-mediated responses to a bystander antigen (ovalbumin [OVA]). MAIP-1 recognized only one of the three polypeptide chains of PAS-1, but neutralized the suppressive effect of the whole worm extract on OVA-specific antibody production. PAS-1 inhibited antibody production against a T-cell-dependent, but not a T-cell-independent, antigen in a dose-dependent way. IgM, IgG1, IgG2b, and also IgE and anaphylactic IgG1 levels were downregulated. In addition, PAS-1 inhibited OVA-specific delayed type hypersensitivity reactions in the footpad of mice, showing a potent immunosuppressive activity on both Th1 and Th2 responses that seems to be mediated by the induction of large amounts of IL-10 and IL-4. Indeed, PAS-1-specific spleen cells secreted sevenfold more IL-10 and threefold more IL-4 than OVA-specific cells in response to in vitro restimulation with the respective antigens. In conclusion, we showed that PAS-1, a single protein component from A. suum, maintains all its immunosuppressive properties.     

Modulation of the induction of lung and airway allergy in the offspring of IFN-gamma-treated mother mice

Recent studies have highlighted the influence of fetal/maternal interactions on the development of asthma. Because IFN-gamma reduces Th2-mediated allergic responses, we assessed its capacity to modulate asthma in the offspring when injected into mothers during pregnancy. IFN-gamma was injected in CD1 female mice on day 6.5 of gestation. Immediately after birth, male newborns were housed in cages with interchanged mothers: the offspring from IFN-gamma-treated mothers were breastfed by normal mothers (IFN/nor), and those from normal mothers were breastfed by IFN-gamma-treated (Nor/IFN) or normal mothers (Nor/nor). Immediately after weaning, the spleen cells from IFN/nor and Nor/IFN mice produced less IL-4 and more IFN-gamma than Nor/nor mice when stimulated with Con A. At the age of 6-7 wk, mice were immunized with OVA on days 0 and 7. From day 14 to 16, they were exposed to aerosolized OVA. The bronchoalveolar lavage fluid from Nor/nor mice showed eosinophilia, a large number of these cells being present in perivascular and peribronchial regions of lung tissues. IFN/nor or Nor/IFN mice showed greatly reduced eosinophil numbers in bronchoalveolar lavage fluid. In addition, lung sections from IFN/nor, but not Nor/IFN mice showed almost normal histology. In OVA-sensitized IFN/nor and Nor/IFN mice, the production of IFN-gamma, IL-4, and IL-5 by spleen cells was significantly reduced as compared with cells from the OVA-sensitized Nor/nor group. IgE and anaphylactic IgG1 were also reduced in plasma of IFN/nor mice. In conclusion, the presence of IFN-gamma during pregnancy confers to the fetus a protection against allergenic provocations in the adult life.     

Biochemical,antigenic and allergenic characterization of crude extracts ofDrechslera (Helminthosporium) monoceras

In a previous study with airborne mould extracts we verified thatDrechslera (Helminthosporium) monoceras presented stronger reactions than those presented by 42 other moulds isolated in São Paulo city. In the present study, we evaluated the biochemical composition and the antigenicity of crude extracts obtained from vegetative and conidial stage ofD. monoceras using Czapeck broth (CB) modified and tris-HCl for extraction. The maximum values of total proteins and lipids were verified in the crude extract obtained in the 28th day of growth, and maximum values of carbohydrates were observed in the extracts of the 16th, 22nd and 26th days. The fractionated proteins by SDS-PAGE presented bands with molecular weights between 14.4 to 67 Kd; the 28th day extract showed a larger number of bands. The carbohydrates and amino acids were characterized by thin-layer chromatography. The antigenicity of the crude extracts was verified by immunodiffusion reaction in agar against rabbit hyperimmune sera. Precipitation lines were observed in all studied extracts and common antigenic molecular populations. Based on the above results, the 28th day extract was selected to verify the induction of IgE antibody responses in immunizations of Balb/c and cAF-1 mice, and titer by passive cutaneous anaphylaxis test using Wistar rats. The maximum titers obtained were 160 in cAF-1 mice and 1.280 in Balb/c mice. The results suggest that the 28th day extract contains allergenic fractions and should be chosen for future studies related to fractionation, characterization and standardization in diagnostic methods and immunotherapy.Part of the thesis of E.A. Menezes to get degree of Doctor Fractionation and characterization of allergenic extract ofDrechslera (Helminthosporium) monoceras.     

Bacterial lipopolysaccharide signaling through Toll-like receptor 4 suppresses asthma-like responses via nitric oxide synthase 2 activity

Asthma results from an intrapulmonary allergen-driven Th2 response and is characterized by intermittent airway obstruction, airway hyperreactivity, and airway inflammation. An inverse association between allergic asthma and microbial infections has been observed. Microbial infections could prevent allergic responses by inducing the secretion of the type 1 cytokines, IL-12 and IFN-gamma. In this study, we examined whether administration of bacterial LPS, a prototypic bacterial product that activates innate immune cells via the Toll-like receptor 4 (TLR4) could suppress early and late allergic responses in a murine model of asthma. We report that LPS administration suppresses the IgE-mediated and mast cell-dependent passive cutaneous anaphylaxis, pulmonary inflammation, airway eosinophilia, mucus production, and airway hyperactivity. The suppression of asthma-like responses was not due to Th1 shift as it persisted in IL-12(-/-) or IFN-gamma(-/-) mice. However, the suppressive effect of LPS was not observed in TLR4- or NO synthase 2-deficient mice. Our findings demonstrate, for the first time, that LPS suppresses Th2 responses in vivo via the TLR4-dependent pathway that triggers NO synthase 2 activity.