Two strikes: limited NIH R55 and R56 retooling funds and abolishment of the A2 grant mechanism

The U.S. National Institutes of Health (NIH) are facing significant budgetary challenges as a result of the current economic climate. The recent sunset of investigator-initiated R01-type research grants after one revised submission, coupled with the present lack of an NIH retooling funding mechanism for such grant applicants, creates a concerning risk that talented and well-trained investigators may be forced to give up their research careers. Existing NIH retooling mechanisms include the R55 Shannon Award, which was established in 1991 and was essentially replaced in 2005 by the R56 award. There is an urgent need to either significantly expand the R55/R56 mechanisms and definition of NIH grant bridging/retooling support for unfunded meritorious proposals or introduce a new mechanism that provides specific support to investigators with competitive but unfunded R01 revised grants. An expanded retooling funding mechanism deserves implementation during continuing assessment of whether allowance of only one revision of research proposals has achieved its initial intended goals.     

To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits

We surveyed 113 astronomers and 82 psychologists active in applying for federally funded research on their grant-writing history between January, 2009 and November, 2012. We collected demographic data, effort levels, success rates, and perceived non-financial benefits from writing grant proposals. We find that the average proposal takes 116 PI hours and 55 CI hours to write; although time spent writing was not related to whether the grant was funded. Effort did translate into success, however, as academics who wrote more grants received more funding. Participants indicated modest non-monetary benefits from grant writing, with psychologists reporting a somewhat greater benefit overall than astronomers. These perceptions of non-financial benefits were unrelated to how many grants investigators applied for, the number of grants they received, or the amount of time they devoted to writing their proposals. We also explored the number of years an investigator can afford to apply unsuccessfully for research grants and our analyses suggest that funding rates below approximately 20%, commensurate with current NIH and NSF funding, are likely to drive at least half of the active researchers away from federally funded research. We conclude with recommendations and suggestions for individual investigators and for department heads.     

Enhancing NIH grant peer review: a broader perspective

Over the next couple of years, NIH will be revising its process of reviewing grant applications. The planned changes will make the NIH system more similar in some ways to those of European funding agencies, while retaining many unique features.     

Examining the Predictive Validity of NIH Peer Review Scores

The predictive validity of peer review at the National Institutes of Health (NIH) has not yet been demonstrated empirically. It might be assumed that the most efficient and expedient test of the predictive validity of NIH peer review would be an examination of the correlation between percentile scores from peer review and bibliometric indices of the publications produced from funded projects. The present study used a large dataset to examine the rationale for such a study, to determine if it would satisfy the requirements for a test of predictive validity. The results show significant restriction of range in the applications selected for funding. Furthermore, those few applications that are funded with slightly worse peer review scores are not selected at random or representative of other applications in the same range. The funding institutes also negotiate with applicants to address issues identified during peer review. Therefore, the peer review scores assigned to the submitted applications, especially for those few funded applications with slightly worse peer review scores, do not reflect the changed and improved projects that are eventually funded. In addition, citation metrics by themselves are not valid or appropriate measures of scientific impact. The use of bibliometric indices on their own to measure scientific impact would likely increase the inefficiencies and problems with replicability already largely attributed to the current over-emphasis on bibliometric indices. Therefore, retrospective analyses of the correlation between percentile scores from peer review and bibliometric indices of the publications resulting from funded grant applications are not valid tests of the predictive validity of peer review at the NIH.     

Facing the credit crunch. Politics sends mixed messages for science in the wake of the global financial crisis

The global response to the credit crunch has varied from belt tightening to spending sprees. Philip Hunter investigates how various countries react to the financial crisis in terms of supporting scientific research.The overall state of biomedical research in the wake of the global financial crisis remains unclear amid growing concern that competition for science funding is compromising the pursuit of research. Such concerns pre-date the credit crunch, but there is a feeling that an increasing amount of time and energy is being wasted in the ongoing scramble for grants, in the face of mounting pressure from funding agencies demanding value for money. Another problem is balancing funding between different fields; while the biomedical sciences have generally fared well, they are increasingly dependent on basic research in physics and chemistry that are in greater jeopardy. This has led to calls for rebalancing funding, in order to ensure the long-term viability of all fields in an increasingly multidisciplinary and collaborative research world.For countries that are cutting funding—such as Spain, Italy and the UK—the immediate priority is to preserve the fundamental research base and avoid a significant drain of expertise, either to rival countries or away from science altogether. This has highlighted the plight of postdoctoral researchers who have traditionally been the first to suffer from funding cuts, partly because they have little immediate impact on on a country''s scientific competitiveness. Postdocs have been the first to go whenever budgets have been cut, according to Richard Frankel, a physicist at California Polytechnic State University in Saint Luis Obispo, who investigates magnetotaxis in bacteria. “In the short term there will be little effect but the long-term effects can be devastating,” he said.…there is a feeling that an increasing amount of time and energy is being wasted in the ongoing scramble for grants, in the face of mounting pressure from funding agencies…According to Peter Stadler, head of a bioinformatics group at the University of Leipzig in Germany, such cuts tend to cause the long-term erosion of a country''s science skills base. “Short-term cuts in science funding translate totally into a brain drain, since they predominantly affect young researchers who are paid from the soft money that is drying up first,” said Stadler. “They either leave science, an irreversible step, or move abroad but do not come back later, because the medium-term effect of cuts is a reduction in career opportunities and fiercer competition giving those already in the system a big advantage.”Even when young researchers are not directly affected, the prevailing culture of short-term funding—which requires ongoing grant applications—can be disruptive, according to Xavier Salvatella, principal investigator in the Laboratory of Molecular Biophysics at the Institute for Research in Biomedicine in Barcelona, Spain. “I do not think the situation is dramatic but too much time is indeed spent writing proposals,” he commented. “Because success rates are decreasing, the time devoted to raise funds to run the lab necessarily needs to increase.”At the University of Adelaide in Australia, Andrew Somogyi, professor of pharmacology, thinks that the situation is serious: “[M]y postdocs would spend about half their time applying for grants.” Somogyi pointed out that the success rate has been declining in Australia, as it has in some other countries. “For ARC [Australian Research Council] the success rate is now close to 20%, which means many excellent projects don''t get funding because the assessment is now so fine cut,” he said.Similar developments have taken place in the USA at both the National Institutes of Health (NIH)—which provides US$16 billion funding per year and the American Cancer Society (ACS), the country''s largest private non-profit funder of cancer research, with a much smaller pot of US$120 million per year. The NIH funded 21% of research proposals submitted to it in 2009, compared with 32% a decade earlier, while the ACS approves only 15% of grant applications, down several percentage points over the past few years.While the NIH is prevented by federal law from allowing observers in to its grant review meetings, the ACS did allow a reporter from Nature to attend one of its sessions on the condition that the names of referees and the applications themselves were not revealed (Powell, 2010). The general finding was that while the review process works well when around 30% of proposals are successful, it tends to break down as the success rate drops, as more arbitrary decisions are made and the risk of strong pitches being rejected increases. This can also discourage the best people from being reviewers because the process becomes more tiring and time-consuming.Even when young researchers are not directly affected, the prevailing culture of short-term funding—which requires ongoing grant applications—can be disruptive…In some countries, funding shortfalls are also leading to the loss of permanent jobs, for example in the UK where finance minister George Osborne announced on October 20 that the science budget would be frozen at £4.6 billion, rather than cut as had been expected. Even so, combined with the cut in funding for universities that was announced on the same day, this raises the prospect of reductions in academic staff numbers, which could affect research projects. This follows several years of increasing funding for UK science. Such uncertainty is damaging, according to Cornelius Gross, deputy head of the mouse biology unit, European Molecular Biology Laboratory in Monterotondo, Italy. “Large fluctuations in funding have been shown to cause damage beyond their direct magnitude as can be seen in the US where the Clinton boom was inevitably followed by a slowdown that led to rapid and extreme tightening of budgets,” he said.Some countries are aware of these dangers and have acted to protect budgets and, in some cases, even increase spending. A report by the OECD argued that countries and companies that boosted research and development spending during the ‘creative destruction'' of an economic downturn tended to gain ground on their competitors and emerge from the crisis in a relatively stronger position (OECD, 2009). This was part of the rationale of the US stimulus package, which was intended to provide an immediate lift to the economy and has been followed by a slight increase in funding. The NIH''s budget is set to increase by $1 billion, or 3% from 2010 to 2011, reaching just over $32 billion. This looks like a real-term increase, since inflation in the USA is now between 1 and 2%. However, there are fears that budgets will soon be cut; even now the small increase at the Federal level is being offset by cuts in state support, according to Mike Seibert, research fellow at the US Department of Energy''s National Renewable Energy Laboratory. “The stimulus funds are disappearing in the US, and the overall budget for science may be facing a correction at the national level as economic, budget, and national debt issues are addressed,” he said. “The states in most cases are suffering their own budget crises and will be cutting back on anything that is not nailed down.”…countries and companies that boosted research and development spending during the ‘creative destruction'' of an economic downturn tended to gain ground on their competitors…In Germany, the overall funding situation is also confused by a split between the Federal and 16 state governments, each of which has its own budget for science. In contrast to many other countries though, both federal and state governments have responded boldly to the credit crisis by increasing the total budget for the DFG (Deutsche Forschungsgemeinschaft)—Germany''s largest research funding agency—to €2.3 billion in 2011. Moreover, total funding for research and education from the BMBF (Federal Ministry for Education and Research) is expected to increase by another 7% from €10.9 billion in 2010 to €11.64 billion, although the overall federal budget is set to shrink by 3.8% under Germany''s austerity measures (Anon, 2010). There have also been increases in funding from non-government sources, such as the Fraunhofer Society, Europe''s largest application-oriented research organization, which has an annual budget of €1.6 billion.The German line has been strongly applauded by the European Union, which since 2007 has channelled its funding for cutting-edge research through the European Research Council (ERC). The ERC''s current budget of €7.5 billion, which runs until 2013, was set in 2007 and negotiations for the next period have not yet begun, but the ERC''s executive agency director Jack Metthey has indicated that it will be increased: “The Commission will firmly sustain in the negotiations the view that research and innovation, central to the Europe 2020 Strategy agreed by the Member States, should be a top budgetary priority.” Metthey also implied that governments cutting funding, as the UK had been planning to do, were making a false economy that would gain only in the short term. “Situations vary at the national level but the European Commission believes that governments should maintain and even increase research and innovation investments during difficult times, because these are pro-growth, anti-crisis investments,” he said.Many other countries have to cope with flat or declining science budgets; some are therefore exploring ways in which to do more with less. In Japan, for instance, money has been concentrated on larger projects and fewer scientists, with the effect of intensifying the grant application process. Since 2002, the total Japanese government budget for science and technology has remained flat at around ¥3,500 billion—or €27 billion at current exchange rates—with a 1% annual decline in university support but increased funding for projects considered to be of high value to the economy. This culminated in March 2010 with the launch of the ¥100 billion (€880 million) programme for World Leading Innovative Research and Development on Science and Technology.But such attempts to make funding more competitive or focus it on specific areas could have unintended side effects on innovation and risk taking. One side effect can be favouring scientists who may be less creative but good at attracting grants, according to Roger Butlin, evolutionary biologist at the University of Sheffield in the UK. “Some productive staff are being targeted because they do not bring in grants, so money is taking precedence over output,” said Butlin. “This is very dangerous if it results in loss of good theoreticians or data specialists, especially as the latter will be a critical group in the coming years.”“Scientists are usually very energetic when they can pursue their own ideas and less so when the research target is too narrowly prescribed”There have been attempts to provide funding for young scientists based entirely on merit, such as the ERC ‘Starting Grant'' for top young researchers, whose budget was increased by 25% to €661 million for 2011. Although they are welcome, such schemes could also backfire unless they are supported by measures to continue supporting the scientists after these early career grants expire, according to Gross. “There are moves to introduce significant funding for young investigators to encourage independence, so called anti-brain-drain grants,” he said. “These are dangerous if provided without later independent positions for these people and a national merit-based funding agency to support their future work.”Such schemes might work better if they are incorporated into longer-term funding programmes that provide some security as well as freedom to expand a project and explore promising side avenues. Butlin cited the Canadian ‘Discovery Grant'' scheme as an example worth adopting elsewhere; it supports ongoing programmes with long-term goals, giving researchers freedom to pursue new lines of investigation, provided that they fit within the overall objective of the project.To some extent the system of ‘open calls''—supported by some European funding agencies—has the same objective, although it might not provide long-term funding. The idea is to allow scientists to manoeuvre within a broad objective, rather than confining them to specific lines of research or ‘thematic calls'', which tend to be highly focused. “The majority of funding should be distributed through open calls, rather than thematic calls,” said Thomas Höfer from the Modeling Research Group at the German Cancer Research Center & BioQuant Center in Heidelberg. “Scientists are usually very energetic when they can pursue their own ideas and less so when the research target is too narrowly prescribed. In my experience as a reviewer at both the national and EU level, open calls are also better at funding high-quality research whereas too narrow thematic calls often result in less coherent proposals.”“Cutting science, and education, is the national equivalent of a farmer eating his ‘seed corn'', and will lead to developing nation status within a generation”Common threads seems to be emerging from the different themes and opinions about funding: budgets should be consistent over time and spread fairly among all disciplines, rather than focused on targeted objectives. They should also be spread across the working lifetime of a scientist rather than being shot in a scatter-gun approach at young researchers. Finally, policies should put a greater emphasis on long-term support for the best scientists and projects, chosen for their merit. Above all, funding policy should reflect the fundamental importance of science to economies, as Seibert concluded: “Cutting science, and education, is the national equivalent of a farmer eating his ‘seed corn'', and will lead to developing nation status within a generation.”     

The Validation of Peer Review through Research Impact Measures and the Implications for Funding Strategies

There is a paucity of data in the literature concerning the validation of the grant application peer review process, which is used to help direct billions of dollars in research funds. Ultimately, this validation will hinge upon empirical data relating the output of funded projects to the predictions implicit in the overall scientific merit scores from the peer review of submitted applications. In an effort to address this need, the American Institute of Biological Sciences (AIBS) conducted a retrospective analysis of peer review data of 2,063 applications submitted to a particular research program and the bibliometric output of the resultant 227 funded projects over an 8-year period. Peer review scores associated with applications were found to be moderately correlated with the total time-adjusted citation output of funded projects, although a high degree of variability existed in the data. Analysis over time revealed that as average annual scores of all applications (both funded and unfunded) submitted to this program improved with time, the average annual citation output per application increased. Citation impact did not correlate with the amount of funds awarded per application or with the total annual programmatic budget. However, the number of funded applications per year was found to correlate well with total annual citation impact, suggesting that improving funding success rates by reducing the size of awards may be an efficient strategy to optimize the scientific impact of research program portfolios. This strategy must be weighed against the need for a balanced research portfolio and the inherent high costs of some areas of research. The relationship observed between peer review scores and bibliometric output lays the groundwork for establishing a model system for future prospective testing of the validity of peer review formats and procedures.     

Sample size and precision in NIH peer review

The Working Group on Peer Review of the Advisory Committee to the Director of NIH has recommended that at least 4 reviewers should be used to assess each grant application. A sample size analysis of the number of reviewers needed to evaluate grant applications reveals that a substantially larger number of evaluators are required to provide the level of precision that is currently mandated. NIH should adjust their peer review system to account for the number of reviewers needed to provide adequate precision in their evaluations.     

Use and importance of the NIH noncompeting continuation application

Each year National Institutes of Health (NIH) grant recipients must submit a noncompeting continuation application before receiving continued federal funding. This paper describes the use and value of the application. Investigators benefit by a yearly self-assessment of the research progress and future plans. The noncompeting continuation application is part of the important communication and interaction that should exist between the investigator and NIH staff. NIH staff members use the application to determine important scientific advances that have resulted from supported grants. Many planning activities and required reports are based on information contained in these applications. NIH staff performs scientific and budgetary review to ensure that research progress is satisfactory and that all budgetary and certification issues are in order. Detailed guidance is provided to help the grantee prepare the application. A separate significance section is suggested as a means to document key findings and their importance.     

The "study" role of past National Institutes of Health study sections

The original National Institutes of Health (NIH) study sections had two missions. The review of grant applications was the enduring one that we all recognize. The second original function, less remembered today, was to stand ready to advise the NIH, and in fact the entire community in a given biomedical field, on the current state of that discipline, as well as to opine on what new vistas were arising and to suggest (or occasionally even launch) appropriate courses of action. The present contribution is intended to remind us of this lesser-known original function of NIH study sections. We might ponder whether today's study sections, although more overworked than Sisyphus, should again take up this second function.     

The future of family practice training in California.

Although the number of physicians in California has doubled since 1963, the number of family and general practice physicians has declined. The ratio of office-based primary care physicians to population has also decreased. Graduate medical education is funded largely from patient care revenues, but the low rate of reimbursement for ambulatory care makes training in primary care specialties especially dependent on public support. Medicare, the Veterans Administration, and the University of California provide more than $325 million a year in support of graduate medical education in California. Federal and state grant programs provide $5 million a year for family physician training in the state, but appropriations to these programs have been reduced in real terms. California family practice residencies are disproportionately located at county hospitals, where funding shortfalls make them especially vulnerable to cuts in grant programs. Additional resources will be needed if more family physicians are to be trained.     

MiR‐29 coordinates age‐dependent plasticity brakes in the adult visual cortex

The authors regret having omitted grant attributions in the original publication. The funding section is herewith updated to reflect the change. “Funding attributed to Tommaso Pizzorusso was provided by EPIGEN Flagship project and PRIN2017HM8FA, funding attributed to Alessandro Cellerino was provided by Fondazione Pisa ETHERNA project, funding attributed to Pierre Baldi was provided by NIH (grant NIH {"type":"entrez-nucleotide","attrs":{"text":"GM123558","term_id":"221306858","term_text":"GM123558"}}GM123558), funding attributed to Jessica Kwok was provided by the Leverhulme Trust project grant (RPG‐2018‐100).”     

Science policies: How should science funding be allocated? An evolutionary biologists’ perspective

In an ideal world, funding agencies could identify the best scientists and projects and provide them with the resources to undertake these projects. Most scientists would agree that in practice, how funding for scientific research is allocated is far from ideal and likely compromises research quality. We, nine evolutionary biologists from different countries and career stages, provide a comparative summary of our impressions on funding strategies for evolutionary biology across eleven different funding agencies. We also assess whether and how funding effectiveness might be improved. We focused this assessment on 14 elements within four broad categories: (a) topical shaping of science, (b) distribution of funds, (c) application and review procedures, and (d) incentives for mobility and diversity. These comparisons revealed striking among‐country variation in those elements, including wide variation in funding rates, the effort and burden required for grant applications, and the extent of emphasis on societal relevance and individual mobility. We use these observations to provide constructive suggestions for the future and urge the need to further gather informed considerations from scientists on the effects of funding policies on science across countries and research fields.     

National Institutes of Health phase I, Small Business Innovation Research applications: fiscal year 1983 results

A review of the 356 disapproved Small Business Innovation Research (SBIR) proposals submitted to the National Institutes of Health (NIH) for fiscal year 1983 funding was undertaken to identify the most common shortcomings of those disapproved applications. The shortcomings were divided into four general classes by using the scheme developed by other authors when describing the reasons for the disapproval of regular NIH research applications. Comparison of the reasons for disapproval of SBIR applications with regular applications suggests comparable difficulties in the areas of the problem and the approach. There is some indication, however, that the SBIR proposals may have been weaker in the category of the principal investigator (PI). In general, it is the responsibility of the PI to demonstrate that the work is timely and can be performed with available technology and expertise, and that the guidelines for the NIH SBIR program have been satisfied.     

What does marine biological research cost? A case history of 25 years at a university research station

A study of research funding at Memorial University's Marine Sciences Research Laboratory shows that true costs far exceed support from grants and contracts alone. Research grant levels have to be matched by a similar level of support for infrastructure, and other university-supplied support (mostly salaries) may amount to twice that supplied by grants. Faced with declining external support for general infrastructure, universities can ill afford to become involved in scientific mega-projects. There is little evidence that increased funding enhances productivity.     

The Impact of National Institutes of Health Funding on U.S. Cardiovascular Disease Research

Background

Intense interest surrounds the recent expansion of US National Institutes of Health (NIH) budgets as part of economic stimulus legislation. However, the relationship between NIH funding and cardiovascular disease research is poorly understood, making the likely impact of this policy change unclear.

Methods

The National Library of Medicine''s PubMed database was searched for articles published from 1996 to 2006, originating from U.S. institutions, and containing the phrases “cardiolog,” “cardiovascular,” or “cardiac,” in the first author''s department. Research methodology, journal of publication, journal impact factor, and receipt of NIH funding were recorded. Differences in means and trends were tested with t-tests and linear regression, respectively, with P≤0.05 for significance.

Results

Of 117,643 world cardiovascular articles, 36,684 (31.2%) originated from the U.S., of which 10,293 (28.1%) received NIH funding. The NIH funded 40.1% of U.S. basic science articles, 20.3% of overall clinical trials, 18.1% of randomized-controlled, and 12.2% of multicenter clinical trials. NIH-funded and total articles grew significantly (65 articles/year, P<0.001 and 218 articles/year, P<0.001, respectively). The proportion of articles receiving NIH funding was stable, but grew significantly for basic science and clinical trials (0.87%/year, P<0.001 and 0.67%/year, P = 0.029, respectively). NIH-funded articles had greater journal impact factors than non NIH-funded articles (5.76 vs. 3.71, P<0.001).

Conclusions

NIH influence on U.S. cardiovascular research expanded in the past decade, during the period of NIH budget doubling. A substantial fraction of research is now directly funded and thus likely sensitive to budget fluctuations, particularly in basic science research. NIH funding predicts greater journal impact.     

Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices

Background

Peer review of grant applications has been criticized as lacking reliability. Studies showing poor agreement among reviewers supported this possibility but usually focused on reviewers’ scores and failed to investigate reasons for disagreement. Here, our goal was to determine how reviewers rate applications, by investigating reviewer practices and grant assessment criteria.

Methods and Findings

We first collected and analyzed a convenience sample of French and international calls for proposals and assessment guidelines, from which we created an overall typology of assessment criteria comprising nine domains relevance to the call for proposals, usefulness, originality, innovativeness, methodology, feasibility, funding, ethical aspects, and writing of the grant application. We then performed a qualitative study of reviewer practices, particularly regarding the use of assessment criteria, among reviewers of the French Academic Hospital Research Grant Agencies (Programmes Hospitaliers de Recherche Clinique, PHRCs). Semi-structured interviews and observation sessions were conducted. Both the time spent assessing each grant application and the assessment methods varied across reviewers. The assessment criteria recommended by the PHRCs were listed by all reviewers as frequently evaluated and useful. However, use of the PHRC criteria was subjective and varied across reviewers. Some reviewers gave the same weight to each assessment criterion, whereas others considered originality to be the most important criterion (12/34), followed by methodology (10/34) and feasibility (4/34). Conceivably, this variability might adversely affect the reliability of the review process, and studies evaluating this hypothesis would be of interest.

Conclusions

Variability across reviewers may result in mistrust among grant applicants about the review process. Consequently, ensuring transparency is of the utmost importance. Consistency in the review process could also be improved by providing common definitions for each assessment criterion and uniform requirements for grant application submissions. Further research is needed to assess the feasibility and acceptability of these measures.     

Ten simple rules for maximizing the recommendations of the NIH data management and sharing plan

The National Institutes of Health (NIH) Policy for Data Management and Sharing (DMS Policy) recognizes the NIH’s role as a key steward of United States biomedical research and information and seeks to enhance that stewardship through systematic recommendations for the preservation and sharing of research data generated by funded projects. The policy is effective as of January 2023. The recommendations include a requirement for the submission of a Data Management and Sharing Plan (DMSP) with funding applications, and while no strict template was provided, the NIH has released supplemental draft guidance on elements to consider when developing a plan. This article provides 10 key recommendations for creating a DMSP that is both maximally compliant and effective.