Size,Accumulation and Performance for Research Grants: Examining the Role of Size for Centres of Excellence

The present paper examines the relation between size, accumulation and performance for research grants, where we examine the relation between grant size for Centres of Excellence (CoE) funded by the Danish National Research Foundation (DNRF) and various ex post research performance measures, including impact and shares of highly cited articles. We examine both the relation between size and performance and also how performance for CoEs evolves over the course of grant periods. In terms of dynamics, it appears that performance over the grant period (i.e. 10 years) is falling for the largest CoEs, while it is increasing for those among the smallest half. Overall, multivariate econometric analysis finds evidence that performance is increasing in grant size and over time. In both cases, the relation appears to be non-linear, suggesting that there is a point at which performance peaks. The CoEs have also been very successful in securing additional funding, which can be viewed as a ‘cumulative effect’ of center grants. In terms of new personnel, the far majority of additional funding is spent on early career researchers, hence, this accumulation would appear to have a ‘generational’ dimension, allowing for scientific expertise to be passed on to an increasing number of younger researchers.     

The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model

How to quantify the impact of a researcher’s or an institution’s body of work is a matter of increasing importance to scientists, funding agencies, and hiring committees. The use of bibliometric indicators, such as the h-index or the Journal Impact Factor, have become widespread despite their known limitations. We argue that most existing bibliometric indicators are inconsistent, biased, and, worst of all, susceptible to manipulation. Here, we pursue a principled approach to the development of an indicator to quantify the scientific impact of both individual researchers and research institutions grounded on the functional form of the distribution of the asymptotic number of citations. We validate our approach using the publication records of 1,283 researchers from seven scientific and engineering disciplines and the chemistry departments at the 106 U.S. research institutions classified as “very high research activity”. Our approach has three distinct advantages. First, it accurately captures the overall scientific impact of researchers at all career stages, as measured by asymptotic citation counts. Second, unlike other measures, our indicator is resistant to manipulation and rewards publication quality over quantity. Third, our approach captures the time-evolution of the scientific impact of research institutions.     

The Impact of Funding through the RF President’s Grants for Young Scientists (the field – Medicine) on Research Productivity: A Quasi-Experimental Study and a Brief Systematic Review

The impact of grants on research productivity has been investigated by a number of retrospective studies. The results of these studies vary considerably. The objective of my study was to investigate the impact of funding through the RF President’s grants for young scientists on the research productivity of awarded applicants. The study compared the number of total articles and citations for awarded and rejected applicants who in 2007 took part in competitions for young candidates of science (CoS’s) and doctors of science (DoS’s) in the scientific field of medicine. The bibliometric analysis was conducted for the period from 2003 to 2012 (five years before and after the competition). The source of bibliometric data is the eLIBRARY.RU database. The impact of grants on the research productivity of Russian young scientists was assessed using the meta-analytical approach based on data from quasi-experimental studies conducted in other countries. The competition featured 149 CoS’s and 41 DoS’s, out of which 24 (16%) and 22 (54%) applicants, respectively, obtained funding. No difference in the number of total articles and citations at baseline, as well as in 2008–2012, for awarded and rejected applicants was found. The combination of data from the Russian study and other quasi-experimental studies (6 studies, 10 competitions) revealed a small treatment effect – an increase in the total number of publications over a 4–5-year period after the competition by 1.23 (95% CI 0.48–1.97). However, the relationship between the number of total publications published by applicants before and after the competition revealed that this treatment effect is an effect of the “maturation” of scientists with a high baseline publication activity – not of grant funding.     

B for bureaucracy

Despite the scientific community''s overwhelming support for the European Research Council, many grant recipients are irked about red tapeThere is one thing that most European researchers agree on: B stands for Brussels and bureaucracy. Research funding from the European Commission (EC), which distributes EU money, is accompanied by strict accountability and auditing rules in order to ensure that European taxpayers'' money is not wasted. All disbursements are treated the same, whether subsidies to farmers or grants to university researchers. However, the creation of the European Research Council (ERC) in 2007 as a new EU funding agency for basic research created high hopes among scientists for a reduced bureaucratic burden.… many researchers who have received ERC funding have been angered with accounting rules inherited from the EC''s Framework Programmes…ERC has, indeed, been a breath of fresh air to European-level research funding as it distributes substantial grants based only on the excellence of the proposal and has been overwhelmingly supported by the scientific community. Nevertheless, many researchers who have received ERC funding have been angered with accounting rules inherited from the EC''s Framework Programmes, and which seem impossible to change. In particular, a requirement to fill out time sheets to demonstrate that scientists spend an appropriate amount of time working on the project for which they received their ERC grant has triggered protests over the paperwork (Jacobs, 2009).Luis Serrano, Coordinator of the Systems Biology Programme at the Centre for Genomic Regulation in Barcelona, Spain, and recipient of a €2 million ERC Advanced Investigator Grant for five years, said the requirement of keeping time sheets is at best a waste of time and worst an insult to the high-level researchers. “Time sheets do not make much sense, to be honest. If you want to cheat, you can always cheat,” he said. He said other grants he receives from the Spanish government and the Human Frontier Science Programme do not require time sheets.Complaints by academic researchers about the creeping bureaucratization of research are not confined to the old continent (see Opinion by Paul van Helden, page 648). As most research, as well as universities and research institutes, is now funded by public agencies using taxpayers'' money, governments and regulators feel to be under pressure to make sure that the funds are not wasted or misappropriated. Yet, the USA and the EU have taken different approaches to making sure that scientists use public money correctly. In the USA, misappropriation of public money is considered a criminal offence that can be penalized by a ban on receiving public funds, fines and even jail time; in fact, a few scientists in the USA have gone to prison.By contrast, the EU puts the onus on controlling how public money is spent upfront. Research funding under the EU''s Framework Programmes requires clearly spelt out deliverables and milestones, and requires researchers to adhere to strict accountability and auditing rules. Not surprisingly, this comes with an administrative burden that has raised the ire of many scientists who feel that their time is better spent doing research. Serrano said in a major research centre such as the CRG, the administration could minimize the paper burden. “My administration prepares them for me and I go one, two, three, four, five and I do all of them. You can even have a machine sign for you,” he commented. “But I can imagine researchers who don''t have the administrative help, this can take up a significant amount of time.” For ERC grants, which by definition are for ‘blue-skies'' research and thus do not have milestones or deliverables, such paperwork is clearly not needed.Complaints by academic researchers about the creeping bureaucratization of research are not confined to the old continentNot everyone is as critical as Serrano though. Vincent Savolainen at the Division of Biology at Imperial College London, UK, and recipient of a €2.5 million, five-year ERC Advanced Investigator Grant, said, “Everything from the European Commission always comes with time sheets, and ERC is part of the European Commission.” Still, he felt it was very confusing to track time spent on individual grants for Principal Investigators such as him. “It is a little bit ridiculous but I guess there are places where people may abuse the system. So I can also see the side of the European Commission,” he said. “It''s not too bad. I can live with doing time sheets every month,” he added. “Still, it would be better if they got rid of it.”Juleen Zierath, an integrative physiologist in the Department of Molecular Medicine at Karolinska Institutet (Stockholm, Sweden), who received a €2.5 million, five-year ERC grant, takes the time sheets in her stride. “If I worked in a company, I would have to fill out a time sheet,” she said. “I''m delighted to have the funding. It''s a real merit. It''s a real honour. It really helps my work. If I have to fill out a time sheet for the privilege of having that amount of funding for five years, it''s not a big issue.”Zierath, a native of Milwaukee (WI, USA) who came to Karolinska for graduate work in 1989, said the ERC''s requirements are certainly “bureaucracy light” compared with the accounting and reporting requirements for more traditional EU funding instruments, such as the ‘Integrated Projects''. “ERC allows you to focus more on the science,” she said. “I don''t take time sheets as a signal that the European Union doesn''t count on us to be doing our work on the project. They have to be able to account for where they''re spending the money somehow and I think it''s okay. I can understand where some people would be really upset about that.”…governments and regulators feel to be under pressure to make sure that the funds are not wasted or misappropriated…The complaints about time sheets and other bureaucratic red tape have caught the attention of high-level scientists and research managers throughout Europe. In March 2009, the EC appointed an outside panel, headed by Vaira Vike-Freiberga, former President of Latvia, to review the ERC''s structures and mechanisms. The panel reported in July last year that the objective of building a world-class institution is not properly served by “undue cumbersome regulations, checks and controls.” Although fraud and mismanagement should be prevented, excessively bureaucratic procedures detract from the mission, and might be counter-productive.Helga Nowotny, President of the ERC, said the agency has to operate within the rules of the EC''s Framework Programme 7, which includes the ERC. She explained that if researchers hold several grants, the EC wants recipients to account for their time. “The Commission and the Rules of Participation of course argue that many of these researchers have more than one grant or they may have other contracts. In order to be accountable, the researchers must tell us how much time they spend on the project. But instead of simply asking if they spent a percentage of time on it, the Commission auditors insist on time sheets. I realize that filling them out has a high symbolic value for a researcher. So, why not leave it to the administration of the host institution?”Particle physicist Ian Halliday, President of the European Science Foundation and a major supporter of the ERC, said that financial irregularities that affected the EU over many years prompted the Commission to tighten its monitoring of cash outlays. “There have been endless scandals over the agricultural subsidies. Wine leaks. Nonexistent olive trees. You name it,” he said. “The Commission''s financial system is designed to cope with that kind of pressure as opposed to trusting the University of Cambridge, for example, which has been there for 800 years or so and has a well-earned reputation by now. That kind of system is applied in every corner of the European Commission. And that is basically what is causing the trouble. But these rules are not appropriate for research.”…financial irregularities that affected the EU over many years prompted the Commission to tighten its monitoring of cash outlaysNowotny is sympathetic and sensitive to the researchers'' complaints, saying that requiring time sheets for researchers sends a message of distrust. “It feels like you''re not trusted. It has this sort of pedantic touch to it,” she said. “If you''ve been recognized for doing this kind of top research, researchers feel, ‘Why bother [with time sheets]?''” But the bureaucratic alternative would not work for the ERC either. This would mean spelling out ‘deliverables'' in advance, which is clearly not possible with frontier research.Moreover, as Halliday pointed out, there is inevitably an element of fiction with time sheets in a research environment. In his area of research, for example, he considers it reasonable to track the hours of a technician fabricating parts of a telescope. But he noted that there is a different dynamic for researchers: “Scientists end up doing their science sitting in their bath at midnight. And you mull over problems and so forth. How do you put that on a time sheet?” Halliday added that one of the original arguments in establishing the ERC was to put it at an arm''s length from the Commission and in particular from financial regulations. But to require scientists to specify what proportion of their neurons are dedicated to a particular project at any hour of the day or night is nonsensical. Nowotny agreed. “The time sheet says I''ve been working on this from 11 in the morning until 6 in the evening or until midnight or whatever. This is not the way frontier research works,” she said.Halliday, who served for seven years as chief executive of the Particle Physics and Astronomy Research Council (Swindon, UK), commented that all governments require accountability. In Great Britain, for instance, much more general accountability rules are applied to grantees, thereby offering a measure of trust. “We were given a lot of latitude. Don''t get me wrong that we allowed fraud, but the system was fit for the purpose of science. If a professor says he''s spending half his time on a certain bit of medical research, let''s say, the government will expect half his salary to show up in the grants he gets from the funding agencies. We believe that if the University of Cambridge says that this guy is spending half his time on this research, then that''s probably right and nobody would get excited if it was 55% or 45%. People would get excited if it was 5%. There are checks and balances at that kind of level, but it''s not at a level of time sheets. It will be checked whether the project has done roughly what it said.”Other funding agencies also take a less bureaucratic approach. Candace Hassall, head of Basic Careers at the Wellcome Trust (London, UK), which funds research to improve human and animal health, said Wellcome''s translation awards have milestones that researchers are expected to meet. But “time sheets are something that the Wellcome Trust hasn''t considered at all. I would be astonished if we would ever consider them. We like to work closely with our researchers, but we don''t require that level of reporting detail,” she said. “We think that such detailed, day-by-day monitoring is actually potentially counterproductive overall. It drives people to be afraid to take risks when risks should be taken.”…to require scientists to specify what proportion of their neurons are dedicated to a particular project at any hour of the day or night is nonsensicalOn the other side of the Atlantic, Jack Dixon, vice president and chief scientific officer at the Howard Hughes Medical Institution (Chevy Chase, MD, USA), who directs Hughes'' investigator programme, said he''d never heard of researchers being asked to keep time sheets: “Researchers filling out time sheets is just something that''s never crossed our minds at the Hughes. I find it sort of goofy if you want to know the truth.”In fact, a system based on trust still works better in the academic worldInstead, Hughes trusts researchers to spend the money according to their needs. “We trust them,” Dixon said. “What we ask each of our scientists to do is devote 75% of their time to research and then we give them 25% of their time which they can use to teach, serve on committees. They can do consulting. They can do a variety of things. Researchers are free to explore.”There is already growing support for eliminating the time sheets and other bureaucratic requirements that come with an ERC grant, and which are obviously just a hangover from the old system. Indeed, there have been complaints, such as reviewers of grant applications having to fax in copies of their passports or identity cards, before being allowed sight of the proposals, said Nowotny. The review panel called on the EC to adapt its rules “based on trust and not suspicion and mistrust” so that the ERC can attain the “full realization of the dream shared by so many Europeans in the academic and policy world as well as in political milieus.”In fact, a system based on trust still works better in the academic world. Hassall commented that lump-sum payments encourage the necessary trust and give researchers a sense of freedom, which is already the principle behind ERC funding. “We think that you have to trust the researcher. Their careers are on the line,” she said. Nowotny hopes ERC will be allowed to take a similar approach to that of the Wellcome Trust, with its grants treated more like “a kind of prize money” than as a contract for services.She sees an opportunity to relax the bureaucratic burden with a scheduled revision of the Rules of Participation but issues a word of caution given that, when it comes to EU money, other players are involved. “We don''t know whether we will succeed in this because it''s up to the finance ministers, not even the research ministers,” she explained. “It''s the finance ministers who decide the rules of participation. If finance ministers agree then the time sheets would be gone.”     

To Apply or Not to Apply: A Survey Analysis of Grant Writing Costs and Benefits

We surveyed 113 astronomers and 82 psychologists active in applying for federally funded research on their grant-writing history between January, 2009 and November, 2012. We collected demographic data, effort levels, success rates, and perceived non-financial benefits from writing grant proposals. We find that the average proposal takes 116 PI hours and 55 CI hours to write; although time spent writing was not related to whether the grant was funded. Effort did translate into success, however, as academics who wrote more grants received more funding. Participants indicated modest non-monetary benefits from grant writing, with psychologists reporting a somewhat greater benefit overall than astronomers. These perceptions of non-financial benefits were unrelated to how many grants investigators applied for, the number of grants they received, or the amount of time they devoted to writing their proposals. We also explored the number of years an investigator can afford to apply unsuccessfully for research grants and our analyses suggest that funding rates below approximately 20%, commensurate with current NIH and NSF funding, are likely to drive at least half of the active researchers away from federally funded research. We conclude with recommendations and suggestions for individual investigators and for department heads.     

Do Scientific Advancements Lean on the Shoulders of Giants? A Bibliometric Investigation of the Ortega Hypothesis

Background

In contrast to Newton''s well-known aphorism that he had been able “to see further only by standing on the shoulders of giants,” one attributes to the Spanish philosopher Ortega y Gasset the hypothesis saying that top-level research cannot be successful without a mass of medium researchers on which the top rests comparable to an iceberg.

Methodology/Principal Findings

The Ortega hypothesis predicts that highly-cited papers and medium-cited (or lowly-cited) papers would equally refer to papers with a medium impact. The Newton hypothesis would be supported if the top-level research more frequently cites previously highly-cited work than that medium-level research cites highly-cited work. Our analysis is based on (i) all articles and proceedings papers which were published in 2003 in the life sciences, health sciences, physical sciences, and social sciences, and (ii) all articles and proceeding papers which were cited within these publications. The results show that highly-cited work in all scientific fields more frequently cites previously highly-cited papers than that medium-cited work cites highly-cited work.

Conclusions/Significance

We demonstrate that papers contributing to the scientific progress in a field lean to a larger extent on previously important contributions than papers contributing little. These findings support the Newton hypothesis and call into question the Ortega hypothesis (given our usage of citation counts as a proxy for impact).     

Facing the credit crunch. Politics sends mixed messages for science in the wake of the global financial crisis

The global response to the credit crunch has varied from belt tightening to spending sprees. Philip Hunter investigates how various countries react to the financial crisis in terms of supporting scientific research.The overall state of biomedical research in the wake of the global financial crisis remains unclear amid growing concern that competition for science funding is compromising the pursuit of research. Such concerns pre-date the credit crunch, but there is a feeling that an increasing amount of time and energy is being wasted in the ongoing scramble for grants, in the face of mounting pressure from funding agencies demanding value for money. Another problem is balancing funding between different fields; while the biomedical sciences have generally fared well, they are increasingly dependent on basic research in physics and chemistry that are in greater jeopardy. This has led to calls for rebalancing funding, in order to ensure the long-term viability of all fields in an increasingly multidisciplinary and collaborative research world.For countries that are cutting funding—such as Spain, Italy and the UK—the immediate priority is to preserve the fundamental research base and avoid a significant drain of expertise, either to rival countries or away from science altogether. This has highlighted the plight of postdoctoral researchers who have traditionally been the first to suffer from funding cuts, partly because they have little immediate impact on on a country''s scientific competitiveness. Postdocs have been the first to go whenever budgets have been cut, according to Richard Frankel, a physicist at California Polytechnic State University in Saint Luis Obispo, who investigates magnetotaxis in bacteria. “In the short term there will be little effect but the long-term effects can be devastating,” he said.…there is a feeling that an increasing amount of time and energy is being wasted in the ongoing scramble for grants, in the face of mounting pressure from funding agencies…According to Peter Stadler, head of a bioinformatics group at the University of Leipzig in Germany, such cuts tend to cause the long-term erosion of a country''s science skills base. “Short-term cuts in science funding translate totally into a brain drain, since they predominantly affect young researchers who are paid from the soft money that is drying up first,” said Stadler. “They either leave science, an irreversible step, or move abroad but do not come back later, because the medium-term effect of cuts is a reduction in career opportunities and fiercer competition giving those already in the system a big advantage.”Even when young researchers are not directly affected, the prevailing culture of short-term funding—which requires ongoing grant applications—can be disruptive, according to Xavier Salvatella, principal investigator in the Laboratory of Molecular Biophysics at the Institute for Research in Biomedicine in Barcelona, Spain. “I do not think the situation is dramatic but too much time is indeed spent writing proposals,” he commented. “Because success rates are decreasing, the time devoted to raise funds to run the lab necessarily needs to increase.”At the University of Adelaide in Australia, Andrew Somogyi, professor of pharmacology, thinks that the situation is serious: “[M]y postdocs would spend about half their time applying for grants.” Somogyi pointed out that the success rate has been declining in Australia, as it has in some other countries. “For ARC [Australian Research Council] the success rate is now close to 20%, which means many excellent projects don''t get funding because the assessment is now so fine cut,” he said.Similar developments have taken place in the USA at both the National Institutes of Health (NIH)—which provides US$16 billion funding per year and the American Cancer Society (ACS), the country''s largest private non-profit funder of cancer research, with a much smaller pot of US$120 million per year. The NIH funded 21% of research proposals submitted to it in 2009, compared with 32% a decade earlier, while the ACS approves only 15% of grant applications, down several percentage points over the past few years.While the NIH is prevented by federal law from allowing observers in to its grant review meetings, the ACS did allow a reporter from Nature to attend one of its sessions on the condition that the names of referees and the applications themselves were not revealed (Powell, 2010). The general finding was that while the review process works well when around 30% of proposals are successful, it tends to break down as the success rate drops, as more arbitrary decisions are made and the risk of strong pitches being rejected increases. This can also discourage the best people from being reviewers because the process becomes more tiring and time-consuming.Even when young researchers are not directly affected, the prevailing culture of short-term funding—which requires ongoing grant applications—can be disruptive…In some countries, funding shortfalls are also leading to the loss of permanent jobs, for example in the UK where finance minister George Osborne announced on October 20 that the science budget would be frozen at £4.6 billion, rather than cut as had been expected. Even so, combined with the cut in funding for universities that was announced on the same day, this raises the prospect of reductions in academic staff numbers, which could affect research projects. This follows several years of increasing funding for UK science. Such uncertainty is damaging, according to Cornelius Gross, deputy head of the mouse biology unit, European Molecular Biology Laboratory in Monterotondo, Italy. “Large fluctuations in funding have been shown to cause damage beyond their direct magnitude as can be seen in the US where the Clinton boom was inevitably followed by a slowdown that led to rapid and extreme tightening of budgets,” he said.Some countries are aware of these dangers and have acted to protect budgets and, in some cases, even increase spending. A report by the OECD argued that countries and companies that boosted research and development spending during the ‘creative destruction'' of an economic downturn tended to gain ground on their competitors and emerge from the crisis in a relatively stronger position (OECD, 2009). This was part of the rationale of the US stimulus package, which was intended to provide an immediate lift to the economy and has been followed by a slight increase in funding. The NIH''s budget is set to increase by $1 billion, or 3% from 2010 to 2011, reaching just over $32 billion. This looks like a real-term increase, since inflation in the USA is now between 1 and 2%. However, there are fears that budgets will soon be cut; even now the small increase at the Federal level is being offset by cuts in state support, according to Mike Seibert, research fellow at the US Department of Energy''s National Renewable Energy Laboratory. “The stimulus funds are disappearing in the US, and the overall budget for science may be facing a correction at the national level as economic, budget, and national debt issues are addressed,” he said. “The states in most cases are suffering their own budget crises and will be cutting back on anything that is not nailed down.”…countries and companies that boosted research and development spending during the ‘creative destruction'' of an economic downturn tended to gain ground on their competitors…In Germany, the overall funding situation is also confused by a split between the Federal and 16 state governments, each of which has its own budget for science. In contrast to many other countries though, both federal and state governments have responded boldly to the credit crisis by increasing the total budget for the DFG (Deutsche Forschungsgemeinschaft)—Germany''s largest research funding agency—to €2.3 billion in 2011. Moreover, total funding for research and education from the BMBF (Federal Ministry for Education and Research) is expected to increase by another 7% from €10.9 billion in 2010 to €11.64 billion, although the overall federal budget is set to shrink by 3.8% under Germany''s austerity measures (Anon, 2010). There have also been increases in funding from non-government sources, such as the Fraunhofer Society, Europe''s largest application-oriented research organization, which has an annual budget of €1.6 billion.The German line has been strongly applauded by the European Union, which since 2007 has channelled its funding for cutting-edge research through the European Research Council (ERC). The ERC''s current budget of €7.5 billion, which runs until 2013, was set in 2007 and negotiations for the next period have not yet begun, but the ERC''s executive agency director Jack Metthey has indicated that it will be increased: “The Commission will firmly sustain in the negotiations the view that research and innovation, central to the Europe 2020 Strategy agreed by the Member States, should be a top budgetary priority.” Metthey also implied that governments cutting funding, as the UK had been planning to do, were making a false economy that would gain only in the short term. “Situations vary at the national level but the European Commission believes that governments should maintain and even increase research and innovation investments during difficult times, because these are pro-growth, anti-crisis investments,” he said.Many other countries have to cope with flat or declining science budgets; some are therefore exploring ways in which to do more with less. In Japan, for instance, money has been concentrated on larger projects and fewer scientists, with the effect of intensifying the grant application process. Since 2002, the total Japanese government budget for science and technology has remained flat at around ¥3,500 billion—or €27 billion at current exchange rates—with a 1% annual decline in university support but increased funding for projects considered to be of high value to the economy. This culminated in March 2010 with the launch of the ¥100 billion (€880 million) programme for World Leading Innovative Research and Development on Science and Technology.But such attempts to make funding more competitive or focus it on specific areas could have unintended side effects on innovation and risk taking. One side effect can be favouring scientists who may be less creative but good at attracting grants, according to Roger Butlin, evolutionary biologist at the University of Sheffield in the UK. “Some productive staff are being targeted because they do not bring in grants, so money is taking precedence over output,” said Butlin. “This is very dangerous if it results in loss of good theoreticians or data specialists, especially as the latter will be a critical group in the coming years.”“Scientists are usually very energetic when they can pursue their own ideas and less so when the research target is too narrowly prescribed”There have been attempts to provide funding for young scientists based entirely on merit, such as the ERC ‘Starting Grant'' for top young researchers, whose budget was increased by 25% to €661 million for 2011. Although they are welcome, such schemes could also backfire unless they are supported by measures to continue supporting the scientists after these early career grants expire, according to Gross. “There are moves to introduce significant funding for young investigators to encourage independence, so called anti-brain-drain grants,” he said. “These are dangerous if provided without later independent positions for these people and a national merit-based funding agency to support their future work.”Such schemes might work better if they are incorporated into longer-term funding programmes that provide some security as well as freedom to expand a project and explore promising side avenues. Butlin cited the Canadian ‘Discovery Grant'' scheme as an example worth adopting elsewhere; it supports ongoing programmes with long-term goals, giving researchers freedom to pursue new lines of investigation, provided that they fit within the overall objective of the project.To some extent the system of ‘open calls''—supported by some European funding agencies—has the same objective, although it might not provide long-term funding. The idea is to allow scientists to manoeuvre within a broad objective, rather than confining them to specific lines of research or ‘thematic calls'', which tend to be highly focused. “The majority of funding should be distributed through open calls, rather than thematic calls,” said Thomas Höfer from the Modeling Research Group at the German Cancer Research Center & BioQuant Center in Heidelberg. “Scientists are usually very energetic when they can pursue their own ideas and less so when the research target is too narrowly prescribed. In my experience as a reviewer at both the national and EU level, open calls are also better at funding high-quality research whereas too narrow thematic calls often result in less coherent proposals.”“Cutting science, and education, is the national equivalent of a farmer eating his ‘seed corn'', and will lead to developing nation status within a generation”Common threads seems to be emerging from the different themes and opinions about funding: budgets should be consistent over time and spread fairly among all disciplines, rather than focused on targeted objectives. They should also be spread across the working lifetime of a scientist rather than being shot in a scatter-gun approach at young researchers. Finally, policies should put a greater emphasis on long-term support for the best scientists and projects, chosen for their merit. Above all, funding policy should reflect the fundamental importance of science to economies, as Seibert concluded: “Cutting science, and education, is the national equivalent of a farmer eating his ‘seed corn'', and will lead to developing nation status within a generation.”     

Incorrect Citations Give Unfair Credit to Review Authors in Ecology Journals

The number of citations that papers receive has become significant in measuring researchers'' scientific productivity, and such measurements are important when one seeks career opportunities and research funding. Skewed citation practices can thus have profound effects on academic careers. We investigated (i) how frequently authors misinterpret original information and (ii) how frequently authors inappropriately cite reviews instead of the articles upon which the reviews are based. To reach this aim, we carried a survey of ecology journals indexed in the Web of Science and assessed the appropriateness of citations of review papers. Reviews were significantly more often cited than regular articles. In addition, 22% of citations were inaccurate, and another 15% unfairly gave credit to the review authors for other scientists'' ideas. These practices should be stopped, mainly through more open discussion among mentors, researchers and students.     

The Effects of Aging on Researchers' Publication and Citation Patterns

The average age at which U.S. researchers receive their first grant from NIH has increased from 34.3 in 1970, to 41.7 in 2004. These data raise the crucial question of the effects of aging on the scientific productivity and impact of researchers. Drawing on a sizeable sample of 6,388 university professors in Quebec who have published at least one paper between 2000 and 2007, our results identify two turning points in the professors'' careers. A first turning point is visible at age 40 years, where researchers start to rely on older literature and where their productivity increases at a slower pace—after having increased sharply since the beginning of their career. A second turning point can be seen around age 50, when researchers are the most productive whereas their average scientific impact is at its lowest. Our results also show that older professors publish fewer first-authored papers and move closer to the end of the list of co-authors. Although average scientific impact per paper decreases linearly until about age 50, the average number of papers in highly cited journals and among highly cited papers rises continuously until retirement. Our results show clearly that productivity and impact are not a simple and declining function of age and that we must take into account the collaborative aspects of scientific research. Science is a collective endeavor and, as our data shows, researchers of all ages play a significant role in its dynamic.     

The Impact of National Institutes of Health Funding on U.S. Cardiovascular Disease Research

Background

Intense interest surrounds the recent expansion of US National Institutes of Health (NIH) budgets as part of economic stimulus legislation. However, the relationship between NIH funding and cardiovascular disease research is poorly understood, making the likely impact of this policy change unclear.

Methods

The National Library of Medicine''s PubMed database was searched for articles published from 1996 to 2006, originating from U.S. institutions, and containing the phrases “cardiolog,” “cardiovascular,” or “cardiac,” in the first author''s department. Research methodology, journal of publication, journal impact factor, and receipt of NIH funding were recorded. Differences in means and trends were tested with t-tests and linear regression, respectively, with P≤0.05 for significance.

Results

Of 117,643 world cardiovascular articles, 36,684 (31.2%) originated from the U.S., of which 10,293 (28.1%) received NIH funding. The NIH funded 40.1% of U.S. basic science articles, 20.3% of overall clinical trials, 18.1% of randomized-controlled, and 12.2% of multicenter clinical trials. NIH-funded and total articles grew significantly (65 articles/year, P<0.001 and 218 articles/year, P<0.001, respectively). The proportion of articles receiving NIH funding was stable, but grew significantly for basic science and clinical trials (0.87%/year, P<0.001 and 0.67%/year, P = 0.029, respectively). NIH-funded articles had greater journal impact factors than non NIH-funded articles (5.76 vs. 3.71, P<0.001).

Conclusions

NIH influence on U.S. cardiovascular research expanded in the past decade, during the period of NIH budget doubling. A substantial fraction of research is now directly funded and thus likely sensitive to budget fluctuations, particularly in basic science research. NIH funding predicts greater journal impact.     

Impact analysis of a regional scientific journal (1980-2000): supporting promising local researchers pays the greatest dividends

During the past 28 years, the journal "Collegium Antropologicum" has continuously served as one of the main disseminators of anthropological scientific production in Central and Eastern Europe. The journal was committed to its role of a multidisciplinary platform for presenting wide range of research topics relevant to anthropology, from investigations within social and cultural anthropology and archaeology to those covering contemporary population genetics, human evolution and biomedical issues. Two key strategies aimed at sustaining and increasing the impact of this journal were oriented towards: (i) identification of promising local groups of researchers who were at disadvantage by many aspects (e.g. educational curricula, financial supports, language barriers etc.) when trying to publish their research internationally, and (ii) invitation and encouragement of already established international scientists to make contributions for "Collegium Antropologicum". From 1980-2000, 89 articles (or 6.3% of all published papers during that period) were cited 6 or more times, contributing disproportionately to journal's impact (nearly a third of all citations received). In an attempt to identify such papers more readily among the submissions to the journal in the future, we analyzed research topics and affiliations of the authors among the 89 papers receiving most citations in comparison to all papers published. Among the papers most frequently cited, we found greater-than-expected prevalence of Croatian researchers (especially when publishing in collaboration with international scientists) and studies of special populations. Several papers received more than 25 citations or had overall citation intensity greater than 2 per year. This implies that an interesting article from a local group of researchers can still resonate with international audience although published in a regional journal. Present analysis supports current editorial strategy that with a help of the international consulting editorial board continuously improves international recognition of this journal. The results imply that a balanced encouragement to promising local groups of researchers and to contributions of already established international scientists is a strategy superior to others in maintaining and increasing the impact of this regional journal.     

What does marine biological research cost? A case history of 25 years at a university research station

A study of research funding at Memorial University's Marine Sciences Research Laboratory shows that true costs far exceed support from grants and contracts alone. Research grant levels have to be matched by a similar level of support for infrastructure, and other university-supplied support (mostly salaries) may amount to twice that supplied by grants. Faced with declining external support for general infrastructure, universities can ill afford to become involved in scientific mega-projects. There is little evidence that increased funding enhances productivity.     

METHODOLOGY IN MEDICAL RESEARCH—The Need for Controlled Clinical Studies

Too many medical researchers vitiate their work by ignoring the problem of uncontrolled variables. They therefore publish clinical impressions “dressed up” in scientifically meaningless numbers. A prototypical example of this practice is contrasted with a controlled study, each employing the same (small) number of patients. It is shown how the use of controls can convert a meaningless experiment into one that has assessable scientific significance.A survey of current literature revealed that in only 21 of 100 articles studied were adequately controlled experimental conditions employed.Since they usually deal with very complex systems, it is urged that medical researchers exercise more scientific rigor with regard to control problems.     

Examining the impact of health research facilitated by small peer-reviewed research operating grants in a women's and children's health centre

Background

There has been limited research on the impact of research funding for small, institutional grants. The IWK Health Centre, a children and women's hospital in Maritime Canada, provides small amounts (up to $15,000) of research funding for staff and trainees at all levels of experience through its Research Operating Grants. These grants are rigorously peer-reviewed. To evaluate the impact of these grants, an assessment was completed of several different areas of impact.

Findings

An online questionnaire was sent to 64 Principal Investigators and Co-Investigators from Research Operating Grants awarded from 2004 to 2006. The questionnaire was designed to assess five areas of potential impact: (1) research, (2) policy, (3) practice, (4) society and (5) personal. Research impact reported by participants included publications (72%), presentations (82%) and knowledge transfer beyond the traditional formats (51%). Practice impact was reported by 67% of participants, policy impact by 15% and societal impact by 18%. All participants reported personal impact.

Conclusions

Small research grants yield similar impacts to relatively large research grants. Regardless of the total amount of research funds awarded, rigorously peer-reviewed research projects have the potential for significant impact at the level of knowledge transfer and changes in clinical practice and policy. Additional findings in the present research indicate that small awards have the potential to have significant impact on the individual grant holder across a variety of capacity building variables. These personal impacts are particularly noteworthy in the context of developing the research programs of novice researchers.

     

Good news for the people who love bad news: an analysis of the funding of the top 1% most highly cited ecologists

The most highly cited ecologists and environmental scientists provide both a benchmark and unique opportunity to consider the importance of research funding. Here, we use citation data and self‐reported funding levels to assess the relative importance of various factors in shaping productivity and potential impact. The elite were senior Americans, well funded, with large labs. In contrast to Canadian NSERC grant holders (not in the top 1%), citations per paper did not increase with higher levels of funding within the ecological elite. We propose that this is good news for several reasons. It suggests that the publications generated by the top ecologists and environmental scientists are subject to limitations, that higher volume of publications is always important, and that increased funding to ecologists in general can shift our discipline to wider research networks. As expected, collaboration was identified as an important factor for the elite, and hopefully, this serves as a positive incentive to funding agencies since it increases the visibility of their research.     

Do Pressures to Publish Increase Scientists' Bias? An Empirical Support from US States Data

The growing competition and “publish or perish” culture in academia might conflict with the objectivity and integrity of research, because it forces scientists to produce “publishable” results at all costs. Papers are less likely to be published and to be cited if they report “negative” results (results that fail to support the tested hypothesis). Therefore, if publication pressures increase scientific bias, the frequency of “positive” results in the literature should be higher in the more competitive and “productive” academic environments. This study verified this hypothesis by measuring the frequency of positive results in a large random sample of papers with a corresponding author based in the US. Across all disciplines, papers were more likely to support a tested hypothesis if their corresponding authors were working in states that, according to NSF data, produced more academic papers per capita. The size of this effect increased when controlling for state''s per capita R&D expenditure and for study characteristics that previous research showed to correlate with the frequency of positive results, including discipline and methodology. Although the confounding effect of institutions'' prestige could not be excluded (researchers in the more productive universities could be the most clever and successful in their experiments), these results support the hypothesis that competitive academic environments increase not only scientists'' productivity but also their bias. The same phenomenon might be observed in other countries where academic competition and pressures to publish are high.     

Can small institutes address some problems facing biomedical researchers?

At a time of historically low National Institutes of Health funding rates and many problems with the conduct of research (unfunded mandates, disgruntled reviewers, and rampant paranoia), there is a concern that biomedical research as a profession is waning in the United States (see ”Rescuing US biomedical research from its systemic flaws” by Alberts and colleagues in the Proceedings of the National Academy of Sciences). However, it is wonderful to discover something new and to tackle tough puzzles. If we could focus more of our effort on discussing scientific problems and doing research, then we could be more productive and perhaps happier. One potential solution is to focus efforts on small thematic institutes in the university structure that can provide a stimulating and supportive environment for innovation and exploration. With an open-lab concept, there are economies of scale that can diminish paperwork and costs, while providing greater access to state-of-the-art equipment. Merging multiple disciplines around a common theme can catalyze innovation, and this enables individuals to develop new concepts without giving up the credit they deserve, because it is usually clear who did the work. Small institutes do not solve larger systemic problems but rather enable collective efforts to address the noisome aspects of the system and foster an innovative community effort to address scientific problems.     

Science: priceless,but costly

Working as a researcher is very satisfying. However, it comes with a price. This is a story about growing up as a scientist in the field of molecular biology. Starting as a young, rather naive researcher, I learned, step by step, not only the facts about my favorite RNA molecules but also the demands and downsides of academia. Going through my recent “scientific awakening,” I fully acknowledged the rules of the game: to write, to publish, to patent, to apply for grants and awards, and finally, to engage in all forms of coscientific endeavors. After going through a divorce, single parenting, immigration, and being scooped, I became a scientist who finally takes her career in her own hands and navigates through, but does not succumb to, the difficulties in science. This is my monument to resilience.     

Why Most Biomedical Findings Echoed by Newspapers Turn Out to be False: The Case of Attention Deficit Hyperactivity Disorder

Context

Because positive biomedical observations are more often published than those reporting no effect, initial observations are often refuted or attenuated by subsequent studies.

Objective

To determine whether newspapers preferentially report on initial findings and whether they also report on subsequent studies.

Methods

We focused on attention deficit hyperactivity disorder (ADHD). Using Factiva and PubMed databases, we identified 47 scientific publications on ADHD published in the 1990s and soon echoed by 347 newspapers articles. We selected the ten most echoed publications and collected all their relevant subsequent studies until 2011. We checked whether findings reported in each “top 10” publication were consistent with previous and subsequent observations. We also compared the newspaper coverage of the “top 10” publications to that of their related scientific studies.

Results

Seven of the “top 10” publications were initial studies and the conclusions in six of them were either refuted or strongly attenuated subsequently. The seventh was not confirmed or refuted, but its main conclusion appears unlikely. Among the three “top 10” that were not initial studies, two were confirmed subsequently and the third was attenuated. The newspaper coverage of the “top 10” publications (223 articles) was much larger than that of the 67 related studies (57 articles). Moreover, only one of the latter newspaper articles reported that the corresponding “top 10” finding had been attenuated. The average impact factor of the scientific journals publishing studies echoed by newspapers (17.1 n = 56) was higher (p<0.0001) than that corresponding to related publications that were not echoed (6.4 n = 56).

Conclusion

Because newspapers preferentially echo initial ADHD findings appearing in prominent journals, they report on uncertain findings that are often refuted or attenuated by subsequent studies. If this media reporting bias generalizes to health sciences, it represents a major cause of distortion in health science communication.     

Levels of citation of nonhuman animal studies conducted at a Canadian research hospital

The publication of scientific articles that receive few or no citations raises questions of the appropriate use of resources as well as ethics. In the case of animal research, the ethics issue extends beyond human patients to nonhuman animals, as the research subjects them to pain and, typically, to death. This study is a citation analysis of animal research conducted at Toronto's Hospital for Sick Children (HSC). Of the 594 publications (1990 to 1995) on animal research by affiliates of HSC, 29% received fewer than 10 citations in a 10-year period. We compare the research history of 13 “best ”and 13 “worst ”HSC scientists. Worst researchers continue to do infrequently cited research. Recommendations indicate how institutions and researchers can become more effective and accountable.