By Roy L. Silverstein, MD

2008-05-01

The NIH peer-review system is not exactly a sinking ship, but the boat is taking on water at an alarming rate and has reached a "tipping point" with respect to efficiency and efficacy. The problem stems from a large increase in grant applications submitted to NIH (now an astounding 80,000/year), an NIH budget that is not keeping up with inflation, and an overburdened research community that is not as willing to serve on study sections. In response to these stresses on peer review, NIH Director Elias Zerhouni assembled a working group to study the problem and make recommendations for change. They solicited input from all stakeholders, including professional societies, such as ASH, and received nearly 3,000 specific comments on their Web site — a strong indication of the importance of the issue and anxieties evoked by any discussion of peer review at times of funding uncertainty. The detailed draft report can be viewed online.

The report identified key principles and significant challenges and outlined a series of potential "actions." Among the more interesting and controversial recommendations were to shorten the length of the application and the reviewers' summary, eliminate "unscoring" (triage) for the lower tier of applications, add a "not recommended for resubmission" rating, eliminate amended applications so all would be considered de novo, and focus review on scientific merit and impact, rather than on specific weaknesses of approach. Noting that the current NIH scoring system is out of step with principles of modern psychometric science, the group recommended aligning applications and reviews with explicit criteria (e.g., impact, investigator, innovation/originality, plan, and environment [including institutional support]). The panel also suggested several strategies to address problems related to poor competitiveness of first-time grant applicants and clinical research projects and with review of "transformative" and multidisciplinary research. To reduce stress on science support systems, the panel made several recommendations that are sure to raise some hackles, including establishment of a minimum-percent effort (e.g., 20 percent) for investigators on research grants and re-examination of incentives in the current NIH system that drive expansion of the research enterprise. The implication is that NIH should pay less and universities/medical centers should pay more.

The report also noted less controversial recommendations. These included enhancing and standardizing training of reviewers and study section chairs, developing incentives for successful scientists to serve on study sections, continuing to pilot use of electronic review tools, introducing more flexibility into the system for reviewers, developing two-way communication between reviewers and applicants, and mandating periodic data-driven assessment of the entire peer-review process.

An implementation plan will soon be developed by NIH, but whatever form the final renovations take, it is highly likely that the relationships between the extramural research committee and the NIH are going to change dramatically.

back to top