Original Article

Virtual and Peer Reviews of Grant Applications at the Agency for Healthcare Research and Quality

Authors: Nghia M. Vo, MD, Rebecca Trocki, MSHAI

Abstract

Objectives: This study documents the first six unplanned virtual review (VR) sessions conducted during the 2012 hurricane season at the Agency for Healthcare Research and Quality and their effects on review outcomes. It also compares these VR sessions with five face-to-face (FF) sessions.

Methods: In the first part of this study, six VR sessions are analyzed in terms of feasibility, reproducibility, and reviewers’ responses to a questionnaire about VR. In the second part, the VR sessions are compared with five other FF meetings in terms of costs and duration per discussed application.

Results: Despite their technical novelty, all of the VR sessions have been successfully conducted to the satisfaction of reviewers and agency organizers. Special emphasis panel reviewers are more receptive to the new technology than study section reviewers: 75% versus 42%, respectively ( P < 0.05). Although the duration per discussed application is comparable to FF, the cost per reviewer is much lower for VR sessions than FF sessions.

Conclusions: VR has successfully been used in six review sessions with a maximum of 34 discussed applications per session, special emphasis panel reviewers are more receptive to VR than SS reviewers, VR is a duplicable and low-cost method of review, and practitioners and scientists are urged to serve as reviewers because doing so may assist them in receiving funding.

This content is limited to qualifying members.

Existing members, please login first

If you have an existing account please login now to access this article or view purchase options.

Purchase only this article ($25)

Create a free account, then purchase this article to download or access it online for 24 hours.

Purchase an SMJ online subscription ($75)

Create a free account, then purchase a subscription to get complete access to all articles for a full year.

Purchase a membership plan (fees vary)

Premium members can access all articles plus recieve many more benefits. View all membership plans and benefit packages.

References

1. Agency for Healthcare Research and Quality. Funding & grants. http://www.ahrq.gov/funding/index.html. Accessed July 20, 2015.
 
2. Gallo SA, Carpenter AS, Glisson SR. Teleconference versus face-to-face scientific peer review of grant application: effects on review outcomes. PLoS One 2013;8:e71693.
 
3. Agency for Healthcare Research and Quality. Browse meetings. https://hhs-ahrq.webex.com. Accessed July 20, 2015.
 
4. NOBSCOT Corporation. How to improve exit interview participation rates. http://www.nobscot.com/library/how_to_improve_participation.cfm. Published 2004. Accessed July 20, 2015.
 
5. Agency for Healthcare Research and Quality. AHRQ announces changes to peer review processes, evaluation review criteria, and new application forms for grant applications. http://grants.nih.gov/grants/guide/notice-files/NOT-HS-10-002.html. Accessed July 20, 2015.
 
6. Gordon R, Poulin BJ. Cost of the NSERC Science Grant Peer Review System exceeds the cost of giving every qualified researcher a baseline grant. Account Res 2009;16:13-40.
 
7. Costello LC. Perspective: is NIH funding the ‘‘best science by the best scientists’’? A critique of the NIH R01 research grant review policies. Acad Med 2010;85:775-779.
 
8. Spiegel AM. Commentary: new guidelines for NIH peer review: improving the system or undermining it? Acad Med 2010;85:746-748.
 
9. National Institutes of Health. Notice of modified AHRQ application submission and referral for appointed AHRQ study section and special emphasis panel members http://grants.nih.gov/grants/guide/notice-files/NOT-HS-14-006.html. Accessed July 20, 2015.