Statistical analysis of the National Institutes of Health peer review system. Academic Article

abstract

• A statistical model is proposed for the analysis of peer-review ratings of R01 grant applications submitted to the National Institutes of Health. Innovations of this model include parameters that reflect differences in reviewer scoring patterns, a mechanism to account for the transfer of information from an application's preliminary ratings and group discussion to final ratings provided by all panel members and posterior estimates of the uncertainty associated with proposal ratings. Application of this model to recent R01 rating data suggests that statistical adjustments to panel rating data would lead to a 25% change in the pool of funded proposals. Viewed more broadly, the methodology proposed in this article provides a general framework for the analysis of data collected interactively from expert panels through the use of the Delphi method and related procedures.

authors

• Johnson, Valen

altmetric score

• 21.52

author list (cited authors)

• Johnson, V. E

citation count

• 43

complete list of authors

• Johnson, Valen E

publication date

• January 2008

publisher

• Proceedings of the National Academy of Sciences  Publisher

published in

• Proceedings of the National Academy of Sciences of the United States of America  Journal

keywords

• Biomedical Research
• Information Systems
• Models, Statistical
• National Institutes Of Health (U.S.)
• Peer Review, Research
• United States

Digital Object Identifier (DOI)

• 10.1073/pnas.0804538105

start page

• 11076

end page

• 11080

volume

• 105

issue

• 32

URL

• http%3A%2F%2Fdx.doi.org%2F10.1073%2Fpnas.0804538105