A new approach to grant review assessments: score, then rank

[1] Heyard R(2021)The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants Humanit Soc Sci Commun. 8 1-16

[2] Hottenrott H(2016)The 10 largest public and philanthropic funders of health research in the world: what they fund and how they distribute their funds Health Res Policy Syst. 14 1-15

[3] Viergever RF(2016)Looking across and looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science Manag Sci. 62 2765-2783

[4] Hendriks TC(2008)Statistical analysis of the National Institutes of Health peer review system Proc Natl Acad Sci. 105 11076-11080

[5] Boudreau KJ(2018)Low agreement among reviewers evaluating the same NIH grant applications Proc Natl Acad Sci. 115 2952-2957

[6] Guinan EC(2008)Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability Am Psychol. 63 160-135

[7] Lakhani KR(1991)The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation Behav Brain Sci. 14 119-886

[8] Riedl C(1981)Chance and consensus in peer review Science. 214 881-919

[9] Johnson VE(2021)When zero may not be zero: A cautionary note on the use of inter-rater reliability in evaluating grant peer review J R Stat Soc Ser A Stat Soc. 184 904-360

[10] Pier EL(2021)Meta-Research: A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018 Elife. 10 e59338-410

[1] Heyard R(2021)The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants Humanit Soc Sci Commun. 8 1-16

[2] Hottenrott H(2016)The 10 largest public and philanthropic funders of health research in the world: what they fund and how they distribute their funds Health Res Policy Syst. 14 1-15

[3] Viergever RF(2016)Looking across and looking beyond the knowledge frontier: Intellectual distance, novelty, and resource allocation in science Manag Sci. 62 2765-2783

[4] Hendriks TC(2008)Statistical analysis of the National Institutes of Health peer review system Proc Natl Acad Sci. 105 11076-11080

[5] Boudreau KJ(2018)Low agreement among reviewers evaluating the same NIH grant applications Proc Natl Acad Sci. 115 2952-2957

[6] Guinan EC(2008)Improving the peer-review process for grant applications: Reliability, validity, bias, and generalizability Am Psychol. 63 160-135

[7] Lakhani KR(1991)The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation Behav Brain Sci. 14 119-886

[8] Riedl C(1981)Chance and consensus in peer review Science. 214 881-919

[9] Johnson VE(2021)When zero may not be zero: A cautionary note on the use of inter-rater reliability in evaluating grant peer review J R Stat Soc Ser A Stat Soc. 184 904-360

[10] Pier EL(2021)Meta-Research: A retrospective analysis of the peer review of more than 75,000 Marie Curie proposals between 2007 and 2018 Elife. 10 e59338-410