What is the sensitivity and specificity of the peer review process?

被引:2
作者
Garcia, Jose A. [1 ]
Chamorro-Padial, Jorge [2 ]
Rodriguez-Sanchez, Rosa [1 ]
Fdez-Valdivia, J. [1 ]
机构
[1] Univ Granada, Dept Ciencias Comp & IA, CITIC UGR, Granada 18071, Spain
[2] Univ Granada, CITIC UGR, Granada, Spain
来源
ACCOUNTABILITY IN RESEARCH-ETHICS INTEGRITY AND POLICY | 2024年 / 31卷 / 04期
关键词
Peer review; sensitivity; specificity; Bayesian inference; mutual information; PUBLICATION BIAS; REPRESENTATION;
D O I
10.1080/08989621.2022.2122817
中图分类号
R-052 [医学伦理学];
学科分类号
0101 ; 120402 ;
摘要
In this paper, we introduce the concepts of sensitivity and specificity to mathematically describe the accuracy of the peer review process. Sensitivity refers to the probability that the final decision for a manuscript would be acceptance, provided the manuscript meets the journal standards required for publication (i.e., true positive rate). Specificity refers to the probability that the final decision would be rejection, provided the work does not meet the standards required for publication (i.e., true negative rate). Therefore, in the peer review process, sensitivity measures the ability to correctly accept manuscripts that meet the required standards (true positives) and specificity measures the ability to correctly reject manuscripts that do not meet those quality standards required for publication (true negatives). Sensitivity and specificity values can inform the editor under what conditions the outcome of a peer review process becomes more precise and, therefore, if this does not occur, when the editor must improve the analysis involved in processing the information received from reviewers' reports. Sensitivity and specificity understood in this way can promote the ethical conduct of peer review processes and improve the validity of manuscript editorial decisions.
引用
收藏
页码:305 / 326
页数:22
相关论文
共 28 条
[1]   PEER-REVIEW IS AN EFFECTIVE SCREENING PROCESS TO EVALUATE MEDICAL MANUSCRIPTS [J].
ABBY, M ;
MASSEY, MD ;
GALANDIUK, S ;
POLK, HC .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1994, 272 (02) :105-107
[2]   STATISTICS NOTES - DIAGNOSTIC-TESTS-1 - SENSITIVITY AND SPECIFICITY .3. [J].
ALTMAN, DG ;
BLAND, JM .
BRITISH MEDICAL JOURNAL, 1994, 308 (6943) :1552-1552
[3]  
Bornmann L., 2008, HUMAN ARCHITECTURE J, V6, P23
[4]   Scientific Peer Review [J].
Bornmann, Lutz .
ANNUAL REVIEW OF INFORMATION SCIENCE AND TECHNOLOGY, 2011, 45 :199-245
[5]   THE EVOLUTION OF EDITORIAL PEER-REVIEW [J].
BURNHAM, JC .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 1990, 263 (10) :1323-1329
[6]  
Chubin D. E., 1990, Peerless Science: Peer Review and US Science Policy
[7]  
Cover TM., 2006, ELEMENTS INFORM THEO, DOI [DOI 10.1002/047174882X, DOI 10.1002/0471200611.CH2]
[8]   Association between time interval to publication and statistical significance [J].
Dickersin, K ;
Olson, CM ;
Rennie, D ;
Cook, D ;
Flanagin, A ;
Zhu, Q ;
Reiling, J ;
Pace, B .
JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2002, 287 (21) :2829-2831
[9]   PUBLICATION BIAS AND CLINICAL-TRIALS [J].
DICKERSIN, K ;
CHAN, S ;
CHALMERS, TC ;
SACKS, HS ;
SMITH, H .
CONTROLLED CLINICAL TRIALS, 1987, 8 (04) :343-353
[10]   Scientific collaboration results in higher citation rates of published articles [J].
Figg, WD ;
Dunn, L ;
Liewehr, DJ ;
Steinberg, SM ;
Thurman, PW ;
Barrett, JC ;
Birkinshaw, J .
PHARMACOTHERAPY, 2006, 26 (06) :759-767