Beware of R2: Simple, Unambiguous Assessment of the Prediction Accuracy of QSAR and QSPR Models

被引:523
作者
Alexander, D. L. J. [1 ]
Tropsha, A. [2 ]
Winkler, David A. [3 ,4 ,5 ,6 ]
机构
[1] CSIRO Digital Product Flagship, Clayton, Vic 3169, Australia
[2] Univ N Carolina, UNC Eshelman Sch Pharm, Chapel Hill, NC 27599 USA
[3] CSIRO Mfg Flagship, Clayton, Vic 3168, Australia
[4] Monash Inst Pharmaceut Sci, Parkville, Vic 3052, Australia
[5] Latrobe Inst Mol Sci, Bundoora, Vic 3046, Australia
[6] Flinders Univ S Australia, Sch Chem & Phys Sci, Bedford Pk, SA 5042, Australia
关键词
DESCRIPTOR SELECTION; EXTERNAL VALIDATION; R(M)(2) METRICS; ORIGIN USEFUL; REGRESSION;
D O I
10.1021/acs.jcim.5b00206
中图分类号
R914 [药物化学];
学科分类号
100701 ;
摘要
The statistical metrics used to characterize the external predictivity of a model, i.e., how well it predicts the properties of an independent test set, have proliferated over the past decade. This paper clarifies some apparent confusion over the use of the coefficient of determination, R-2, as a measure of model fit and predictive power in QSAR and QSPR modeling. R-2 (or r(2))has been used in various contexts in the literature in conjunction with training and test data for both ordinary linear regression and regression through the origin as well as with linear and nonlinear regression models. We analyze the widely adopted model fit criteria suggested by Golbraikh and Tropsha (J. Mol. Graphics Modell. 2002 , 20 , 269-276) in a strict statistical manner. Shortcomings in these criteria are identified, and a clearer and simpler alternative method to characterize model predictivity is provided. The intent is not to repeat the well-documented arguments for model validation using test data but rather to guide the application of R-2 as a model fit statistic. Examples are used to illustrate both correct and incorrect uses of R-2. Reporting the root-mean-square error or equivalent measures of dispersion, which are typically of more practical importance than R-2, is also encouraged, and important challenges in addressing the needs of different categories of users such as computational chemists, experimental scientists, and regulatory decision support specialists are outlined.
引用
收藏
页码:1316 / 1322
页数:7
相关论文
共 21 条
  • [1] Trends and plot methods in MLR studies
    Besalu, Emili
    de Julian-Ortiz, Jesus V.
    Pogliani, Lionello
    [J]. JOURNAL OF CHEMICAL INFORMATION AND MODELING, 2007, 47 (03) : 751 - 760
  • [2] Optimal Sparse Descriptor Selection for QSAR Using Bayesian Methods
    Burden, F. R.
    Winkler, D. A.
    [J]. QSAR & COMBINATORIAL SCIENCE, 2009, 28 (6-7): : 645 - 653
  • [3] Robust QSAR models using Bayesian regularized neural networks
    Burden, FR
    Winkler, DA
    [J]. JOURNAL OF MEDICINAL CHEMISTRY, 1999, 42 (16) : 3183 - 3187
  • [4] Burden Frank, 2008, V458, P25
  • [5] An Optimal Self-Pruning Neural Network and Nonlinear Descriptor Selection in QSAR
    Burden, Frank R.
    Winkler, David A.
    [J]. QSAR & COMBINATORIAL SCIENCE, 2009, 28 (10): : 1092 - 1097
  • [6] Beware of q2!
    Golbraikh, A
    Tropsha, A
    [J]. JOURNAL OF MOLECULAR GRAPHICS & MODELLING, 2002, 20 (04) : 269 - 276
  • [7] Hastie T., 2009, ELEMENTS STAT LEARNI, P745, DOI [DOI 10.1007/978-0-387-21606-5, DOI 10.1007/978-0-387-84858-7, 10.1111/j.1467-985X.2010.00646_6.x, DOI 10.1111/J.1467-985X.2010.00646_6.X]
  • [8] Kendall M.G, 1970, RANK CORRELATION MET
  • [9] Three-dimensional quantitative similarity-activity relationships (3D QSiAR) from SEAL similarity matrices
    Kubinyi, H
    Hamprecht, FA
    Mietzner, T
    [J]. JOURNAL OF MEDICINAL CHEMISTRY, 1998, 41 (14) : 2553 - 2564
  • [10] CAUTIONARY NOTE ABOUT R2
    KVALSETH, TO
    [J]. AMERICAN STATISTICIAN, 1985, 39 (04) : 279 - 285