Discussion on Comparing Machine Learning Models for Health Outcome Prediction

This position paper argues the need for more details than simple statistical accuracy measures when comparing machine learning models constructed for patient outcome prediction. First, statistical accuracy measures are briefly discussed, including AROC, APRC, predictive accuracy, precision, recall, and their variants. Then, model correlation plots are introduced that compare outputs from two models. Finally, a more detailed analysis of inputs to the models is presented. The discussions are illustrated with two classification problems in predicting patient mortality and high utilization of medical services.