Performance Evaluation of MHC Class-I Binding Prediction Tools Based on an Experimentally Validated MHC-Peptide Binding Data Set

Knowing whether a protein can be processed and the resulting peptides presented by major histocompatibility complex (MHC) is highly important for immunotherapy design. MHC ligands can be predicted by in silico peptideMHC class-I binding prediction algorithms. However, prediction performance differs considerably, depending on the selected algorithm, MHC class-I type, and peptide length. We evaluated the prediction performance of 13 algorithms based on binding affinity data of 8- to 11-mer peptides derived from the HPV16 E6 and E7 proteins to the most prevalent human leukocyte antigen (HLA) types. Peptides from high to low predicted binding likelihood were synthesized, and their HLA binding was experimentally verified by in vitro competitive binding assays. Based on the actual binding capacity of the peptides, the performance of prediction algorithms was analyzed by calculating receiver operating characteristics (ROC) and the area under the curve (AROC). No algorithm outperformed others, but different algorithms predicted best for particular HLA types and peptide lengths. The sensitivity, specificity, and accuracy of decision thresholds were calculated. Commonly used decision thresholds yielded only 40% sensitivity. To increase sensitivity, optimal thresholds were calculated, validated, and compared. In order to make maximal use of prediction algorithms available online, we developed MHCcombine, a web application that allows simultaneous querying and output combination of up to 13 prediction algorithms. Taken together, we provide here an evaluation of peptide-MHC class-I binding prediction tools and recommendations to increase prediction sensitivity to extend the number of potential epitopes applicable as targets for immunotherapy.