Exploratory Analysis of Quality Assessment of Putative Intrinsic Disorder in Proteins

Intrinsically disorder proteins are abundant in nature and can be accurately identified from sequences using computational predictors. While predictions of disorder are relatively easy to obtain there are no tools to assess their quality for a particular amino acid or protein. Quality assessment (QA) scores that quantify correctness of the predictions are not available. We define QA for the prediction of intrinsic disorder and use a large dataset of over 25 thousand proteins and ten modern predictors of disorder to empirically assess the first approach to quantify QA scores. We formulate the QA scores based on the readily available propensities of the intrinsic disorder generated by the ten methods. Our evaluation reveals that these QA scores offer good predictive performance for native structured residues (AUC > 0.74) and poor predictive performance for native disordered residues (AUC < 0.67). Specifically, we show that most of the native disordered residues that are incorrectly predicted as structured have high QA values that inaccurately suggest that these predictions are correct. Consequently, more research is needed to develop high-quality QA scores. We also outline three possible future research directions.