CONSIDERATION OF SUBJECTIVE EVALUATION METHOD FOR QUALITY OF IMAGE-CODING

Subjective evaluations of coded images are often made by using the double-stimulus continuous-quality-scale method of the ITU-R Recommendation (old CCIR Wireless Communications Group). In this paper, the reliability of the double-stimulus continuous-quality-scale method is investigated. Using this method, subjective evaluations of coded video sequences are made and the consistency of the test scores of the reference sequences is analyzed. Although it is desirable to have consistent reference test scores, results show that the test scores vary due to the influence of the quality of the video sequences being tested. As a result, dealing with image evaluation scores obtained from the double-stimulus continuous-quality-scale method (with scores made on an absolute scale) may be ill advised. Next, the reliability of data obtained from processing the differences between scores (relative scale scores) of reference and test images obtained from the double-stimulus continuous-quality-scale method is investigated. Results show that compared to this method, scores obtained from the modified EBU method (modified in the sense that the video presentation method differs from the standard EBU method) have higher reliability when impairments are small and are suitable for the subjective evaluation of coded images.