On Evaluation the Quality of Subjective S3D Comfort Assessment

Stereoscopic 3D (S3D) image technology has been extensively developed in the last decades. Visual discomfort such as eye strain, headache, fatigue, asthenopia, and other phenomena leading to a less pleasant viewing experience is still a potential issue in S3D applications. How to evaluate S3D image quality that related to visual discomfort is still a challenging problem. A larger number of studies have been done on S3D Image Quality Assessment (S3D IQA) where the subjective assessed S3D Image Databases play an important role. The subjective scores were collected for each S3D image in database with a number of viewers. Usually, Likert scale is adopted for observers to mark their subjective quality score, and then mean opinion score (MOS) is estimated. Due to the law of comparative judgment, the quality of subjective scores varies among observers and depends on the judgment method. This paper studied the quality of two subjective assessment methodologies - single stimulus (SS) and pairwise comparison (PC). Considering the S3D IQA as a S3D images' quality ranking problem, we applied single stimulus and pairwise comparison subjective testing on a set of S3D images with known geometric distortions. From SS subjective testing results, the S3D images' ranking can be derived by sorting MOSs directly. From PC subjective testing results, the ranking can be derived from DMOS scores. The distorted S3D images can be ranked via their geometric distortion parameters. The quality of subjective assessed results from SS and PC are then evaluated on the correlation between their ranking results to corresponding geometric distortions. With the collected MOSs for geometric distorted S3D image database, a deep-learning based S3D IQA model was used to study the relationship between the model performance and the quality of subjective assessment.