Quality assessment is becoming an important issue in the framework of image and video processing. Images are generally intended to be viewed by human observers and thus the consideration of the visual perception is an intrinsic aspect of the effective assessment of image quality. This observation has been made for different application domains such as printing, compression, transmission, and so on. Recently hundreds of research paper have proposed objective quality metrics dedicated to several image and video applications. With this abundance of quality tools, it is more than ever important to have a set of rules/methods allowing to assess the efficiency of a given metric. In this direction, technical groups such as VQEG (Video Quality Experts Group) or JPEG AIC (Advanced Image Coding) have focused their interest on the definition of test-plans to measure the impact of a metric. Following this wave in the image and video community, we propose in this paper a web-service or a web-application dedicated to the benchmark of quality metrics for image compression and open to all possible extensions. This application is intended to be the reference tool for the JPEG committee in order to ease the evaluation of new compression technologies. Also it is seen as a global help for our community to help researchers time while trying to evaluate their algorithms of watermarking, compression, enhancement, ... As an illustration of the web-application, we propose a benchmark of many well-known metrics on several image databases to provide a small overview of the possible use.
