Information mining in remote sensing image archives: System evaluation

In this paper, we present an algorithmic protocol for the evaluation of a content-based remote sensing image information mining system. In order to provide users fast access to the content of large image databases, the system is composed of two main modules. The first includes computationally intensive algorithms for off-line data ingestion in the archive, image feature extraction, and indexing. The second module consists of a graphical man-machine interface that manages the information fusion for interactive interpretation and the image information mining functions. According to the system architecture, the proposed evaluation methodology aims to determine the objective technical quality of the system and includes subjective human factors as well. Since the query performance of a content-based image retrieval system mainly depends on the datasets stored in the archive, we first analyze the complexity of image data. Then, we determine the accuracy of the interactive training that can be considered as a supervised Bayesian classification of the entire archive. Based on the stochastic nature of user-defined cover types, the system retrieves images using probabilistic measurements. The information quality of the queried results is measured by target and misclassified images, precision and recall, and the probability to forget and to overretrieve images. Since the queried images are the result of a number of interactions between user and system, we analyze the man-machine communication dialogue and the system operation, too. Finally, we compare the objective component of the evaluation protocol with the users' degree of satisfaction to point out the significance of the computed measurements.