On Evaluation of Segmentation-Free Word Spotting Approaches Without Hard Decisions

Word spotting systems are intended to retrieve occurrences of a given keyword in document images without actually recognizing the full document content. As there is a trend towards segmentation-free word spotting methods, we propose a methodology to evaluate these methods by employing measures that take the quality of the retrieved word locations into account without making hard decisions. We derive a desired evaluation behavior with the help of synthetic examples and show discrepancies of existing evaluation methods. New measures following this behavior are introduced and their differences exemplarily described. The proposed evaluation method is applied to a state-of-the-art word spotting approach.