The good, the bad, the difficult, and the easy: Something wrong with information retrieval evaluation?

被引：0

作者：

Mizzaro, Stefano ^{[1
]}

机构：

[1] Univ Udine, Dept Math & Comp Sci, I-33100 Udine, Italy

来源：

ADVANCES IN INFORMATION RETRIEVAL | 2008年 / 4956卷

关键词：

evaluation; TREC; topic ease and difficulty;

D O I：

暂无

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

TREC-like evaluations do not consider topic ease and difficulty. However, it seems reasonable to reward good effectiveness on difficult topics more than good effectiveness on easy topics, and to penalize bad effectiveness on easy topics more than bad effectiveness on difficult topics. This paper shows how this approach leads to evaluation results that could be more reasonable, and that are different to some extent. I provide a general analysis of this issue, propose a novel framework, and experimentally validate a part of it.

引用

页码：642 / 646

页数：5