Exploratory Visualization Tool for the Continuous Evaluation of Information Retrieval Systems

被引：0

作者：

Gonzalez-Saez, Gabriela ^{[1
]}

Galuscakova, Petra ^{[1
]}

Deveaud, Romain ^{[2
]}

Goeuriot, Lorraine ^{[1
]}

Mulhem, Philippe ^{[1
]}

机构：

[1] Univ Grenoble Alpes, CNRS, Grenoble INP, LIG,Inst Engn, F-38000 Grenoble, France

[2] Qwant, Paris, France

来源：

PROCEEDINGS OF THE 46TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2023 | 2023年

基金：

奥地利科学基金会;

关键词：

information retrieval; continuous evaluation; visualization;

D O I：

10.1145/3539618.3591825

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

This paper introduces a novel visualization tool that facilitates the exploratory analysis of continuous evaluation for information retrieval systems. We base our analysis on score standardization and meta-analysis techniques applied to Information Retrieval evaluation. We present three functionalities: evaluation overview, delta evaluation, and meta-analysis applied to three perspectives: evaluation rounds, queries, and systems. To illustrate the use of the tool, we provide an example using the TREC-COVID test collection.

引用

页码：3220 / 3224

页数：5

共 18 条

[1]

Abdulghani Tamer, 2018, C RECH INF APPL CORI, DOI DOI 10.24348/CORIA.2018.PAPER21SHORT

[2] LongEval: Longitudinal Evaluation of Model Performance at CLEF 2023 [J].

Alkhalifa, Rabab ;

Bilal, Iman ;

Borkakoty, Hsuvas ;

Camacho-Collados, Jose ;

Deveaud, Romain ;

El-Ebshihy, Alaa ;

Espinosa-Anke, Luis ;

Gonzalez-Saez, Gabriela ;

Galuscakova, Petra ;

Goeuriot, Lorraine ;

Kochkina, Elena ;

Liakata, Maria ;

Loureiro, Daniel ;

Madabushi, Harish Tayyar ;

Mulhem, Philippe ;

Piroi, Florina ;

Popel, Martin ;

Servan, Christophe ;

Zubiaga, Arkaitz .

ADVANCES IN INFORMATION RETRIEVAL, ECIR 2023, PT III, 2023, 13982 :499-505

[3] RecDelta: An Interactive Dashboard for Cross-model Evaluation of Top-.. Recommendation [J].

Chiang, Yi-Shyuan ;

Liu, Yu-Ze ;

Tsai, Chen-Feng ;

Lou, Jing-Kai ;

Tsai, Ming-Feng ;

Wang, Chuan-Ju .

PROCEEDINGS OF THE 45TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '22), 2022, :3224-3228

[4] Continuous Result Delta Evaluation of IR Systems [J].

Gonzalez-Saez, Gabriela .

PROCEEDINGS OF THE 45TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL (SIGIR '22), 2022, :3493-3493

[5] Towards the Evaluation of Information Retrieval Systems on Evolving Datasets with Pivot Systems [J].

Gonzalez-Saez, Gabriela Nicole ;

Mulhem, Philippe ;

Goeuriot, Lorraine .

EXPERIMENTAL IR MEETS MULTILINGUALITY, MULTIMODALITY, AND INTERACTION, CLEF 2021, 2021, 12880 :91-102

[6] DiffIR: Exploring Differences in Ranking Models' Behavior [J].

Jose, Kevin Martin ;

Thong Nguyen ;

MacAvaney, Sean ;

Dalton, Jeffrey ;

Yates, Andrew .

SIGIR '21 - PROCEEDINGS OF THE 44TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 2021, :2595-2599

[7]

McKinney W., 2011, Python for High Performance and Scientific, VComput14, P1

[8]

Rorvig Mark E., 1999, NIST SPECIAL PUBLICA, V500- 246

[9]

Sakai T, 2016, PROCEEDINGS OF THE 2016 ACM INTERNATIONAL CONFERENCE ON THE THEORY OF INFORMATION RETRIEVAL, ICTIR 2016, P95, DOI 10.1145/2970398.2970399

[10]

Seabold S., 2010, 9 PYTH SCI C, P92, DOI 10.25080/Majora-92bf1922-011

← 1 2 →