Sinkhorn Transformations for Single-Query Postprocessing in Text-Video Retrieval

被引：1

作者：

Yakovlev, Konstantin ^{[1
]}

Polyakov, Gregory ^{[1
]}

Alimova, Ilseyar ^{[1
]}

Podolskiy, Alexander ^{[1
]}

Bout, Andrey ^{[1
]}

Nikolenko, Sergey ^{[2
,3
]}

Piontkovskaya, Irina ^{[1
]}

机构：

[1] Huawei Noahs Ark Lab, Moscow, Russia

[2] RAS, Ivannikov Inst Syst Programming, Moscow, Russia

[3] Steklov Inst Math, St Petersburg Dept, St Petersburg, Russia

来源：

PROCEEDINGS OF THE 46TH INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, SIGIR 2023 | 2023年

关键词：

video-text retrieval; dual-softmax loss; Sinkhorn algorithm;

D O I：

10.1145/3539618.3592064

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

A recent trend in multimodal retrieval is related to postprocessing test set results via the dual-softmax loss (DSL). While this approach can bring significant improvements, it usually presumes that an entire matrix of test samples is available as DSL input. This work introduces a new postprocessing approach based on Sinkhorn transformations that outperforms DSL. Further, we propose a new postprocessing setting that does not require access to multiple test queries. We show that our approach can significantly improve the results of state of the art models such as CLIP4Clip, BLIP, X-CLIP, and DRL, thus achieving a new state-of-the-art on several standard text-video retrieval datasets both with access to the entire test set and in the single-query setting.

引用

页码：2394 / 2398

页数：5