Robust query performance prediction for dense retrievers via adaptive disturbance generation

This paper introduces ADG-QPP (Adaptive Disturbance Generation), an unsupervised Query Performance Prediction (QPP) method designed specifically for dense neural retrievers. The underlying foundation of ADG-QPP is to measure query performance based on its degree of robustness towards perturbations. Traditional QPP methods rely on predefined lexical perturbations on the query, which only apply to sparse retrieval methods and fail to maintain consistent performance across different datasets. In our work, we address these limitations by perturbing the query by injecting disturbance leveraged by the focal network-based measurements including node-based, edge-based, and cluster-based metrics, into its neural embedding representation. Rather than applying the same perturbation across all queries, our approach develops an instance-wise disturbance for each query that is then used for its perturbation. Through extensive experiments on three benchmark datasets, we demonstrate that ADG-QPP outperforms state-of-the-art baselines in terms of Kendall tau\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document}, Spearman rho\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho$$\end{document}, and Pearson's rho\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho$$\end{document} correlations.