QPP++ 2025: Query Performance Prediction and its Applications in the Era of Large Language Models
on the 47th European Conference on Information Retrieval (ECIR 2025)
6th to 10th April 2025, Lucca, Italy.

About the workshop

Query performance prediction (QPP), a.k.a. query difficulty prediction, is a key task in information retrieval (IR). QPP has been studied for decades in the IR community. The task of QPP is defined as estimating search effectiveness without human relevance judgments.

We are excited to announce the QPP++ 2025 workshop, titled "Query Performance Prediction and its Applications in the Era of Large Language Models," co-located with the 47th European Conference on Information Retrieval (ECIR 2025), from 6th to 10th April 2025 in Lucca, Italy.

In this workshop, we aim to bring together researchers and practitioners from academia and industry to discuss new perspectives on QPP in the era of large language models (LLMs). QPP++ 2025 is the continuation of the QPP++ 2023 workshop held in ECIR 2023.

Call for Papers

Themes and Topics

Given the rapid advancement of LLMs, we present examples of topics we encourage authors to explore in this workshop:

  • Predicting the performance of LLM-based retrievers/re-rankers, or generative AI systems. Most QPP studies still focus on predicting the performance of rankers based on small-scale pre-trained language models (e.g., BERT or T5). However, little work has explored predicting the performance of newly emerged LLM-based retrievers/re-rankers , or more broadly, LLM-based generative AI systems. Therefore, we aim to ad- dress questions including, but not limited to, the following:
    • To what extent does the performance of existing QPP methods (designed for retrievers/re-rankers using small-scale pre-trained languages) generalise to LLM-based retrievers and re-rankers?
    • How can we effectively predict the performance of emerging new re-ranking paradigms based on LLMs, such as pair-wise or list-wise re-rankers?
    • How can we predict the performance of generative AI systems? E.g., how to predict the text generation quality of an LLM in response to a prompt?
  • Leveraging the capabilities of LLMs to enhance QPP quality. LLMs have been applied to a wide range of natural language processing (NLP) and IR tasks, achieving numerous state-of-the-art results. However, few studies (e.g., QPP-GenRE) have explored leveraging LLM to model QPP. We aim to address questions including, but not limited to, the following:
    • What kind of features from LLMs (e.g., embedding) can we use for QPP?
    • How well a QPP model based on LLMs performs when predicting the ranking quality of an LLM-based retriever/re-ranker or the text generation quality of an LLM?
  • Applying QPP to benefit various downstream tasks. Most studies evaluate QPP methods only using metrics not directly correlated to downstream tasks, e.g., linear correlation coefficients. However, it is under-explored whether QPP methods evaluated in such a way can be effectively applied to benefit downstream tasks, especially when it comes to applying them to important tasks in the era of LLMs, e.g., retrieval-augmented generation (RAG). So we aim to address questions including, but not limited to, the following:
    • Which downstream tasks (especially in the era of LLMs) does QPP have the potential to benefit? E.g., in RAG, can QPP be used to determine when to rely on the retrieved documents? In LLM-based re-ranking, can QPP be used to determine query-specific re-ranking depths?
    • How exactly to use QPP to benefit those tasks?
  • Exploring QPP in the context of multi-modal content. Most studies focus on QPP in the context of text. However, research on QPP in the context of multi-modal content, such as images or even video, remains limited. We aim to address questions including, but not limited to, the following:
    • QPP for text-to-image search, image-to-image search, or image generation.
    • QPP in the context of video search/generation.
  • Exploring multilingual QPP. There’s limited research on QPP for languages other than English. How well do current QPP methods generalise to non-English languages?

Beyond the above aspects, we also welcome submissions on other QPP-related aspects. There is a need for deeper exploration in other QPP-related areas, including, but not limited to, the following: QPP for conversational search, recommendation systems, question answering, and fairness. We aim to stimulate discussions on these topics.

Paper type

We welcome manuscripts about any QPP-related subjects. We welcome a diverse range of submission types:

  • Original papers. We welcome paper submissions in various types, including research papers, position papers, reproducibility papers, survey papers, and data collection papers. All submissions should be ranged from 4 to 10 pages in length, including references.
  • Published papers. We welcome papers already accepted at top-tier conferences or in journals, e.g., SIGIR, CIKM, WWW, ECIR, ACL, EMNLP, TOIS, IP&M. Authors should submit 1-page abstract for a published paper, including the full reference where the paper was accepted.

Paper submission & selection

Note that submissions be written in English. For original papers, double submission is not allowed except for papers already on ArXiv.

The papers (.pdf format) should be submitted using the EasyChair submission system at https://easychair.org/conferences/?conf=qpp2025. The paper template can be found at https://ceur-ws.org/HOWTOSUBMIT.html.

The review process is single-blind. Each manuscript will be peer-reviewed by at least three programme committee (PC) members.

Authors of accepted papers will be invited to give oral presentations at the workshop. The accepted papers will be published in the CEUR-WS.org proceedings series. Since the proceedings only accept papers with a minimum of 4 pages, we will combine 1-page abstract submissions into one or more papers for publication.

Extension on TOIS Special Issue

The best papers submitted to the QPP++ 2025 workshop will be invited for an extension on the TOIS Special Issue on "Query Performance Prediction Towards Novel Information Retrieval Paradigms" . The authors of the best papers are invited to implement suggestions and comments provided by the other participants to the workshop and submit an extended version of their work that will be published on the Transactions on Information Systems (TOIS).

Recommended benchmarks & tools

To facilitate workshop discussions, we encourage (though it is not mandatory) authors to use common, publicly available benchmarks. This will help authors share their experimental experiences and insights more effectively. We also plan to set a discussion session for sharing experience on common datasets.

  • Image search: iQPP.

We also recommend open-source implementations of QPP methods:

Important Dates

  • Submission deadline: 24th February, 2025 (11:59 PM, AoE)
  • Acceptance notification: 18th March, 2025 (11:59 PM, AoE)
  • Camera ready: 25th March, 2025 (11:59 PM, AoE)
  • Workshop date: 10th April, 2025

Keynote Speakers

TBD

Workshop Schedule

TBD

QPP++ 2025 is a full-day workshop, which will be held on the last day on ECIR 2025 (10th April 2025) at the IMT School for Advanced Studies Lucca.

Workshop Organisers

Chuan Meng

Chuan Meng

University of Amsterdam

The Netherlands

c.meng@uva.nl

Guglielmo Faggioli

Guglielmo Faggioli

University of Padova

Italy

guglielmo.faggioli@dei.unipd.it

Mohammad Aliannejadi

Mohammad Aliannejadi

University of Amsterdam

The Netherlands

m.aliannejadi@uva.nl

Nicola Ferro

Nicola Ferro

University of Padova

Italy

ferro@dei.unipd.it

Josiane Mothe

Josiane Mothe

Université de Toulouse

France

josiane.mothe@irit.fr

Program Committee

TBD

Copyright © QPP++ 2025