Given the rapid advancement of LLMs, we present examples of topics we encourage authors to explore in this workshop:

• Predicting the performance of LLM-based retrievers/re-rankers, or generative AI systems. Most QPP studies still focus on predicting the performance of rankers based on small-scale pre-trained language models (e.g., BERT or T5). However, little work has explored predicting the performance of newly emerged LLM-based retrievers/re-rankers , or more broadly, LLM-based generative AI systems. Therefore, we aim to ad- dress questions including, but not limited to, the following:
• To what extent does the performance of existing QPP methods (designed for retrievers/re-rankers using small-scale pre-trained languages) generalise to LLM-based retrievers and re-rankers?
• How can we effectively predict the performance of emerging new re-ranking paradigms based on LLMs, such as pair-wise or list-wise re-rankers?
• How can we predict the performance of generative AI systems? E.g., how to predict the text generation quality of an LLM in response to a prompt?

• Leveraging the capabilities of LLMs to enhance QPP quality. LLMs have been applied to a wide range of natural language processing (NLP) and IR tasks, achieving numerous state-of-the-art results. However, few studies (e.g., QPP-GenRE) have explored leveraging LLM to model QPP. We aim to address questions including, but not limited to, the following:
• What kind of features from LLMs (e.g., embedding) can we use for QPP?
• How well a QPP model based on LLMs performs when predicting the ranking quality of an LLM-based retriever/re-ranker or the text generation quality of an LLM?

• Applying QPP to benefit various downstream tasks. Most studies evaluate QPP methods only using metrics not directly correlated to downstream tasks, e.g., linear correlation coefficients. However, it is under-explored whether QPP methods evaluated in such a way can be effectively applied to benefit downstream tasks, especially when it comes to applying them to important tasks in the era of LLMs, e.g., retrieval-augmented generation (RAG). So we aim to address questions including, but not limited to, the following:
• Which downstream tasks (especially in the era of LLMs) does QPP have the potential to benefit? E.g., in RAG, can QPP be used to determine when to rely on the retrieved documents? In LLM-based re-ranking, can QPP be used to determine query-specific re-ranking depths?
• How exactly to use QPP to benefit those tasks?

• Exploring QPP in the context of multi-modal content. Most studies focus on QPP in the context of text. However, research on QPP in the context of multi-modal content, such as images or even video, remains limited. We aim to address questions including, but not limited to, the following:
• QPP for text-to-image search, image-to-image search, or image generation.
• QPP in the context of video search/generation.

• Exploring multilingual QPP. There’s limited research on QPP for languages other than English. How well do current QPP methods generalise to non-English languages?

Beyond the above aspects, we also welcome submissions on other QPP-related aspects. There is a need for deeper exploration in other QPP-related areas, including, but not limited to, the following: QPP for conversational search, recommendation systems, question answering, and fairness. We aim to stimulate discussions on these topics.