This study evaluates 30 IR evaluation measures or their instances, of which nine are for
adhoc IR and 21 are for diversified IR, primarily from the viewpoint of whether their …

NDCG and similar measures remain standard for the offline evaluation of search,
recommendation, question answering and similar systems. These measures require …

A core step in production model research and development involves the offline evaluation of
a system before production deployment. Traditional offline evaluation of search …

While great strides have been made in the field of search and recommendation, there are
still challenges and opportunities to address information access issues that involve solving …

Most of information retrieval effectiveness evaluation metrics assume that systems
appending irrelevant documents at the bottom of the ranking are as effective as (or not …

Information Retrieval evaluation has traditionally focused on defining principled ways of
assessing the relevance of a ranked list of documents with respect to a query. Several …

In many search scenarios, such as exploratory, comparative, or survey-oriented search,
users interact with dynamic search systems to satisfy multi-aspect information needs. These …

Many real-world searches are examples of complex information needs such as exploratory,
comparative or survey oriented searches. In these search scenarios, users engage …

1 INTRODUCTION IR test collections are notoriously incomplete: not all documents are
assessed, and many more documents are assessed as non-relevant than as relevant. This …