Towards a formal framework for utility-oriented measurements of retrieval effectiveness

Marco Ferrante, Nicola Ferro, Maria Maistro

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningpeer review

28 Citationer (Scopus)

Abstract

In this paper we present a formal framework to define and study the properties of utility-oriented measurements of retrieval effectiveness, like AP, RBP, ERR and many other popular IR evaluation measures. The proposed framework is laid in the wake of the representational theory of measurement, which provides the foundations of the modern theory of measurement in both physical and social sciences, thus contributing to explicitly link IR evaluation to a broader context. The proposed framework is minimal, in the sense that it relies on just one axiom, from which other properties are derived. Finally, it contributes to a better understanding and a clear separation of what issues are due to the inherent problems in comparing systems in terms of retrieval effectiveness and what others are due to the expected numerical properties of a measurement.

OriginalsprogEngelsk
TitelICTIR 2015 - Proceedings of the 2015 ACM SIGIR International Conference on the Theory of Information Retrieval
Antal sider10
ForlagAssociation for Computing Machinery, Inc.
Publikationsdato27 sep. 2015
Sider21-30
ISBN (Elektronisk)9781450338332
DOI
StatusUdgivet - 27 sep. 2015
Udgivet eksterntJa
Begivenhed5th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2015 - Northampton, USA
Varighed: 27 sep. 201530 sep. 2015

Konference

Konference5th ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2015
Land/OmrådeUSA
ByNorthampton
Periode27/09/201530/09/2015
SponsorACM Special Interest Group on Information Retrieval (SIGIR)

Citationsformater