UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Going Beyond Relevance: Role of effort in Information Retrieval

Verma, Manisha; (2017) Going Beyond Relevance: Role of effort in Information Retrieval. Doctoral thesis (Ph.D), UCL (University College London). Green open access

[thumbnail of phd-thesis.pdf]
phd-thesis.pdf - Submitted version

Download (4MB) | Preview


The primary focus of Information Retrieval (IR) systems has been to optimize for Relevance. Existing approaches to rank documents or evaluate IR systems does not account for “user effort”. Currently, judges only determine whether the information provided in a given document would satisfy the underlying information need in a query. The current mechanism of obtaining relevance judgments does not account for time and effort that an end user must put forth to consume its content. While a judge may spend a lot of time assessing a document, an impatient user may not devote the same amount of time and effort to consume its content. This problem is exacerbated on smaller devices like mobile. While on mobile or tablets, with limited interaction, users may not put in too much effort in finding information. This thesis characterizes and incorporates effort in Information Retrieval. Comparison of explicit and implicit relevance judgments across several datasets reveals that certain documents are marked relevant by the judges but are of low utility to an end user. Experiments indicate that document-level effort features can reliably predict the mismatch between dwell time and judging time of documents. Explicit and preference-based judgments were collected to determine which factors associated with effort agreed the most with user satisfaction. The ability to locate relevant information or findability was found to be in highest agreement with preference judgments. Findability judgments were also gathered to study the association of different annotator, query or document related properties with effort judgments. We also investigate how can existing systems be optimized for relevance and effort. Finally, we investigate the role of effort on smaller devices with the help of cost-benefit models.

Type: Thesis (Doctoral)
Qualification: Ph.D
Title: Going Beyond Relevance: Role of effort in Information Retrieval
Event: UCL (University College London)
Open access status: An open access version is available from UCL Discovery
Language: English
Keywords: Infomation retrieval.
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
URI: https://discovery.ucl.ac.uk/id/eprint/10046355
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item