UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Deep Sequential Models for Task Satisfaction Prediction

Mehrotra, R; Awadallah, AH; Shokouhi, M; Yilmaz, E; Zitouni, I; El Kholy, A; Khabsa, M; (2017) Deep Sequential Models for Task Satisfaction Prediction. In: (Proceedings) ACM Conference on Information and Knowledge Management (CIKM). (pp. pp. 737-746). ACM Green open access

CIKM17-taskSAT.pdf - Published version

Download (1MB) | Preview


Detecting and understanding implicit signals of user satisfaction are essential for experimentation aimed at predicting searcher satisfaction. As retrieval systems have advanced, search tasks have steadily emerged as accurate units not only to capture searcher's goals but also in understanding how well a system is able to help the user achieve that goal. However, a major portion of existing work on modeling searcher satisfaction has focused on query level satisfaction. The few existing approaches for task satisfaction prediction have narrowly focused on simple tasks aimed at solving atomic information needs. In this work we go beyond such atomic tasks and consider the problem of predicting user's satisfaction when engaged in complex search tasks composed of many different queries and subtasks. We begin by considering holistic view of user interactions with the search engine result page (SERP) and extract detailed interaction sequences of their activity. We then look at query level abstraction and propose a novel deep sequential architecture which leverages the extracted interaction sequences to predict query level satisfaction. Further, we enrich this model with auxiliary features which have been traditionally used for satisfaction prediction and propose a unified multi-view model which combines the benefit of user interaction sequences with auxiliary features. Finally, we go beyond query level abstraction and consider query sequences issued by the user in order to complete a complex task, to make task level satisfaction predictions. We propose a number of functional composition techniques which take into account query level satisfaction estimates along with the query sequence to predict task level satisfaction. Through rigorous experiments, we demonstrate that the proposed deep sequential models significantly outperform established baselines at both query and task satisfaction prediction. Our findings have implications on metric development for gauging user satisfaction and on designing systems which help users accomplish complex search tasks.

Type: Proceedings paper
Title: Deep Sequential Models for Task Satisfaction Prediction
Event: ACM Conference on Information and Knowledge Management (CIKM)
Location: Singapore, SINGAPORE
Dates: 06 November 2017 - 10 November 2017
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3132847.3133001
Publisher version: https://doi.org/10.1145/3132847.3133001
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Science & Technology, Technology, Computer Science, Information Systems, Computer Science, Theory & Methods, Computer Science
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10070788
Downloads since deposit
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item