UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data

Kodagoda, N; Pontis, S; Simmie, D; Attfield, S; Wong, BLW; Blandford, A; Hankin, C; (2017) Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data. Journal of Cognitive Engineering and Decision Making , 11 (1) pp. 23-41. 10.1177/1555343416672782. Green open access

[thumbnail of Kodagoda_Using_Machine_Learning.pdf]
Preview
Text
Kodagoda_Using_Machine_Learning.pdf - Accepted Version

Download (978kB) | Preview

Abstract

The reconstruction of analysts’ reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos’s data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

Type: Article
Title: Using Machine Learning to Infer Reasoning Provenance from User Interaction Log Data
Open access status: An open access version is available from UCL Discovery
DOI: 10.1177/1555343416672782
Publisher version: https://doi.org/10.1177/1555343416672782
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: analytic provenance, reasoning provenance, data provenance, sensemaking, data/frame model, interaction frame mapper, machine learning
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/1542698
Downloads since deposit
302Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item