UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Detecting errors in pick and place procedures

Bovo, R; Binetti, N; Brumby, DP; Julier, S; (2020) Detecting errors in pick and place procedures. In: Paternò, F and Oliver, N and Conati, C and Spano, LD and Tintarev, N, (eds.) IUI '20: Proceedings of the 25th International Conference on Intelligent User Interfaces. (pp. pp. 536-545). Association for Computing Machinery (ACM): New York, NY, USA. Green open access

[img]
Preview
Text
iui20a-sub1182-cam-i16.pdf - Accepted version

Download (1MB) | Preview

Abstract

Many human activities, such as manufacturing and assembly, are sequence-constrained procedural tasks (SPTs): they consist of a series of steps that must be executed in a specific spatial/temporal order. However, these tasks can be error prone - steps can be missed out, executed out-of-order, and repeated. The ability to automatically predict if a person is about to commit an error could greatly help in these cases. The prediction could be used, for example, to provide feedback to prevent mistakes or mitigate their effects. In this paper, we present a novel approach for real-time error prediction for multi-step sequence tasks which uses a minimum viable set of behavioural signals. We have three main contributions. The first we present an architecture for real-time error prediction based on task tracking and intent prediction. The second is to explore the effectiveness of using hand position and eye-gaze tracking for task tracking. We confirm that eye-gaze is more effective for intent prediction, hand tracking is more accurate for task tracking and that combining the two provides the best overall response. We show that using Hands and Gaze tracking data we can predict selection/placement errors with an F1 score of 97%, approximately 300ms before the error would occur. Finally, we discuss the application of this hand-gaze error detection architecture used in conjunction with head-mounted AR displays, to support industrial manual assembly.

Type: Proceedings paper
Title: Detecting errors in pick and place procedures
Event: 25th International Conference on Intelligent User Interfaces (IUI '20)
ISBN-13: 9781450371186
Open access status: An open access version is available from UCL Discovery
DOI: 10.1145/3377325.3377497
Publisher version: https://doi.org/10.1145/3377325.3377497
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Manual Assembly Procedures, Error Prediction, Human-centered design, User Intent prediction, Long-Short Term Memory, Intelligent Assistive Systems
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > UCL Interaction Centre
URI: https://discovery.ucl.ac.uk/id/eprint/10095655
Downloads since deposit
20Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item