UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Multimodal Egocentric Analysis of Focused Interactions

Bano, S; Suveges, T; Zhang, J; McKenna, SJ; (2018) Multimodal Egocentric Analysis of Focused Interactions. IEEE Access , 6 pp. 37493-37505. 10.1109/ACCESS.2018.2850284. Green open access

[img]
Preview
Text
Bano_Multimodal egocentric analysis of focused interactions_VoR.pdf - Published version

Download (4MB) | Preview

Abstract

Continuous detection of social interactions from wearable sensor data streams has a range of potential applications in domains, including health and social care, security, and assistive technology. We contribute an annotated, multimodal data set capturing such interactions using video, audio, GPS, and inertial sensing. We present methods for automatic detection and temporal segmentation of focused interactions using support vector machines and recurrent neural networks with features extracted from both audio and video streams. The focused interaction occurs when the co-present individuals, having the mutual focus of attention, interact by first establishing the face-to-face engagement and direct conversation. We describe an evaluation protocol, including framewise, extended framewise, and event-based measures, and provide empirical evidence that the fusion of visual face track scores with audio voice activity scores provides an effective combination. The methods, contributed data set, and protocol together provide a benchmark for the future research on this problem.

Type: Article
Title: Multimodal Egocentric Analysis of Focused Interactions
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/ACCESS.2018.2850284
Publisher version: https://doi.org/10.1109/ACCESS.2018.2850284
Language: English
Additional information: © 2019 IEEE. This work is licensed under a Creative Commons Attribution 3.0 License (http://creativecommons.org/licenses/by/3.0/).
Keywords: Social interaction, egocentric sensing, multimodal analysis, temporal segmentation
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science
URI: https://discovery.ucl.ac.uk/id/eprint/10066375
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item