Jaber, Razan; Zhong, Sabrina; Kuoppamäki, Sanna; Hosseini, Aida; Gessinger, Iona; Brumby, Duncan; Cowan, Benjamin; Jaber, Razan; Zhong, Sabrina; Kuoppamäki, Sanna; Hosseini, Aida; Gessinger, Iona; Brumby, Duncan; Cowan, Benjamin; McMillan, Donald; - view fewer (2024) Cooking With Agents: Designing Context-aware Voice Interaction for Complex Tasks. In: CHI '24: Proceedings of the CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery (ACM)

Preview

Text Brumby_3613904.3642183.pdf Download (5MB) | Preview

Abstract

Voice Agents (VAs) are touted as being able to help users in complex tasks such as cooking and interacting as a conversational partner to provide information and advice while the task is ongoing. Through conversation analysis of 7 cooking sessions with a commercial VA, we identify challenges caused by a lack of contextual awareness leading to irrelevant responses, misinterpretation of requests, and information overload. Informed by this, we evaluated 16 cooking sessions with a wizard-led context-aware VA. We observed more fluent interaction between humans and agents, including more complex requests, explicit grounding within utterances, and complex social responses. We discuss reasons for this, the potential for personalisation, and the division of labour in VA communication and proactivity. Then, we discuss the recent advances in generative models and the VAs interaction challenges. We propose limited context awareness in VAs as a step toward explainable, explorable conversational interfaces.

Download activity - last month

Export as CSVJSONXML

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item