Gu, Yan; Donnellan, Ed; Grzyb, Beata; Murgiano, Margherita; Brieke, Ricarda; Perniss, Pamela; Vigliocco, Gabriella; (2025) The ECOLANG Multimodal Corpus of adult-child and adult-adult Language. Scientific data 10.1038/s41597-025-04405-1. (In press).

Text ECOLANG_SciData_accepted_final_version.pdf - Accepted Version Access restricted to UCL open access staff until 16 July 2025. Download (2MB)

Abstract

Communication comprises a wealth of multimodal signals (e.g., gestures, eyegaze, intonation) in addition to speech and there is a growing interest in the study of multimodal language by psychologists, linguists, neuroscientists and computer scientists. The ECOLANG corpus provides audiovisual recordings and ELAN annotations of multimodal behaviours (speech transcription, gesture, object manipulation, and eye gaze) by British and American English-speaking adults engaged in semi-naturalistic conversation with their child (N = 38, children 3-4 years old, face-blurred) or a familiar adult (N = 31). Speakers were asked to talk about objects to their interlocutors. We further manipulated whether the objects were familiar or novel to the interlocutor and whether the objects could be seen and manipulated (present or absent) during the conversation. These conditions reflect common interaction scenarios in realworld communication. Thus, ECOLANG provides ecologically-valid data about the distribution and co-occurrence of the multimodal signals across these conditions for cognitive scientists and neuroscientists interested in addressing questions concerning real-world language acquisition, production and comprehension, and for computer scientists to develop multimodal language models and more human-like artificial agents.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as XMLJSONCSV

Downloads by country - last 12 months

Export as XMLCSVJSON

Archive Staff Only

View Item