UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Sight and sound out of synch: fragmentation and renormalisation of audiovisual integration and subjective timing

Freeman, ED; Ipser, A; Palmbaha, A; Paunoiu, D; Brown, P; Lambert, C; Leff, A; (2013) Sight and sound out of synch: fragmentation and renormalisation of audiovisual integration and subjective timing. Cortex , 49 (10) pp. 2875-2887. 10.1016/j.cortex.2013.03.006. Green open access

[thumbnail of Leff_1-s2.0-S0010945213000890-main.pdf]
Preview
Text
Leff_1-s2.0-S0010945213000890-main.pdf

Download (1MB) | Preview

Abstract

The sight and sound of a person speaking or a ball bouncing may seem simultaneous, but their corresponding neural signals are spread out over time as they arrive at different multisensory brain sites. How subjective timing relates to such neural timing remains a fundamental neuroscientific and philosophical puzzle. A dominant assumption is that temporal coherence is achieved by sensory resynchronisation or recalibration across asynchronous brain events. This assumption is easily confirmed by estimating subjective audiovisual timing for groups of subjects, which is on average similar across different measures and stimuli, and approximately veridical. But few studies have examined normal and pathological individual differences in such measures. Case PH, with lesions in pons and basal ganglia, hears people speak before seeing their lips move. Temporal order judgements (TOJs) confirmed this: voices had to lag lip-movements (by ∼200 msec) to seem synchronous to PH. Curiously, voices had to lead lips (also by ∼200 msec) to maximise the McGurk illusion (a measure of audiovisual speech integration). On average across these measures, PH's timing was therefore still veridical. Age-matched control participants showed similar discrepancies. Indeed, normal individual differences in TOJ and McGurk timing correlated negatively: subjects needing an auditory lag for subjective simultaneity needed an auditory lead for maximal McGurk, and vice versa. This generalised to the Stream-Bounce illusion. Such surprising antagonism seems opposed to good sensory resynchronisation, yet average timing across tasks was still near-veridical. Our findings reveal remarkable disunity of audiovisual timing within and between subjects. To explain this we propose that the timing of audiovisual signals within different brain mechanisms is perceived relative to the average timing across mechanisms. Such renormalisation fully explains the curious antagonistic relationship between disparate timing estimates in PH and healthy participants, and how they can still perceive the timing of external events correctly, on average.

Type: Article
Title: Sight and sound out of synch: fragmentation and renormalisation of audiovisual integration and subjective timing
Location: Italy
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.cortex.2013.03.006
Publisher version: http://dx.doi.org/10.1016/j.cortex.2013.03.006
Language: English
Additional information: Copyright © 2014 The Authors. Published by Elsevier Ltd. Open Access funded by Wellcome Trust. Published under a Creative Commons license (http://creativecommons.org/licenses/by/3.0/).
Keywords: Audiovisual integration, Illusions, Individual differences, Psychophysics, Sensory timing, Acoustic Stimulation, Adolescent, Adult, Aged, Aging, Algorithms, Attention, Auditory Perception, Basal Ganglia, Cognition Disorders, Computer Simulation, Diffusion Tensor Imaging, Female, Humans, Illusions, Image Processing, Computer-Assisted, Intelligence Tests, Magnetic Resonance Imaging, Male, Middle Aged, Myasthenia Gravis, Photic Stimulation, Pons, Psychometrics, Space Perception, Speech Perception, Visual Perception, Young Adult
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Brain Repair and Rehabilitation
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/1393910
Downloads since deposit
86Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item