Joseph, SA; (2014) Multi-sensory working memory - in vision, audition and touch. Doctoral thesis , UCL (University College London).

PDF SJoseph_thesis_final.pdf.REDACTED.pdf Download (9MB)

Abstract

Our nervous systems can perform a vast variety of cognitive tasks, many involving several different senses. Although sensory systems provide a basis for the creation of mental representations, we rely on memory to form mental representations of information that is no longer present in our external world. Focussing on the initial stage of this process, working memory (WM), where information is retained actively over a short time course, experiments included in this thesis were directed toward understanding the nature of sensory representations across the senses (vision, audition and touch). Instead of quantifying how many items one can hold in each sensory modality (all-or-none representations), new response methods were devised to capture the qualitative nature of sensory representations. Measuring quality rather than quantity of information held in WM, has led to the re-evaluation of the nature of its underlying capacity limits. Rather than assuming that WM capacity is limited to a fixed number of items, it may be more suitable to describe WM as a resource which can be shared and flexibly distributed across sensory information. Thus it has been proposed that at low loads we can hold information at a high resolution. However, as soon as memory load is increased, there is a deterioration of the quality at which each individual item can be represented in WM. The resource model of WM has been applied to describe processes of visual WM, but has not been investigated for other sensory modalities. In the first part of my thesis I demonstrate behaviourally that the resource model can be extended to account for processes in auditory WM, associated with the storage of sound frequency (pitch, chapter 2) and speech sounds (phonemes, chapter 3). I then show that it can also be extended to account for storage of tactile vibrational frequencies (chapter 4). Overall, the results suggest that memory representations become noisier with an increase in information load, consistent with the concept that representations are coded as distributed patterns. A pattern may code for individual object features or entire objects. As studies in chapter 2 - 4 only looked at a single type of feature each in separation, I next examined WM information storage for auditory objects, composed of multiple features (chapter 5). Object formation involves binding of features, which become reorganized to create more complex unified representations of previously distributed information. The results revealed a clear feature extraction cost when recall was tested on individual features rather than on integrated objects. One interpretation of these findings is that, at some level in the auditory system, sounds may be stored as integrated objects. In a final study, using fMRI with MVPA (mulitvoxel pattern analysis), memory traces represented as distributed patterns of brain activity were decoded from different regions of the auditory system (chapter 6). The major goal was to resolve the debate on the role of early sensory cortices in cognition: are they primarily involved in the perception of low-level stimulus features or also in maintenance of the same features in memory? I demonstrate that perception and memory share common neural substrates, where early auditory cortex serves as a substrate to accommodate both processes. Overall, in this thesis memory representations were characterized across the senses in three different ways: (1) measuring them in terms of their quality or resolution, (2) testing whether the preferred format is on the feature or integrated object level; and (3) as patterns of brain activity. Findings converge along the concept that noisy representations actively held in WM are coded as distributed patterns in the brain.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item