UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling

Fradkin, Isaac; Nour, Matthew M; Dolan, Raymond J; (2023) Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging 10.1016/j.bpsc.2023.05.005. (In press). Green open access

[thumbnail of 1-s2.0-S2451902223001258-main.pdf]
Preview
Text
1-s2.0-S2451902223001258-main.pdf - Published Version

Download (1MB) | Preview

Abstract

BACKGROUND: Natural language processing (NLP) holds promise to transform psychiatric research and practice. A pertinent example is the success of NLP in the automatic detection of speech disorganization in formal thought disorder (FTD). However, we lack an understanding of precisely what common NLP metrics measure and how they relate to theoretical accounts of FTD. We propose tackling these questions by using deep generative language models to simulate FTD-like narratives by perturbing computational parameters instantiating theory-based mechanisms of FTD. METHODS: We simulated FTD-like narratives using Generative-Pretrained-Transformer-2 by either increasing word selection stochasticity or limiting the model's memory span. We then examined the sensitivity of common NLP measures of derailment (semantic distance between consecutive words or sentences) and tangentiality (how quickly meaning drifts away from the topic) in detecting and dissociating the 2 underlying impairments. RESULTS: Both parameters led to narratives characterized by greater semantic distance between consecutive sentences. Conversely, semantic distance between words was increased by increasing stochasticity, but decreased by limiting memory span. An NLP measure of tangentiality was uniquely predicted by limited memory span. The effects of limited memory span were nonmonotonic in that forgetting the global context resulted in sentences that were semantically closer to their local, intermediate context. Finally, different methods for encoding the meaning of sentences varied dramatically in performance. CONCLUSIONS: This work validates a simulation-based approach as a valuable tool for hypothesis generation and mechanistic analysis of NLP markers in psychiatry. To facilitate dissemination of this approach, we accompany the paper with a hands-on Python tutorial.

Type: Article
Title: Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling
Location: United States
Open access status: An open access version is available from UCL Discovery
DOI: 10.1016/j.bpsc.2023.05.005
Publisher version: https://doi.org/10.1016/j.bpsc.2023.05.005
Language: English
Additional information: © 2023 Society of Biological Psychiatry. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Keywords: Computational psychiatry, GPT-2, Natural language processing, Psychosis, Schizophrenia, Thought disorder
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience
URI: https://discovery.ucl.ac.uk/id/eprint/10173306
Downloads since deposit
29Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item