Fradkin, Isaac;
Nour, Matthew M;
Dolan, Raymond J;
(2023)
Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling.
Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
10.1016/j.bpsc.2023.05.005.
(In press).
Preview |
Text
1-s2.0-S2451902223001258-main.pdf - Published Version Download (1MB) | Preview |
Abstract
BACKGROUND: Natural language processing (NLP) holds promise to transform psychiatric research and practice. A pertinent example is the success of NLP in the automatic detection of speech disorganization in formal thought disorder (FTD). However, we lack an understanding of precisely what common NLP metrics measure and how they relate to theoretical accounts of FTD. We propose tackling these questions by using deep generative language models to simulate FTD-like narratives by perturbing computational parameters instantiating theory-based mechanisms of FTD. METHODS: We simulated FTD-like narratives using Generative-Pretrained-Transformer-2 by either increasing word selection stochasticity or limiting the model's memory span. We then examined the sensitivity of common NLP measures of derailment (semantic distance between consecutive words or sentences) and tangentiality (how quickly meaning drifts away from the topic) in detecting and dissociating the 2 underlying impairments. RESULTS: Both parameters led to narratives characterized by greater semantic distance between consecutive sentences. Conversely, semantic distance between words was increased by increasing stochasticity, but decreased by limiting memory span. An NLP measure of tangentiality was uniquely predicted by limited memory span. The effects of limited memory span were nonmonotonic in that forgetting the global context resulted in sentences that were semantically closer to their local, intermediate context. Finally, different methods for encoding the meaning of sentences varied dramatically in performance. CONCLUSIONS: This work validates a simulation-based approach as a valuable tool for hypothesis generation and mechanistic analysis of NLP markers in psychiatry. To facilitate dissemination of this approach, we accompany the paper with a hands-on Python tutorial.
| Type: | Article |
|---|---|
| Title: | Theory-Driven Analysis of Natural Language Processing Measures of Thought Disorder Using Generative Language Modeling |
| Location: | United States |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1016/j.bpsc.2023.05.005 |
| Publisher version: | https://doi.org/10.1016/j.bpsc.2023.05.005 |
| Language: | English |
| Additional information: | © 2023 Society of Biological Psychiatry. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). |
| Keywords: | Computational psychiatry, GPT-2, Natural language processing, Psychosis, Schizophrenia, Thought disorder |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > UCL Queen Square Institute of Neurology > Imaging Neuroscience |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10173306 |
Archive Staff Only
 |
View Item |
