Giulianelli, Mario; Frisch, Ivar; (2024) LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models. In: Deshpande, Ameet and Hwang, EunJeong and Murahari, Vishvak and ParK, Joon Sung and Yang, Diyi and Sabharwal, Ashish and Narasimhan, Karthik and Kalyan, Ashwin, (eds.) (Proceedings) 1st Workshop on Personalization of Generative AI Systems (PERSONALIZE 2024). (pp. pp. 102-111). Association for Computational Linguistics

Preview

Text 2024.personalize-1.9.pdf - Published Version Download (451kB) | Preview

Abstract

While both agent interaction and personalisation are vibrant topics in research on large language models (LLMs), there has been limited focus on the effect of language interaction on the behaviour of persona-conditioned LLM agents. Such an endeavour is important to ensure that agents remain consistent to their assigned traits yet are able to engage in open, naturalistic dialogues. In our experiments, we condition GPT-3.5 on personality profiles through prompting and create a twogroup population of LLM agents using a simple variability-inducing sampling algorithm. We then administer personality tests and submit the agents to a collaborative writing task, finding that different profiles exhibit different degrees of personality consistency and linguistic alignment to their conversational partners. Our study seeks to lay the groundwork for better understanding of dialogue-based interaction between LLMs and highlights the need for new approaches to crafting robust, more human-like LLM personas for interactive environments.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as JSONXMLCSV

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item