Trozze, Arianna;
Davies, Toby;
Kleinberg, Bennett;
(2024)
Large language models in cryptocurrency securities cases: can a GPT model meaningfully assist lawyers?
Artificial Intelligence and Law
10.1007/s10506-024-09399-6.
(In press).
Preview |
Text
Davies_Trozze et al. - 2024 - Large language models in cryptocurrency securities.pdf - Published Version Download (1MB) | Preview |
Abstract
Large Language Models (LLMs) could be a useful tool for lawyers. However, empirical research on their effectiveness in conducting legal tasks is scant. We study securities cases involving cryptocurrencies as one of numerous contexts where AI could support the legal process, studying GPT-3.5’s legal reasoning and ChatGPT’s legal drafting capabilities. We examine whether a) GPT-3.5 can accurately determine which laws are potentially being violated from a fact pattern, and b) whether there is a difference in juror decision-making based on complaints written by a lawyer compared to ChatGPT. We feed fact patterns from real-life cases to GPT-3.5 and evaluate its ability to determine correct potential violations from the scenario and exclude spurious violations. Second, we had mock jurors assess complaints written by ChatGPT and lawyers. GPT-3.5’s legal reasoning skills proved weak, though we expect improvement in future models, particularly given the violations it suggested tended to be correct (it merely missed additional, correct violations). ChatGPT performed better at legal drafting, and jurors’ decisions were not statistically significantly associated with the author of the document upon which they based their decisions. Because GPT-3.5 cannot satisfactorily conduct legal reasoning tasks, it would be unlikely to be able to help lawyers in a meaningful way at this stage. However, ChatGPT’s drafting skills (though, perhaps, still inferior to lawyers) could assist lawyers in providing legal services. Our research is the first to systematically study an LLM’s legal drafting and reasoning capabilities in litigation, as well as in securities law and cryptocurrency-related misconduct.
| Type: | Article |
|---|---|
| Title: | Large language models in cryptocurrency securities cases: can a GPT model meaningfully assist lawyers? |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1007/s10506-024-09399-6 |
| Publisher version: | http://dx.doi.org/10.1007/s10506-024-09399-6 |
| Language: | English |
| Additional information: | Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. |
| Keywords: | Artificial Intelligence, Artificial intelligence (AI), ChatGPT, Computer Science, Computer Science, Computer Science, Cryptocurrency, Government & Law, Interdisciplinary Applications, Large language models (LLMs), Law, Science & Technology, Securities law, Social Sciences, Technology |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Security and Crime Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10194203 |
Archive Staff Only
 |
View Item |
