Albuquerque, Josmario; Rienties, Bart; Holmes, Wayne; Hlosta, Martin; (2025) From hype to evidence: exploring large language models for inter-group bias classification in higher education. Interactive Learning Environments , 33 (3) pp. 2332-2354. 10.1080/10494820.2024.2408554.

Preview

Text Albuquerque et al. - 2025 - From hype to evidence exploring large language models for inter-group bias classification in higher.pdf - Published Version Download (1MB) | Preview

Abstract

Inter-group biases can diminish student achievements in several ways. Yet, manually identifying those biases in vast learning texts is challenging because of their subtle nature. In the light of processing nuanced language, approaches based on large language models (LLMs) have emerged as promising mechanisms (e.g. ChatGPT). However, their potential for classifying bias in learning text seems under-explored. This study examines the ability of three LLMs (BERT, GPT, and PaLM) to classify inter-group bias within learning texts. Accordingly, 2024 sentences from 91 Higher Education (HE) courses at The Open University UK were assessed for potential biases by LLM-based approaches. Then, we used a sub-sample of sentences (n = 30) for comparison with student classifications. The models suggested varying degrees of biases within the larger sample (BERT = 49.5%; GPT = 10.2%; PaLM = 11.0%). However, varied agreement levels were found within the sub-samples, where Kappa values for model-to-model and model-to-human comparisons ranged from.0 to.77. This underscores the complexity of inter-group bias in learning texts where context and culture can be crucial. We discuss the relevance of those potential biases for learning settings and advocate for tools that are both context- and culture-aware, ensuring more inclusive learning experiences.

Download activity - last month

Export as CSVXMLJSON

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item