Kelly, Kate Tremain;
Richardson, Mary;
Isaacs, Talia;
(2022)
Critiquing the rationales for using comparative judgement: a call for clarity.
Assessment in Education: Principles, Policy & Practice
10.1080/0969594x.2022.2147901.
(In press).
Preview |
Text
Critiquing the rationales for using comparative judgement a call for clarity - Online version.pdf - Published Version Download (806kB) | Preview |
Abstract
Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the aggregate view of expert judges. We explore these contentions. We argue that the psychological underpinnings used to justify the method are superficially treated in the literature. We conceptualise and critique the notion that comparative judgment is ‘intrinsically valid’ due to its use of expert judges. We conclude that the rationales as presented by the comparative judgment literature are incomplete and inconsistent. We recommend that future work should clarify its position regarding the psychological underpinnings of comparative judgment, and if necessary present a more compelling case; for example, by integrating the comparative judgment literature with evidence from other fields.
| Type: | Article |
|---|---|
| Title: | Critiquing the rationales for using comparative judgement: a call for clarity |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1080/0969594x.2022.2147901 |
| Publisher version: | https://doi.org/10.1080/0969594X.2022.2147901 |
| Language: | English |
| Additional information: | © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way. |
| Keywords: | Comparative judgment, pairwise comparisons, validity, scoring, human judgements |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Culture, Communication and Media UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Curriculum, Pedagogy and Assessment |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10160096 |
Archive Staff Only
 |
View Item |
