Isaacs, Talia;
(2023)
Critiquing the “intrinsic validity” argument for
comparative judgement: A call for evidence.
Assessment in Education
10.1080/0969594X.2022.2147901.
(In press).
Preview |
PDF
Kelly et al. 2022.pdf - Published Version Download (806kB) | Preview |
Abstract
Comparative judgment is gaining popularity as an assessment tool, including for high-stakes testing purposes, despite relatively little research on the use of the technique. Advocates claim two main rationales for its use: that comparative judgment is valid because humans are better at comparative than absolute judgment, and because it distils the aggregate view of expert judges. We explore these contentions. We argue that the psychological underpinnings used to justify the method are superficially treated in the literature. We conceptualise and critique the notion that comparative judgment is ‘intrinsically valid’ due to its use of expert judges. We conclude that the rationales as presented by the comparative judgment literature are incomplete and inconsistent. We recommend that future work should clarify its position regarding the psychological underpinnings of comparative judgment, and if necessary present a more compelling case; for example, by integrating the comparative judgment literature with evidence from other fields.
| Type: | Article |
|---|---|
| Title: | Critiquing the “intrinsic validity” argument for comparative judgement: A call for evidence |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1080/0969594X.2022.2147901 |
| Publisher version: | https://doi.org/10.1080/0969594X.2022.2147901 |
| Language: | English |
| Additional information: | © 2022 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives License (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited, and is not altered, transformed, or built upon in any way |
| Keywords: | Comparative judgment; pairwise comparisons; validity; scoring; human judgements |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Culture, Communication and Media |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10160817 |
Archive Staff Only
 |
View Item |
