UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Economic behavior of information acquisition: Impact of peer grading in MOOCs

Yoo, OS; Zhan, D; (2021) Economic behavior of information acquisition: Impact of peer grading in MOOCs. Operations Research 10.1287/opre.2021.2131. (In press). Green open access

[thumbnail of OPRE-2018-05-288_R3_FINAL.pdf]
Preview
Text
OPRE-2018-05-288_R3_FINAL.pdf - Accepted Version

Download (1MB) | Preview

Abstract

A critical issue in operating massive open online courses (MOOCs) is the scalability of providing feedback. Because it is not feasible for instructors to grade a large number of students’ assignments, MOOCs use peer grading systems. This study investigates the efficacy of that practice when student graders are rational economic agents. We characterize grading as a process of (a) acquiring information to assess an assignment’s quality and (b) reporting a score. This process entails a tradeoff between the cost of acquiring information and the benefits of accurate grading. Because the true quality is not observable, any measure of inaccuracy must reference the actions of other graders, which motivates student graders to behave strategically. We present the unique equilibrium information level and reporting strategy of a homogeneous group of student graders and then examine the outcome of peer grading. We show how both the peer grading structure and the nature of MOOC courses affect peer grading accuracy, and we identify conditions under which the process fails. There is a systematic grading bias toward the mean, which discourages students from learning. To improve current practice, we introduce a scale-shift grading scheme, theoretically examine how it can improve grading accuracy and adjust grading bias and discuss how it can be practically implemented.

Type: Article
Title: Economic behavior of information acquisition: Impact of peer grading in MOOCs
Open access status: An open access version is available from UCL Discovery
DOI: 10.1287/opre.2021.2131
Publisher version: https://doi.org/10.1287/opre.2021.2131
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher's terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > UCL School of Management
URI: https://discovery.ucl.ac.uk/id/eprint/10120633
Downloads since deposit
0Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item