UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions

Isaacs, T; Thomson, RI; (2013) Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions. Language Assessment Quarterly , 10 (2) pp. 135-159. 10.1080/15434303.2013.769545. Green open access

[thumbnail of Isaacs_rater_experience_rating_scale_length.pdf]
Preview
Text
Isaacs_rater_experience_rating_scale_length.pdf

Download (421kB) | Preview

Abstract

This mixed-methods study examines the effects of rating scale length and rater experience on listeners' judgments of second-language (L2) speech. Twenty experienced and 20 novice raters, who were randomly assigned to 5-point or 9-point rating scale conditions, judged speech samples of 38 newcomers to Canada on numerical rating scales for comprehensibility, accentedness, and fluency. Results yielded high Cronbach's alpha coefficients and no group differences for rating scale length or rater experience. However, Rasch category probability plots revealed that raters had difficulty differentiating between scale steps, particularly in mid-scale range—a challenge that was exacerbated in the 9-point scale condition. Evidence from verbal protocols and posttask interviews suggested that experienced and novice raters adopted strategies to either draw on or offset their perceived experience with L2 speech in conducting their ratings. Implications for L2 pronunciation research are discussed, including the need to draw on raters' perceptions to develop a greater understanding of major listener perceptual constructs in L2 pronunciation and to move beyond adherence to the native speaking standard in data collection procedures.

Type: Article
Title: Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting research conventions
Open access status: An open access version is available from UCL Discovery
DOI: 10.1080/15434303.2013.769545
Publisher version: http://dx.doi.org/10.1080/15434303.2013.769545
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
UCL classification: UCL
UCL > Provost and Vice Provost Offices > School of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education
UCL > Provost and Vice Provost Offices > School of Education > UCL Institute of Education > IOE - Culture, Communication and Media
URI: https://discovery.ucl.ac.uk/id/eprint/1527145
Downloads since deposit
664Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item