UCL Discovery
UCL home » Library Services » Electronic resources » UCL Discovery

The Reliability of the ITU-P.85 Standard for the Evaluation of Text-to-Speech Systems

Vazquez-Alvarez, Y; Huckvale, M; (2002) The Reliability of the ITU-P.85 Standard for the Evaluation of Text-to-Speech Systems. In: Proceedings of the 7th International Conference on Spoken Language Processing: ICSLP-2002. (pp. 329 - 332). ISCA: Denver, Colorado, USA. Green open access

[img]
Preview
Text
Huckvale_icslp02eval.pdf

Download (163kB) | Preview

Abstract

An evaluation of the reliability of the ITU-T P.85 recommended standard for the evaluation of voice output systems was conducted using six English TTS systems. The P.85 standard is based on mean-opinion-score judgements of a listening panel on a number of rating scales. The study looked at how the ranking of the six systems on the scales varied across four different text genres and across two listening sessions. Rankings were also compared with a much simpler pair-comparison test across genres and listening sessions. For the ITU test a large degree of correlation was found across scales, implying that these were not really testing different aspects of the systems. There were surprisingly similar results across sessions, implying that listeners were indeed making real judgements. In comparison, the pair comparison test gave (almost) identical rankings for systems with far less variability, making statistically significant comparisons between systems possible, even across genres.

Type: Proceedings paper
Title: The Reliability of the ITU-P.85 Standard for the Evaluation of Text-to-Speech Systems
Event: 7th International Conference on Spoken Language Processing: ICSLP-2002
Open access status: An open access version is available from UCL Discovery
Publisher version: https://www.isca-speech.org/archive/icslp_2002/i02...
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: text-to-speech systems, language, language processing
UCL classification: UCL
UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences
UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Speech, Hearing and Phonetic Sciences
URI: https://discovery.ucl.ac.uk/id/eprint/74328
Downloads since deposit
25Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item