Vazquez-Alvarez, Y;
Huckvale, M;
(2002)
The Reliability of the ITU-P.85 Standard for the Evaluation of Text-to-Speech Systems.
In:
Proceedings of the 7th International Conference on Spoken Language Processing: ICSLP-2002.
(pp. 329 - 332).
ISCA: Denver, Colorado, USA.
Preview |
Text
Huckvale_icslp02eval.pdf Download (163kB) | Preview |
Abstract
An evaluation of the reliability of the ITU-T P.85 recommended standard for the evaluation of voice output systems was conducted using six English TTS systems. The P.85 standard is based on mean-opinion-score judgements of a listening panel on a number of rating scales. The study looked at how the ranking of the six systems on the scales varied across four different text genres and across two listening sessions. Rankings were also compared with a much simpler pair-comparison test across genres and listening sessions. For the ITU test a large degree of correlation was found across scales, implying that these were not really testing different aspects of the systems. There were surprisingly similar results across sessions, implying that listeners were indeed making real judgements. In comparison, the pair comparison test gave (almost) identical rankings for systems with far less variability, making statistically significant comparisons between systems possible, even across genres.
| Type: | Proceedings paper |
|---|---|
| Title: | The Reliability of the ITU-P.85 Standard for the Evaluation of Text-to-Speech Systems |
| Event: | 7th International Conference on Spoken Language Processing: ICSLP-2002 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://www.isca-speech.org/archive/icslp_2002/i02... |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| Keywords: | text-to-speech systems, language, language processing |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Brain Sciences > Div of Psychology and Lang Sciences > Speech, Hearing and Phonetic Sciences |
| URI: | https://discovery.ucl.ac.uk/id/eprint/74328 |
Archive Staff Only
 |
View Item |
