Mikhailiuk, A;
Perez-Ortiz, M;
Mantiuk, R;
(2018)
Psychometric scaling of TID2013 dataset.
In: Atzori, L, (ed.)
Proceedings of the 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX).
IEEE: Cagliari, Italy.
Preview |
Text
mikhailiuk2018tid_psych_scaling.pdf - Accepted Version Download (18MB) | Preview |
Abstract
TID2013 is a subjective image quality assessment dataset with a wide range of distortion types and over 3000 images. The dataset has proven to be a challenging test for objective quality metrics. The dataset mean opinion scores were obtained by collecting pairwise comparison judgments using the Swiss tournament system, and averaging votes of observers. However, this approach differs from the usual analysis of multiple pairwise comparisons, which involves psychometric scaling of the comparison data using either Thurstone or Bradley-Terry models. In this paper we investigate how quality scores change when they are computed using such psychometric scaling instead of averaging vote counts. In order to properly scale TID2013 quality scores, we conduct four additional experiments of two different types, which we found necessary to produce a common quality scale: comparisons with reference images, and cross-content comparisons. We demonstrate on a fifth validation experiment that the two additional types of comparisons are necessary and in conjunction with psychometric scaling improve the consistency of quality scores, especially across images depicting different contents.
| Type: | Proceedings paper |
|---|---|
| Title: | Psychometric scaling of TID2013 dataset |
| Event: | 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX) |
| ISBN-13: | 9781538626054 |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.1109/QoMEX.2018.8463376 |
| Publisher version: | https://doi.org/10.1109/QoMEX.2018.8463376 |
| Language: | English |
| Additional information: | This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > UCL BEAMS UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Engineering Science > Dept of Computer Science |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10062446 |
Loading...
Loading...Loading...| 1. |  United States | 4 |
| 2. |  China | 2 |
| 3. |  Switzerland | 1 |
| 4. |  Ecuador | 1 |
| 5. |  France | 1 |
Archive Staff Only
 |
View Item |
