UCL logo

UCL Discovery

UCL home » Library Services » Electronic resources » UCL Discovery

On Hypothesis Testing for Comparing Image Quality Assessment Metrics

Zhu, R; Zhou, F; Yang, W; Xue, J-H; (2018) On Hypothesis Testing for Comparing Image Quality Assessment Metrics. IEEE Signal Processing Magazine , 35 (4) pp. 133-136. 10.1109/MSP.2018.2829209. Green open access

[img]
Preview
Text
RuiZhu-SPM-2018-UCL.pdf - ["content_typename_Accepted version" not defined]

Download (598kB) | Preview

Abstract

In developing novel image quality assessment (IQA) metrics, researchers should compare their proposed metrics with state-of-the-art metrics. A commonly adopted approach is by comparing two residuals between the nonlinearly mapped scores of two IQA metrics and the difference mean opinion score, which are assumed from Gaussian distributions with zero means. An F-test is then used to test the equality of variances of the two sets of residuals. If the variances are significantly different, then we conclude that the residuals are from different Gaussian distributions and that the two IQA metrics are significantly different. The F-test assumes that the two sets of residuals are independent. However, given that the IQA metrics are calculated on the same database, the two sets of residuals are paired and may be correlated. We note this improper usage of the F-test by practitioners, which can result in misleading comparison results of two IQA metrics. To solve this practical problem, we introduce the Pitman test to investigate the equality of variances for two sets of correlated residuals. Experiments on the Laboratory for Image and Video Engineering (LIVE) database show that the two tests can provide different conclusions.

Type: Article
Title: On Hypothesis Testing for Comparing Image Quality Assessment Metrics
Open access status: An open access version is available from UCL Discovery
DOI: 10.1109/MSP.2018.2829209
Publisher version: https://doi.org/10.1109/MSP.2018.2829209
Language: English
Additional information: This version is the author accepted manuscript. For information on re-use, please refer to the publisher’s terms and conditions.
Keywords: Image quality, Correlation, Gaussian distribution, Gaussian noise, Testing, Measurement
UCL classification: UCL > Provost and Vice Provost Offices
UCL > Provost and Vice Provost Offices > UCL BEAMS
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences
UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of Maths and Physical Sciences > Dept of Statistical Science
URI: http://discovery.ucl.ac.uk/id/eprint/10054459
Downloads since deposit
27Downloads
Download activity - last month
Download activity - last 12 months
Downloads by country - last 12 months

Archive Staff Only

View Item View Item