Advanced search
2 files | 10.07 MB Add to list

Framework for reproducible objective video quality research with case study on PSNR implementations

(2018) DIGITAL SIGNAL PROCESSING. 77. p.195-206
Author
Organization
Abstract
Reproducibility is an important and recurrent issue in objective video quality research because the presented algorithms are complex, depend on specific implementations in software packages or their parameters need to be trained on a particular, sometimes unpublished, dataset. Textual descriptions often lack the required detail and even for the simple Peak Signal to Noise Ratio (PSNR) several mutations exist for images and videos, in particular considering the choice of the peak value and the temporal pooling. This work presents results achieved through the analysis of objective video quality measures evaluated on a reproducible large scale database containing about 60,000 HEVC coded video sequences. We focus on PSNR, one of the most widespread measures, considering its two most common definitions. The sometimes largely different results achieved by applying the two definitions highlight the importance of the strict reproducibility of the research in video quality evaluation in particular. Reproducibility is also often a question of computational power and PSNR is a computationally inexpensive algorithm running faster than realtime. Complex algorithms cannot be reasonably developed and evaluated on the abovementioned 160 hours of video sequences. Therefore, techniques to select subsets of coding parameters are then introduced. Results show that an accurate selection can preserve the variety of the results seen on the large database but with much lower complexity. Finally, note that our SoftwareX accompanying paper presents the software framework which allows the full reproducibility of all the research results presented here, as well as how the same framework can be used to produce derived work for other measures or indexes proposed by other researchers which we strongly encourage for integration in our open framework.
Keywords
Video quality, Large-scale database, Objective video quality metric, Video coding

Downloads

  • (...).pdf
    • full text
    • |
    • UGent only
    • |
    • PDF
    • |
    • 5.52 MB
  • DS78 i.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 4.55 MB

Citation

Please use this url to cite or link to this publication:

MLA
Aldahdooh, Ahmed, et al. “Framework for Reproducible Objective Video Quality Research with Case Study on PSNR Implementations.” DIGITAL SIGNAL PROCESSING, vol. 77, 2018, pp. 195–206, doi:10.1016/j.dsp.2017.09.013.
APA
Aldahdooh, A., Masala, E., Van Wallendael, G., & Barkowsky, M. (2018). Framework for reproducible objective video quality research with case study on PSNR implementations. DIGITAL SIGNAL PROCESSING, 77, 195–206. https://doi.org/10.1016/j.dsp.2017.09.013
Chicago author-date
Aldahdooh, Ahmed, Enrico Masala, Glenn Van Wallendael, and Marcus Barkowsky. 2018. “Framework for Reproducible Objective Video Quality Research with Case Study on PSNR Implementations.” DIGITAL SIGNAL PROCESSING 77: 195–206. https://doi.org/10.1016/j.dsp.2017.09.013.
Chicago author-date (all authors)
Aldahdooh, Ahmed, Enrico Masala, Glenn Van Wallendael, and Marcus Barkowsky. 2018. “Framework for Reproducible Objective Video Quality Research with Case Study on PSNR Implementations.” DIGITAL SIGNAL PROCESSING 77: 195–206. doi:10.1016/j.dsp.2017.09.013.
Vancouver
1.
Aldahdooh A, Masala E, Van Wallendael G, Barkowsky M. Framework for reproducible objective video quality research with case study on PSNR implementations. DIGITAL SIGNAL PROCESSING. 2018;77:195–206.
IEEE
[1]
A. Aldahdooh, E. Masala, G. Van Wallendael, and M. Barkowsky, “Framework for reproducible objective video quality research with case study on PSNR implementations,” DIGITAL SIGNAL PROCESSING, vol. 77, pp. 195–206, 2018.
@article{8581448,
  abstract     = {{Reproducibility is an important and recurrent issue in objective video quality research because the presented algorithms are complex, depend on specific implementations in software packages or their parameters need to be trained on a particular, sometimes unpublished, dataset. Textual descriptions often lack the required detail and even for the simple Peak Signal to Noise Ratio (PSNR) several mutations exist for images and videos, in particular considering the choice of the peak value and the temporal pooling. This work presents results achieved through the analysis of objective video quality measures evaluated on a reproducible large scale database containing about 60,000 HEVC coded video sequences. We focus on PSNR, one of the most widespread measures, considering its two most common definitions. The sometimes largely different results achieved by applying the two definitions highlight the importance of the strict reproducibility of the research in video quality evaluation in particular. Reproducibility is also often a question of computational power and PSNR is a computationally inexpensive algorithm running faster than realtime. Complex algorithms cannot be reasonably developed and evaluated on the abovementioned 160 hours of video sequences. Therefore, techniques to select subsets of coding parameters are then introduced. Results show that an accurate selection can preserve the variety of the results seen on the large database but with much lower complexity. Finally, note that our SoftwareX accompanying paper presents the software framework which allows the full reproducibility of all the research results presented here, as well as how the same framework can be used to produce derived work for other measures or indexes proposed by other researchers which we strongly encourage for integration in our open framework.}},
  author       = {{Aldahdooh, Ahmed and Masala, Enrico and Van Wallendael, Glenn and Barkowsky, Marcus}},
  issn         = {{1051-2004}},
  journal      = {{DIGITAL SIGNAL PROCESSING}},
  keywords     = {{Video quality,Large-scale database,Objective video quality metric,Video coding}},
  language     = {{eng}},
  pages        = {{195--206}},
  title        = {{Framework for reproducible objective video quality research with case study on PSNR implementations}},
  url          = {{http://dx.doi.org/10.1016/j.dsp.2017.09.013}},
  volume       = {{77}},
  year         = {{2018}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: