access icon free Macro-pixel-wise CNN-based filtering for quality enhancement of light field images

© The Institution of Engineering and Technology
Received 10/08/2020, Published 04/11/2020

The Letter introduces a novel filtering method based on convolutional neural networks (CNNs) for quality enhancement of light field (LF) images captured by a plenoptic camera and compressed using high-efficiency video coding (HEVC). The method takes advantage of the macro-pixel (MP) structure specific to the LF images and proposes a novel MP-wise filtering approach based on a novel deep neural network architecture. The proposed CNN-based method achieves an outstanding performance when HEVC is employed without its in-loop filters. Experimental results show high luminance-peak signal-to-noise ratio (Y-PSNR) gains and average Y-Bjøntegaard delta (BD)-rate savings of over HEVC on a large data set.

Inspec keywords: filtering theory; video coding; convolutional neural nets; image enhancement; cameras

Other keywords: convolutional neural networks; novel MP-wise filtering approach; macro-pixel structure; novel filtering method; deep neural network architecture; quality enhancement; in-loop filters; macro-pixel-wise CNN-based; light field images; high-efficiency video coding; HEVC; plenoptic camera; LF images; CNN-based method

Subjects: Neural computing techniques; Computer vision and image processing techniques; Image sensors; Image and video coding; Video signal processing; Filtering methods in signal processing

References

    1. 1)
      • 1. Adelson, E.H., Bergen, J.R.: ‘The plenoptic function and the elements of early vision’ in Landy, M., Movshon, J.A. (Eds): ‘Computational models of visual processing’ (MIT Press, Cambridge, MA, USA, 1991), pp. 3–20.
    2. 2)
    3. 3)
    4. 4)
      • 5. Schelkens, P., Astola, P., da Silva, E. A. B., et al: ‘JPEG Pleno light field coding technologies’, in Tescher, A.G., Ebrahimi, T (Eds): ‘Applications of Digital Image Processing XLII, Proceedings of SPIE’ (SPIE, San Diego, CA, USA, 2019) Vol. 11137, article no. 111371G.
    5. 5)
      • 8. Schiopu, I., Huang, H., Munteanu, A.: ‘CNN-based intra-prediction for lossless HEVC’, IEEE Trans. Circuits Syst. Video Technol., 2020, 30, (7), pp. 1816–1828.
    6. 6)
      • 7. Huang, H., Schiopu, I., Munteanu, A.: ‘Deep learning based angular intra-prediction for lossless HEVC video coding’. Data Compression Conf., Snowbird, UT, March 2019, p. 579.
    7. 7)
      • 3. Liu, D., Wang, L., Li, L., et al: ‘Pseudo-sequence-based light field image compression’. IEEE Int. Conf. on Multimedia & Expo Workshops, Seattle, WA, 2016, pp. 1–4.
    8. 8)
      • 9. Rerabek, M., Ebrahimi, T.: ‘New light field image dataset’. Int. Workshop on Quality of Multimedia Experience, Lisbon, Portugal, 2016.
    9. 9)
      • 10. Schiopu, I., Gabbouj, M., Gotchev, A., et al: ‘Lossless compression of subaperture images using context modeling’. 3DTV Conf.: The True Vision – Capture, Transmission and Display of 3D Video, Copenhagen, Denmark, June 2017, pp. 1–4.
    10. 10)
      • 6. Bossen, F.: ‘Common HM test conditions and software reference configurations’. JCT-VC document JCTVC-L1100, Geneva, Switzerland, January 2013, pp. 1–4.
http://iet.metastore.ingenta.com/content/journals/10.1049/el.2020.2344
Loading

Related content

content/journals/10.1049/el.2020.2344
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading