access icon free UCT-GAN: underwater image colour transfer generative adversarial network

Underwater image enhancement algorithms improve image quality and indirectly enhance underwater visibility. Although many underwater image enhancement neural networks have been proposed, they require large amounts of data. To reduce the amount of data required while providing better image enhancement, this study proposes an underwater image colour transfer generative adversarial network (UCT-GAN). The authors first design a non-linear mapping function to generate colour cast images according to original images. Then, the authors utilise these image pairs (i.e. colour cast images and corresponding original images) to guide the UCT-GAN in learning the inverse function of the designed non-linear mapping function. Finally, colour cast images are restored via the inverse function. A data augmentation method based on Poisson fusion and block combination is also proposed to overcome the problem of requiring a large amount of training data. Moreover, the authors extend UCT-GAN into a multi-class colour transfer network to achieve an array of underwater image enhancements. Experimental results indicate that the proposed UCT-GAN can more effectively resolve underwater image colour cast compared to existing algorithms.

Inspec keywords: image restoration; image enhancement; image fusion; image colour analysis; neural nets; geophysical image processing

Other keywords: block combination; inverse function; image enhancement algorithms; multiclass colour transfer network; underwater image colour transfer generative adversarial network; UCT-GAN; nonlinear mapping function; image pairs; training data; underwater image enhancement neural networks; colour cast image restoration; underwater visibility enhancement; image quality; Poisson fusion; data augmentation method

Subjects: Geophysics computing; Instrumentation and techniques for geophysical, hydrospheric and lower atmosphere research; Oceanographic and hydrological techniques and equipment; Data and information; acquisition, processing, storage and dissemination in geophysics; Computer vision and image processing techniques; Sensor fusion; Optical, image and video signal processing; Neural computing techniques

References

    1. 1)
      • 21. Chen, X., Xu, C., Yang, X., et al: ‘Gated-GAN: adversarial gated networks for multi-collection style transfer’, IEEE Trans. Image Process., 2018, 28, (2), pp. 546560.
    2. 2)
      • 4. Fu, X., Fan, Z., Ling, M., et al: ‘Two-step approach for single underwater image enhancement’. 2017 Int. Symp. on Intelligent Signal Processing and Communication Systems (ISPACS), Xiamen, People's Republic of China, 2017, pp. 789794.
    3. 3)
      • 11. Wang, Y., Zhang, J., Cao, Y., et al: ‘A deep CNN method for underwater image enhancement’. 2017 IEEE Int. Conf. on Image Processing (ICIP), Beijing, People's Republic of China, 2017, pp. 13821386.
    4. 4)
      • 12. Zhang, S., Zhang, J., Fang, S., et al: ‘Underwater stereo image enhancement using a new physical model’. 2014 IEEE Int. Conf. on Image Processing (ICIP), Paris, France, 2014, pp. 54225426.
    5. 5)
      • 30. https://github.com/kskin/WaterGAN.
    6. 6)
      • 6. He, K, Sun, J., Tang, X.: ‘Single image haze removal using dark channel prior’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 33, (12), pp. 23412353.
    7. 7)
      • 27. Luo, P., Ren, J., Peng, Z., et al: ‘Differentiable learning-to-normalize via switchable normalization’, arXiv preprint arXiv:1806.10779, 2018.
    8. 8)
      • 14. Wang, N., Zhou, Y., Han, F., et al: ‘Uwgan: Underwater GAN for real-world underwater color restoration and dehazing’. arXiv preprint arXiv:1912.10269, 2019.
    9. 9)
      • 20. Nazeri, K., Ng, E., Joseph, T., et al: ‘Edgeconnect: Generative image inpainting with adversarial edge learning’, arXiv preprint arXiv:1901.00212, 2019.
    10. 10)
      • 9. Jian, Z., Shifei, D., Nan, Z.: ‘An overview on probability undirected graphs and their applications in image processing’, Neurocomputing, 2018, 321, pp. 156168.
    11. 11)
      • 33. Mittal, A., Moorthy, A.K., Bovik, A.C.: ‘No-reference image quality assessment in the spatial domain’, IEEE Trans. Image Process., 2012, 21, (12), pp. 46954708.
    12. 12)
      • 25. Fabbri, C., Islam, Md. J., Sattar, J.: ‘Enhancing underwater imagery using generative adversarial networks’. 2018 IEEE Int. Conf. on Robotics and Automation (ICRA), Brisbane,QLD, Australia, 2018, pp. 71597165.
    13. 13)
      • 29. Arjovsky, M., Chintala, S., Bottou, L.: ‘Wasserstein GAN’, arXiv preprint arXiv:1701.07875, 2017.
    14. 14)
      • 1. Chao, L., Wang, M.: ‘Removal of water scattering’. 2010 Second Int. Conf. on Computer Engineering and Technology, vol. 2, Chengdu, People's Republic of China, 2010, pp. V2-35.
    15. 15)
      • 32. Mittal, A., Soundararajan, R., Bovik, A.C.: ‘Making a ‘completely blind’ image quality analyzer’, IEEE Signal Process. Lett., 2013, 20, (3), pp. 209212.
    16. 16)
      • 31. Kang, L., Ye, P., Li, Y., et al: ‘Convolutional neural networks for no-reference image quality assessment’. 2014 Ieee Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 17331740.
    17. 17)
      • 28. Gulrajani, I., Ahmed, F., Arjovsky, M., et al: ‘Improved training of Wasserstein GANs’. Advances in Neural Information Processing Systems 30 (Nips 2017), Long Beach, CA, USA, 2017.
    18. 18)
      • 7. Wen, H., Tian, Y., Huang, T., et al: ‘Single underwater image enhancement with a new optical model’. 2013 IEEE Int. Symp. on Circuits and Systems (ISCAS 2013), Beijing, People's Republic of China, 2013, pp. 753756.
    19. 19)
      • 3. Mathur, M., Goel, N.: ‘Enhancement of underwater images using white balancing and rayleigh-stretching’. 2018 5th Int. Conf. on Signal Processing and Integrated Networks (SPIN), Noida, India, 2018, pp. 924929.
    20. 20)
      • 5. Li, C., Quo, J., Pang, Y., et al: ‘Single underwater image restoration by blue-green channels dehazing and red channel correction’. 2016 IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, People's Republic of China, 2016, pp. 17311735.
    21. 21)
      • 10. Sun, X., Liu, L., Li, Q., et al: ‘Deep pixel-to-pixel network for underwater image enhancement and restoration’, IET Image Process., 2018, 13, (3), pp. 469474.
    22. 22)
      • 18. Karras, T., Laine, S., Aittala, M., et al: ‘Analyzing and improving the image quality of stylegan’, arXiv preprint arXiv:1912.04958, 2019.
    23. 23)
      • 22. Li, C.Y., Guo, J.C., Guo, C.L.: ‘Emerging from water: underwater image color correction based on weakly supervised color transfer’, IEEE Signal Process. Lett., 2018, 25, (3), pp. 323327.
    24. 24)
      • 16. Wang, H., Wang, Y., Zhou, Z., et al: ‘Cosface: large margin cosine loss for deep face recognition’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake City,UT, USA, 2018, pp. 52655274.
    25. 25)
      • 13. Li, J., Skinner, K.A., Eustice, R.M., et al: ‘Watergan: unsupervised generative network to enable real-time color correction of monocular underwater images’, IEEE Robot. Autom. Lett., 2017, 3, (1), pp. 387394.
    26. 26)
      • 24. Zhu, J.Y., Park, T., Isola, P., et al: ‘Unpaired image-to-image translation using cycle-consistent adversarial networks’. 2017 Ieee Int. Conf. on Computer Vision (Iccv), Venice, Italy, 2017, pp. 22422251.
    27. 27)
      • 8. Borkar, S., Bonde, S.V.: ‘Underwater image restoration using single color channel prior’. 2016 Int. Conf. on Signal and Information Processing (IConSIP), Vishnupuri, India, 2016, pp. 14.
    28. 28)
      • 2. Hitam, M.S., Awalludin, E.A., , Wan Yussof, W.N.J.H., et al: ‘Mixture contrast limited adaptive histogram equalization for underwater image enhancement’. 2013 Int. Conf. Computer Applications Technology (ICCAT), Sousse, Tunisia, 2013, pp. 15.
    29. 29)
      • 17. Pérez, P., Gangnet, M, Blake, A.: ‘Poisson image editing’, ACM Trans. Graphics (TOG), 2003, 22, (3), pp. 313318.
    30. 30)
      • 26. Huang, G., Liu, Z., van der Maaten, L., et al: ‘Densely connected convolutional networks’. 30th Ieee Conf. on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA, 2017, pp. 22612269.
    31. 31)
      • 23. Wang, Z., Bovik, A.C., Sheikh, H.R., et al: ‘Image quality assessment: from error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13, (4), pp. 600612.
    32. 32)
      • 15. Chen, X., Yu, J., Kong, S., et al: ‘Towards real-time advancement of underwater visual quality with GAN’, IEEE Trans. Ind. Electron., 2019, 66, (12), pp. 11.
    33. 33)
      • 19. Zeng, K., Ding, S., Jia, W.: ‘Single image super-resolution using a polymorphic parallel CNN’, Appl. Intell., 2019, 49, (1), pp. 292300.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2020.0003
Loading

Related content

content/journals/10.1049/iet-ipr.2020.0003
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading