Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Hyperspectral image super-resolution under misaligned hybrid camera system

© The Institution of Engineering and Technology
Received 14/12/2017, Accepted 25/04/2018, Revised 19/04/2018, Published 30/04/2018

Hyperspectral imaging has been widely used for agriculture, astronomy, surveillance, and so on. However, hyperspectral imaging usually suffers from low-spatial resolution, due to the limited photons in individual bands. Recently, more hyperspectral image super-resolution methods have been developed by fusing the low-resolution hyperspectral image and high-resolution RGB image, but most of them did not consider the misalignment between two input images. In this study, the authors present an effective method to restore a high-resolution hyperspectral image from the misaligned low-resolution hyperspectral image and high-resolution RGB image, which exploits spectral and spatial correlation in hyperspectral and RGB images. Specifically, they employ the spectral sparsity to restore the high-resolution hyperspectral image on the misaligned part, and then simultaneously employ spectral and spatial structure correlation to restore the high-resolution hyperspectral image on the aligned area, which can be fused to obtain the high-quality hyperspectral image restoration under a misaligned hybrid camera system. Experimental results show that the proposed method outperforms the state-of-the-art hyperspectral image super-resolution methods under a misaligned hybrid camera system in terms of both objective metric and subjective visual quality.

Inspec keywords: correlation methods; cameras; hyperspectral imaging; image colour analysis; image restoration; image resolution; image fusion

Other keywords: spatial correlation; subjective visual quality; objective metric; low-resolution hyperspectral image fusion; low-spatial resolution; hyperspectral image super-resolution methods; high-resolution RGB image fusion; misaligned hybrid camera system; spectral correlation; hyperspectral imaging

Subjects: Sensor fusion; Computer vision and image processing techniques; Image sensors; Optical, image and video signal processing

References

    1. 1)
      • 17. Kawakami, R., Wright, J., Tai, Y.-W., et al: ‘High-resolution hyperspectral imaging via matrix factorization’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011, pp. 2329–2336.
    2. 2)
      • 13. Glasner, D., Bagon, S., Irani, M.: ‘Super-resolution from a single image’. Proc. Int. Conf. on Computer Vision (ICCV), Kyoto, Japan, IEEE, 2009, pp. 349–356.
    3. 3)
      • 2. Ojha, L., Wilhelm, M.B., Murchie, S.L., et al: ‘Spectral evidence for hydrated salts in recurring slope lineae on Mars’ Nat. Geosci., 2015, 8, 829–832.
    4. 4)
      • 10. Gat, N., Scriven, G., Garman, J., et al: ‘Development of four-dimensional imaging spectrometers (4d-IS)’. Proc. SPIE Optics + Photonics, Vol. 6302, 2006, pp. 63020M–63020M-11.
    5. 5)
      • 28. Buades, A., Coll, B., Morel, J.M.: ‘A non-local algorithm for image denoising’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), San Diego, USA, 2005, pp. 60–65.
    6. 6)
      • 11. Gao, L., Kester, R.T., Hagen, N., et al: ‘Snapshot image mapping spectrometer (IMS) with high sampling density for hyperspectral microscopy’, Opt. Express, 2010, 18, (14), p. 14330.
    7. 7)
      • 8. Yamaguchi, M., Haneishi, H., Fukuda, H., et al: ‘High-fidelity video and still-image communication based on spectral information: natural vision system and its applications’. Proc. 8th Int. Symp. on Multispectral Color Science, San Jose, USA, 2006, pp. 60620G–60620G-12.
    8. 8)
      • 12. Cao, X., Du, H., Tong, X., et al: ‘A prism-mask system for multispectral video acquisition’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), 2011, 33, (12), pp. 2423–2435.
    9. 9)
      • 26. Cao, X., Tong, X., Dai, Q., et al: ‘High resolution multispectral video capture with a hybrid camera system’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, 2011, pp. 297–304.
    10. 10)
      • 21. Lanaras, C., Baltsavias, E., Schindler, K.: ‘Hyperspectral super-resolution by coupled spectral unmixing’. Proc. Int. Conf. on Computer Vision (ICCV), Santiago, Chile, 2015, pp. 3586–3594.
    11. 11)
      • 9. Ford, B., Descour, M., Lynch, R.: ‘Large-image-format computed tomography imaging spectrometer for fluorescence microscopy’, Opt. Express, 2001, 9, (9), p. 444.
    12. 12)
      • 18. Akhtar, N., Shafait, F., Mian, A.: ‘Sparse spatio-spectral representation for hyperspectral image super-resolution’. Proc. European Conf. on Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 63–78.
    13. 13)
      • 4. Chen, G., Li, C., Sun, W.: ‘Hyperspectral face recognition via feature extraction and CRC-based classifier’, IET Image Process., 2017, 11, (4), pp. 266–272.
    14. 14)
      • 24. Dong, W., Fu, F., Shi, G., et al: ‘Hyperspectral image super-resolution via Non-negative structured sparse representation’, IEEE Trans. Image Process., 2016, 25, (5), pp. 2337–2352.
    15. 15)
      • 32. Wang, Z., Bovik, A., Sheikh, H., et al: ‘Image quality assessment: from error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13, (4), pp. 600–612.
    16. 16)
      • 14. Ben, M.-E., Lin, Z., Wilburn, B., et al: ‘Penrose pixels for super-resolution’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (7), pp. 1370–1383.
    17. 17)
      • 6. Fu, Y., Lam, A., Sato, I., et al: ‘Separating reflective and fluorescent components using high frequency illumination in the spectral domain’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (5), pp. 965–978.
    18. 18)
      • 16. Huang, J.-B., Singh, A., Ahuja, N.: ‘Single image super-resolution from transformed self-exemplars’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015, pp. 5197–5206.
    19. 19)
      • 7. Gat, N.: ‘Imaging spectroscopy using tunable filters: a review’. Proc. Society of Photographic Instrumentation Engineers, Wavelet Applications VII, Orlando, USA, Vol. 4056, 2000, pp. 50–64.
    20. 20)
      • 25. Dian, R., Fang, L., Li, S.: ‘Hyperspectral image super-resolution via non-local sparse tensor factorization’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017, pp. 5344–5353.
    21. 21)
      • 30. Yasuma, F., Mitsunaga, T., Iso, D., et al: ‘Generalized assorted pixel camera: post-capture control of resolution, dynamic range and spectrum’. Tech. Report (November 2008).
    22. 22)
      • 5. Imani, M., Ghassemian, H.: ‘Edge patch image-based morphological profiles for classification of multispectral and hyperspectral data’, IET Image Process., 2017, 11, (3), pp. 164–172.
    23. 23)
      • 22. Wycoff, E., Chan, T.-H., Jia, K., et al: ‘A non-negative sparse promoting algorithm for high resolution hyperspectral imaging’. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, Canada, 2013, pp. 1409–1413.
    24. 24)
      • 23. Kwon, H., Tai, Y.-W.: ‘RGB-guided hyperspectral image upsampling’. Proc. Int. Conf. on Computer Vision (ICCV), Santiago, Chile, 2015, pp. 307–315.
    25. 25)
      • 3. Nguyen, H.V., Banerjee, A., Chellappa, R.: ‘Tracking via object reflectance using a hyperspectral video camera’. IEEE Conf. Computer Vision and Pattern Recognition – Workshops, San Francisco, USA, 2010, pp. 44–51.
    26. 26)
      • 33. Kruse, F.A., Lefkoff, A.B., Boardman, J.W., et al: ‘The spectral image processing system (SIPS)–interactive visualization and analysis of imaging spectrometer data’, Remote Sens. Environ., 1993, 44, (2–3), pp. 145–163.
    27. 27)
      • 27. Mairal, J., Bach, F., Ponce, J., et al: ‘Online dictionary learning for sparse coding’. Proc. Int. Conf. on Machine Learning (ICML), Montreal, Canada, 2009, pp. 689–696.
    28. 28)
      • 20. Akhtar, N., Shafait, F., Mian, A.: ‘Hierarchical beta process with Gaussian process prior for hyperspectral image super resolution’. Proc. European Conf. on Computer Vision (ECCV), Amsterdam, The Netherlands, 2016, pp. 103–120.
    29. 29)
      • 19. Akhtar, N., Shafait, F., Mian, A.: ‘Bayesian sparse representation for hyperspectral image super resolution’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, USA, 2015.
    30. 30)
      • 15. Timofte, R., De Smet, V., Van Gool, L.: ‘A+: adjusted anchored neighborhood regression for fast super-resolution’. Proc. Asian Conf. on Computer Vision (ACCV), Singapore, 2014, pp. 111–126.
    31. 31)
      • 1. Borengasser, M., Hungate, W.S., Watkins, R.: ‘Hyperspectral remote sensing: principles and applications, remote sensing applications Series’ (CRC Press, Boca Raton, FL, USA, 2007).
    32. 32)
      • 31. Nguyen, R.M.H., Prasad, D.K., Brown, M.S.: ‘Training-Based spectral reconstruction from a single RGB image’. Proc. European Conf. on Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 186–201.
    33. 33)
      • 29. Eckstein, J., Yao, W.: ‘Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results’. RUTCOR Research Reports 32, 2012.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2017.1340
Loading

Related content

content/journals/10.1049/iet-ipr.2017.1340
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address