access icon free Improvement of global motion estimation in two-dimensional digital video stabilisation methods

A large amount of video content has been produced by compact and portable cameras. Several applications have been benefited from such growth of multimedia data, such as telemedicine, business conferencing, surveillance and security, entertainment, distance learning, and robotics. Video stabilisation is the process of detecting and removing undesired motion or instabilities from a video stream caused during the acquisition stage when handling the camera. In this work, the authors introduce and analyse a novel approach that identifies failures in the global motion estimation of the camera by means of local features. Moreover, they propose an optimisation method for computing a new estimate of the corrected motion. Experiments conducted on different video sequences are performed to demonstrate the effectiveness of the developed method. Results obtained with the stabilisation process are compared against the state-of-the-art YouTube method.

Inspec keywords: distance learning; video streaming; cameras; video signal processing; image sequences; motion estimation

Other keywords: business conferencing; undesired motion; compact cameras; corrected motion; video stream; stabilisation process; portable cameras; global motion estimation; distance learning; YouTube method; two-dimensional digital video stabilisation methods; video sequences; optimisation method; multimedia data; video content

Subjects: Video signal processing; Optical, image and video signal processing; Computer vision and image processing techniques

References

    1. 1)
      • 30. Liu, S., Xu, B., Deng, C., et al: ‘A hybrid approach for near-range video stabilization’, IEEE Trans. Circuits Syst. Video Technol., 2017, 27, (9), pp. 19221933.
    2. 2)
      • 50. Moreira, T., Menotti, D., Pedrini, H.: ‘First-person action recognition through visual rhythm texture description’. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, New Orleans, LA, USA, March 2017, pp. 26272631.
    3. 3)
      • 18. Chang, H.-C., Lai, S.-H., Lu, K.-R.: ‘A robust and efficient video stabilization algorithm’. IEEE Int. Conf. on Multimedia and Expo, Taipei, Taiwan, 2004, pp. 2932.
    4. 4)
      • 48. Wang, Z., Bovik, A.C., Sheikh, H.R., et al: ‘Image quality assessment: from error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13, (4), pp. 600612.
    5. 5)
      • 46. Bay, H., Ess, A., Tuytelaars, T., et al: ‘Speeded-up robust features (SURF)’, Comput. Vis. Image Underst., 2008, 110, (3), pp. 346359.
    6. 6)
      • 21. Shen, Y., Guturu, P., Damarla, T., et al: ‘Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework’, IEEE Trans. Consum. Electron., 2009, 55, (3), pp. 17141721.
    7. 7)
      • 26. Li, S., Yuan, L., Sun, J., et al: ‘Dual-feature warping-based motion model estimation’. IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 42834291.
    8. 8)
      • 41. Maes, F., Collignon, A., Vandermeulen, D., et al: ‘Multimodality image registration by maximization of mutual information’, IEEE Trans. Med. Imaging, 1997, 16, (2), pp. 187198.
    9. 9)
      • 8. Marcenaro, L., Vernazza, G., Regazzoni, C.S.: ‘Image stabilization algorithms for video-surveillance applications’. IEEE Int. Conf. on Image Processing, Thessaloniki, Greece, 2001, pp. 349352.
    10. 10)
      • 15. Grundmann, M., Kwatra, V., Essa, I.: ‘Auto-directed video stabilization with Robust L1 optimal camera paths’. IEEE Conf. on Computer Vision and Pattern Recognition, Colorado Springs, USA, 2011.
    11. 11)
      • 5. Ko, S.-J., Lee, S.-H., Lee, K.-H.: ‘Digital image stabilizing algorithms based on bit-plane matching’, IEEE Trans. Consum. Electron., 1998, 44, (3), pp. 617622.
    12. 12)
      • 38. Jenkinson, M., Bannister, P., Brady, M., et al: ‘Improved optimization for the robust and accurate linear registration and motion correction of brain images’, Neuroimage, 2002, 17, (2), pp. 825841.
    13. 13)
      • 27. Liu, F., Gleicher, M., Wang, J., et al: ‘Subspace video stabilization’, ACM Trans. Graph., 2011, 30, (1), p. 4.
    14. 14)
      • 35. Jia, C., Evans, B.L.: ‘Online motion smoothing for video stabilization via constrained multiple-model estimation’, EURASIP J. Image Video Process., 2017, 2017, (1), p. 25.
    15. 15)
      • 19. Puglisi, G., Battiato, S.: ‘A robust image alignment algorithm for video stabilization purposes’, IEEE Trans. Circuits Syst. Video Technol., 2011, 21, (10), pp. 13901400.
    16. 16)
      • 25. Liu, S., Yuan, L., Tan, P., et al: ‘Steadyflow: spatially smooth optical flow for video stabilization’. IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, USA, 2014, pp. 42094216.
    17. 17)
      • 39. Klein, S., Pluim, J.P., Staring, M., et al: ‘Adaptive stochastic gradient descent optimisation for image registration’, Int. J. Comput. Vis., 2009, 81, (3), p. 227.
    18. 18)
      • 33. Hamza, A., Hafiz, R., Khan, M.M., et al: ‘Stabilization of panoramic videos from mobile multi-camera platforms’, Image Vis. Comput., 2015, 37, pp. 2030.
    19. 19)
      • 32. Kim, S.W., Yin, S., Yun, K., et al: ‘Spatio-temporal weighting in local patches for direct estimation of camera motion in video stabilization’, Comput. Vis. Image Underst., 2014, 118, pp. 7183.
    20. 20)
      • 14. Liu, S., Yuan, L., Tan, P., et al: ‘Bundled camera paths for video stabilization’, ACM Trans. Graph., 2013, 32, (4), p. 78.
    21. 21)
      • 16. Goshtasby, A.A.: ‘Image registration: principles, tools and methods’ (Springer Science & Business Media, Berlin Heidelberg, 2012).
    22. 22)
      • 29. Goldstein, A., Fattal, R.: ‘Video stabilization using epipolar geometry’, ACM Trans. Graph., 2012, 31, (5), pp. 110.
    23. 23)
      • 34. Kopf, J.: ‘360 video stabilization’, ACM Trans. Graph., 2016, 35, (6), p. 195.
    24. 24)
      • 47. Choi, S., Kim, T., Yu, W.: ‘Robust video stabilization to outlier motion using adaptive RANSAC’. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, St. Louis, USA, 2009, pp. 18971902.
    25. 25)
      • 1. Amanatiadis, A.A., Andreadis, I.: ‘Digital image stabilization by independent component analysis’, IEEE Trans. Instrum. Meas., 2010, 59, (7), pp. 17551763.
    26. 26)
      • 6. Kumar, S., Azartash, H., Biswas, M., et al: ‘Real-time affine global motion estimation using phase correlation and its application for digital image stabilization’, IEEE Trans. Image Process., 2011, 20, (12), pp. 34063418.
    27. 27)
      • 4. Jia, R., Zhang, H., Wang, L., et al: ‘Digital image stabilization based on phase correlation’. IEEE Int. Conf. on Artificial Intelligence and Computational Intelligence, Shanghai, China, 2009, pp. 485489.
    28. 28)
      • 49. Powell, M.J.: ‘An efficient method for finding the minimum of a function of several variables without calculating derivatives’, Comput. J., 1964, 7, (2), pp. 155162.
    29. 29)
      • 40. Klein, S., Staring, M., Pluim, J.P.: ‘Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-splines’, IEEE Trans. Image Process., 2007, 16, (12), pp. 28792890.
    30. 30)
      • 23. Ertürk, S.: ‘Image sequence stabilisation based on kalman filtering of frame positions’, Electron. Lett., 2001, 37, (20), p. 1.
    31. 31)
      • 13. Huang, T.S.: ‘Image sequence analysis’, (Springer Science & Business Media, Berlin Heidelberg, 2013).
    32. 32)
      • 36. Liu, S., Li, M., Zhu, S., et al: ‘Coding flow: enable video coding for video stabilization’. IEEE Trans. Image Process., 2017, 26, (7), pp. 32913302.
    33. 33)
      • 42. Maes, F., Vandermeulen, D., Suetens, P.: ‘Comparative evaluation of multiresolution optimization strategies for multimodality image registration by maximization of mutual information’, Med. Image Anal., 1999, 3, (4), pp. 373386.
    34. 34)
      • 2. Chang, J.-Y., Hu, W.-F., Cheng, M.-H., et al: ‘Digital image translational and rotational motion stabilization using optical flow technique’, IEEE Trans. Consum. Electron., 2002, 48, (1), pp. 108115.
    35. 35)
      • 11. Cirne, M.V.M., Pedrini, H.: ‘A video summarization method based on spectral clustering’. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Havana, Cuba, 2013, pp. 479486.
    36. 36)
      • 20. Battiato, S., Gallo, G., Puglisi, G., et al: ‘SIFT features tracking for video stabilization’. IEEE 14th Int. Conf. on Image Analysis and Processing, Modena, Italy, 2007, pp. 825830.
    37. 37)
      • 22. Matsushita, Y., Ofek, E., Ge, W., et al: ‘Full-frame video stabilization with motion inpainting’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (7), pp. 11501163.
    38. 38)
      • 24. Litvin, A., Konrad, J., Karl, W.C.: ‘Probabilistic video stabilization using Kalman filtering and mosaicing’, in Vasudev, B., Hsing, T.R., Tescher, A.G., et al (Eds.): ‘Electronic imaging’ (SPIE, Bellingham, USA, 2003), pp. 663674.
    39. 39)
      • 37. Bajcsy, R., Kovačič, S.: ‘Multiresolution elastic matching’, Comput. Vis. Graph. Image Process., 1989, 46, (1), pp. 121.
    40. 40)
      • 51. Pinto, A., Schwartz, W., Pedrini, H., et al: ‘Using visual rhythms for detecting video-based facial spoof attacks’, IEEE Trans. Inf. Forensics Sec., 2015, 10, (5), pp. 10251038.
    41. 41)
      • 17. Lee, K.-Y., Chuang, Y.-Y., Chen, B.-Y., et al: ‘Video stabilization using robust feature trajectories’. IEEE 12th Int. Conf. on Computer Vision, Kyoto, Japan, 2009, pp. 13971404.
    42. 42)
      • 45. Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: ‘Optimization by simulated annealing’, Science, 1983, 220, (4598), pp. 671680.
    43. 43)
      • 3. Ertürk, S.: ‘Real-time digital image stabilization using Kalman filters’, Real-Time Imaging, 2002, 8, (4), pp. 317328.
    44. 44)
      • 31. Liu, S., Wang, Y., Yuan, L., et al: ‘Video stabilization with a depth camera’. IEEE Conf. on Computer Vision and Pattern Recognition, Providence, USA, 2012, pp. 8995.
    45. 45)
      • 9. Morimoto, C., Chellappa, R.: ‘Fast electronic digital image stabilization’. IEEE 13th Int. Conf. on Pattern Recognition, Vienna, Austria, 1996, pp. 284288.
    46. 46)
      • 12. Cirne, M.V.M., Pedrini, H.: ‘Summarization of videos by image quality assessment’. Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Puerto Vallarta, Mexico, 2014, pp. 901908.
    47. 47)
      • 43. Wachowiak, M.P., Smolková, R., Zheng, Y., et al: ‘An approach to multimodal biomedical image registration utilizing particle swarm optimization’, IEEE Trans. Evol. Comput., 2004, 8, (3), pp. 289301.
    48. 48)
      • 28. Zhao, Z., Ma, X.: ‘Video stabilization based on local trajectories and robust mesh transformation’. IEEE Int. Conf. on Image Processing, Phoenix, USA, 2016, pp. 40924096.
    49. 49)
      • 10. Ryu, Y.G., Chung, M.J.: ‘Robust online digital image stabilization based on point-feature trajectory without accumulative global motion estimation’, IEEE Signal Process. Lett., 2012, 19, (4), pp. 223226.
    50. 50)
      • 7. Lin, C.-T., Hong, C.-T., Yang, C.-T.: ‘Real-time digital image stabilization system using modified proportional integrated controller’, IEEE Trans. Circuits Syst. Video Technol., 2009, 19, (3), pp. 427431.
    51. 51)
      • 44. Winter, S., Brendel, B., Pechlivanis, I., et al: ‘Registration of CT and intraoperative 3-D ultrasound images of the spine using evolutionary and gradient-based methods’, IEEE Trans. Evol. Comput., 2008, 12, (3), pp. 284296.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.5445
Loading

Related content

content/journals/10.1049/iet-ipr.2018.5445
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading