The recent development of advanced television systems has demonstrated a need for an efficient video conversion technique. In this scenario, frame rate up conversion (FRUC) solutions play an important role due to their benefits in both increasing the viewing quality experience and reducing the cost of video transmission. However, with the recent increase in video resolution, notably from standard definition to high definition (HD) and ultra HD, FRUC now requires not only better interpolated frame quality but also lower FRUC time processing. Considering this problem, this study proposes a novel statistical learning based adaptive search range (SR) solution to enable an effective FRUC mechanism. In the proposed adaptive SR solution, a set of spatial-temporal features are carefully defined and exploited to adaptively assign an appropriate SR value to each considered block, notably by formulating the SR adaptation as a classification problem and using the well-known support vector machine framework for the classification task. Experimental results conducted on a rich set of common video test sequences show the advantages of the proposed adaptive SR solution, notably in both interpolated frame quality improvement and time processing reduction.

References

1. 1)
  - 23. Xu, C., Chen, Y., Gao, Z., et al: ‘Frame rate up-conversion with true motion estimation and adaptive motion vector refinement’. Int. Congress on Image and Signal Processing (CISP), 2011, pp. 353–356.
2. 2)
  - 19. Kang, S.-J., Yoo, S., Kim, Y.H.: ‘Dual motion estimation for frame rate up-conversion’, IEEE Trans. Circuits Syst. Video Technol., 2010, 20, (12), pp. 1909–1914.
3. 3)
  - 13. Tsai, T.-H., Chen, H.-G., Lin, H.-Y.: ‘Frame rate up-conversion using adaptive bilateral motion estimation’. Proc. 8th WSEAS Int. Conf. on Applied Computer and Applied Computational Science, 2009, pp. 199–202.
4. 4)
  - 15. Wang, T.-S., Choi, K.-S., Jang, H.-S., et al: ‘Enhanced frame rate up-conversion method for UHD video’, IEEE Trans. Consum. Electron., 2010, 56, (2), pp. 1108–1114.
5. 5)
  - 26. Jung, H.S., Kim, U.S., Sunwoo, M.H.: ‘Simplified frame rate up-conversion algorithm with low computational complexity’. Proc. 20th European on Signal Processing Conf. (EUSIPCO), 2012, pp. 385–389.
6. 6)
  - 6. Zhai, J., Yu, K., Li, J., et al: ‘A low complexity motion compensated frame interpolation method’. IEEE Int. Symp. on Circuits and systems, May 2005, pp. 4927–4930.
7. 7)
  - 22. Wang, Y., Ma, S., Gao, W.: ‘Frame rate up conversion via Bayesian motion estimation’. Proc. SPIE Visual Communications and Image Processing, 2010, vol. 7744.
8. 8)
  - 21. HoangVan, X., Ascenso, J., Pereira, F.: ‘Improved matching criterion for frame rate up conversion with trilateral filtering’, IET Electron. Lett., 2013, 49, (2), pp. 106–107.
9. 9)
  - 16. Jacobson, N., Lee, Y.-L., Mahadevan, V., et al: ‘Motion vector refinement for FRUC using saliency and segmentation’. Proc. IEEE Int. Conf. on Multimedia and Expo (ICME), 2010, pp. 778–783.
10. 10)
  - 31. Choi, D., Song, W., Kim, T.: ‘MAP-based motion refinement algorithms for block-based motion-compensated frame interpolation’, IEEE Trans. Circuits Syst. Video Technol., 2016, 26, (10), pp. 1789–1804.
11. 11)
  - 38. Mualla, M.E.A., Canagarajah, C.N., Bull, D.R.: ‘Motion field interpolation for temporal error concealment’, IEE Proc., Vis. Image Signal Process., 2000, 147, (5), pp. 445–452.
12. 12)
  - 29. Dikbas, S., Altunbasak, Y.: ‘Novel true-motion estimation algorithms and its application to motion-compensated temporal frame interpolation’, IEEE Trans. Image Process., 2013, 22, (8), pp. 2931–2945.
13. 13)
  - 39. Joachims, T.: ‘SVM light,’ Cornell University, 14 08 2008. [Online]. Available: http://download.joachims.org/svm_light/current/svm_light.tar.gz. [Accessed 15 06 2012].
14. 14)
  - 11. Patras, I., Hendriks, E.A., Lagendijk, R.L.: ‘Probabilistic confidence measures for block matching motion estimation’, IEEE Trans. Circuits Syst. Video Technol., 2007, 17, (8), pp. 988–995.
15. 15)
  - 32. Kaviani, H.R., Shirani, S.: ‘Frame rate up-conversion using optical flow and patch-based reconstruction’, IEEE Trans. Circuits Syst. Video Technol., 2016, 26, (9), pp. 1581–1594.
16. 16)
  - 25. Lim, H., Park, H.W.: ‘A symmetric motion estimation method for motion-compensated frame interpolation’, IEEE Trans. Image Process., 2011, 20, (12), pp. 3653–3658.
17. 17)
  - 3. Castagno, R., Haavisto, P., Ramponi, G.: ‘A method for motion adaptive frame rate up-conversion’, IEEE Trans. Circuits Syst. Video Technol., 1996, 6, (5), pp. 436–446.
18. 18)
  - 14. Huang, A.-M., Nguyen, T.: ‘Correlation-based motion vector processing with adaptive interpolation scheme for motion-compensated frame interpolation’, IEEE Trans. Image Process., 2009, 18, (4), pp. 740–752.
19. 19)
  - 28. Qian, C., Bajic, I.V.: ‘Frame rate up-conversion using global and local higher-order motion’. Proc. IEEE Int. Conf. on Multimedia and Expo, July 2013, pp. 1–6.
20. 20)
  - 12. Huang, A.-M., Nguyen, T.Q.: ‘A multistage motion vector processing method for motion-compensated frame interpolation’, IEEE Trans. Image Process., 2008, 17, (5), pp. 694–708.
21. 21)
  - 36. Chapelle, O., Haffner, P., Vapnik, V.N.: ‘Support vector machines for histogram-based image classification’, IEEE Trans. Neural Netw., 1999, 10, (5), pp. 1055–1064.
22. 22)
  - 24. Tran, T.-H., LeDinh, C.-T.: ‘Frame rate converter with pixel-based motion vector selection and halo reduction using preliminary interpolation’, IEEE J. Sel. Top. Signal Process., 2011, 5, (2), pp. 252–261.
23. 23)
  - 8. Beric, A., van Meerbergen, J., de Haan, G.: ‘Memory – centric video processing’, IEEE Trans. Circuits Syst. Video Technol., 2008, 18, (4), pp. 439–452.
24. 24)
  - 1. Yang, Y.-T., Tung, Y.-S., Wu, J.-L.: ‘Quality enhancement of frame rate up-converted video by adaptive frame skip and reliable motion extraction’, IEEE Trans. Circuits Syst. Video Technol., 2007, 17, (12), pp. 1700–1713.
25. 25)
  - 18. Tsai, T.-H., Lin, H.-Y., Chen, H.-G.: ‘Overlapped block-based adaptive bilateral motion estimation’. IEEE Asia Pacific Conf. on Circuits and Systems (APCCAS), 2010, pp. 571–575.
26. 26)
  - 17. Jacobson, N., Lee, Y.-L., Mahadevan, V., et al: ‘A novel approach to FRUC using discriminant saliency and frame segmentation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2924–2934.
27. 27)
  - 2. Wong, C.K., Au, O.C., Tang, C.W.: ‘Motion compensated temporal interpolation with overlapping’. Proc. IEEE Int. Symp. on Circuits and Systems, May 1996, pp. 608–611.
28. 28)
  - 34. Cortes, C., Vapnik, V.: ‘Support-vector networks’, Mach. Learn., 1995, 20, (3), pp. 273–297.
29. 29)
  - 27. Kim, U.S., Sunwoo, M.H.: ‘New frame up-conversion algorithms with low computational complexity’, IEEE Trans. Circuits Syst. Video Technol., 2014, 24, (3), pp. 384–393.
30. 30)
  - 20. Wang, C., He, Y., Tan, Y.: ‘Frame rate up-conversion using trilateral filtering’, IEEE Trans. Circuits Syst. Video Technol., 2010, 20, (6), pp. 886–893.
31. 31)
  - 33. HoangVan, X., Duong, D.T., Ha, L.T.: ‘Spatial temporal feature extraction based adaptive search range for effective frame rate up-conversion’. Proc. Int. Conf. on Advanced Technologies for Communications, Hanoi, Vietnam, October 2016, pp. 287–292.
32. 32)
  - 4. Alparone, L., Barni, M., Bartolini, F., et al: ‘Adaptive weighted vector-median filters for motion-fields smoothing’. Proc. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Washington, DC, USA, 1996, pp. 2267–2270.
33. 33)
  - 35. Weston, J., Watkins, C.: ‘Multi-class support vector machines’, Royal Holloway, University of London, London, 1998.
34. 34)
  - 37. Joachims, T.: ‘Making large scale SVM learning practical’ in advances in kernel methods: support vector Learning’ (MIT Press, Cambridge, MA, 1999), pp. 169–184.
35. 35)
  - 10. Choi, B.-D., Han, J.-W., Kim, C.-S., et al: ‘Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation’, IEEE Trans. Circuits Syst. Video Technol., 2007, 17, (4), pp. 407–416.
36. 36)
  - 7. Beric, A., de Haan, G., Sethuraman, R., et al: ‘An efficient picture – rate up - converter’, J. VLSI Signal Process., 2005, 41, pp. 49–63.
37. 37)
  - 9. Dane, G., Nguyen, T.Q..: ‘Optimal temporal interpolation filter for motion-compensated frame rate up conversion’, IEEE Trans. Image Process., 2006, 15, (4), pp. 978–991.
38. 38)
  - 30. Lee, W.H., Choi, K., Ra, J.B.: ‘Frame rate up conversion based on variational image fusion’, IEEE Trans. Image Process., 2014, 23, (1), pp. 399–412.
39. 39)
  - 5. Dane, G., Nguyen, T.Q.: ‘Smooth motion vector resampling for standard compatible video post-Processing’. Conf. Record of the 38th Asilomar Conf. on Signals, Systems and Computers, California, November 2004, pp. 1731–1735.

Statistical search range adaptation solution for effective frame rate up-conversion

References

Related content