Sensor-based human activity classification requires time and frequency domain feature extraction techniques. The set of choice in time and frequency domain features may have a significant impact on the overall classification accuracy. Another problem is to train deep learning models with sufficient dataset. The use of generative models eliminates the requirement of choosing certain features of the signal. As a generative model, restricted Boltzmann machine (RBM) is an energy-based probabilistic graphical model which factorises the probability distribution of a random variable over a binary probability distribution. Conditional restricted Boltzmann machines (CRBMs) is an extension to RBM, which can capture temporal information in time-series signals and can be deployed as a generative model in classification. In this study, the authors show how CRBMs can be trained to learn signal features. They present four generative model training results, RBM, CRBM, generative adversarial network, Wasserstein generative adversarial network – gradient penalty and compare the models' performances with a performance criterion. They show that the CRBM model can generate signals closest to true signals with a significantly higher success rate as compared to other presented generative models. They present a statistical analysis of the findings and show that the findings significantly hold.

References

1. 1)
  - 33. Ayrulu-Erdem, B., Barshan, B.: ‘Leg motion classification with artificial neural networks using wavelet-based features of gyroscope signals’, Sensors, 2011, 11, (2), pp. 1721–1743.
2. 2)
  - 59. Goodfellow, I.: ‘NIPS 2016 tutorial: generative adversarial networks’, arXiv, 2017, 1701.00160.
3. 3)
  - 58. Salimans, T., Goodfellow, I., Zaremba, W., et al: ‘Improved techniques for training GANs’. Adv. Neural Inf. Process. Syst., Barcelona, Spain, 2016, pp. 2234–2242.
4. 4)
  - 57. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., et al: ‘Generative adversarial nets’, Adv. Neural Inf. Process. Syst., 2014, 3, (January), pp. 2672–2680.
5. 5)
  - 31. Yang, J.Y., Wang, J.S., Chen, Y.P.: ‘Using acceleration measurements for activity recognition: an effective learning algorithm for constructing neural classifiers’, Pattern Recognit. Lett., 2008, 29, (16), pp. 2213–2220.
6. 6)
  - 9. Lei, Y., Wu, Z.: ‘Time series classification based on statistical features’, Eurasip J. Wirel. Commun. Netw., 2020, 2020, (1), pp. 1–13.
7. 7)
  - 45. Altın, C., Er, O.: ‘Comparison of different time and frequency domain feature extraction methods on elbow gesture's EMG’, Eur. J. Interdiscip. Stud., 2016, 2, (3), pp. 35–44.
8. 8)
  - 43. Zeng, M., Nguyen, L.T., Yu, B., et al: ‘Convolutional neural networks for human activity recognition using mobile sensors’. 6th Int. Conf. on Mobile Computing, Applications and Services, Austin, TX, November 2015, pp. 197–205.
9. 9)
  - 64. Meehan, C., Chaudhuri, K., Dasgupta, S.: ‘A non-parametric test to detect data-copying in generative models’, arXiv, 2020, 2004.05675.
10. 10)
  - 53. Taylor, G.W., Hinton, G.E., Roweis, S.: ‘Modeling human motion using binary latent variables’. Advances in Neural Information Processing Systems, Vancouver, British Columbia, Canada, 2007, pp. 1345–1352.
11. 11)
  - 41. Xu, H., Liu, J., Hu, H., et al: ‘Wearable sensor-based human activity recognition method with multi-features extracted from Hilbert-Huang transform’, Sensors (Basel)., 2016, 16, (12), p. 2048.
12. 12)
  - 20. Mannini, A., Sabatini, A.M.: ‘Machine learning methods for classifying human physical activity from on-body accelerometers’, Sensors, 2010, 10, (2), pp. 1154–1175.
13. 13)
  - 7. Li, X., Kang, Y., Li, F.: ‘Forecasting with time series imaging’, Expert Syst. Appl., 2020, 160, (April), p. 113680.
14. 14)
  - 51. Hinton, G.E., Sejnowski, T., Ackley, D.: ‘Boltzmann machines: constraint satisfaction networks that learn’ (Carnegie-Mellon University, 1984).
15. 15)
  - 66. Wang, Y., Zhao, S., Qu, D., et al: ‘Using conditional restricted Boltzmann machines for spectral envelope modeling in speech bandwidth extension’. IEEE Int. Conf. on Acoustics, Speech and Signal Processing (ICASSP), vol. 1, Shanghai, 2016, pp. 5930–5934.
16. 16)
  - 24. Nie, S., Wang, Z., Ji, Q.: ‘A generative restricted Boltzmann machine based method for high-dimensional motion data modeling’, Comput. Vis. Image Underst., 2015, 136, pp. 14–22.
17. 17)
  - 5. Denton, F., Feaver, C., Spencer, B.: ‘Time series analysis and stochastic forecasting: an econometric study of mortality and life expectancy’, J. Popul. Econ., 2005, 18, (2), pp. 203–227.
18. 18)
  - 10. Fulcher, B.D.: ‘Feature-based time-series analysis’, In Dong, G., Liu, H. (Eds.): Feature Eng. Mach. Learn. Data Anal., (CRC Press, 2018), pp. 87–116.
19. 19)
  - 54. Fischer, A., Igel, C.: ‘Training restricted Boltzmann machines: an introduction’, Pattern Recognit., 2014, 47, (1), pp. 25–39.
20. 20)
  - 19. Krishnan, N.C., Juillard, C., Colbry, D., et al: ‘Recognition of hand movements using wearable accelerometers’, J. Ambient Intell. Smart Environ., 2009, 1, (2), pp. 143–155.
21. 21)
  - 50. Hinton, G.E.: ‘A practical guide to training restricted Boltzmann machines’, in: Berlin, H. (Ed.): ‘Neural networks: tricks of the trade. lecture notes in computer science’ (Springer, 2012), pp. 599–619.
22. 22)
  - 16. Kuremoto, T., Kimura, S., Kobayashi, K., et al: ‘Time series forecasting using a deep belief network with restricted Boltzmann machines’, Neurocomputing, 2014, 137, (November 2019), pp. 47–56.
23. 23)
  - 61. Gulrajani, I., Ahmed, F., Arjovsky, M., et al: ‘Improved training of wasserstein GANs’, Adv. Neural Inf. Process. Syst., 2017, December, pp. 5768–5778.
24. 24)
  - 37. Kunze, K., Lukowicz, P., Junker, H., et al: ‘Where am I: recognizing on-body positions of wearable sensors’. Location-and Context-Awareness, First Int. Workshop (LoCA), Oberpfaffenhofen, Germany, 2005, pp. 264–275.
25. 25)
  - 23. Figo, D., Diniz, P.C., Ferreira, D.R., et al: ‘Preprocessing techniques for context recognition from accelerometer data’, Pers. Ubiquitous Comput., 2010, 14, (7), pp. 645–662.
26. 26)
  - 55. Mnih, V., Larochelle, H., Hinton, G.E.: ‘Conditional restricted Boltzmann machines for structured output prediction’. Proc. 27th Conf. Uncertainty Artificial Intelligence (UAI 2011), Barcelona, Spain, 2011, pp. 514–522.
27. 27)
  - 56. Reyes-Ortiz, J.L., Oneto, L., Samà, A., et al: ‘Transition-aware human activity recognition using smartphones’, Neurocomputing, 2016, 171, pp. 754–767.
28. 28)
  - 42. He, Z., Jin, L.: ‘Activity recognition from acceleration data based on discrete consine transform and SVM’. 2009 IEEE Int. Conf. on Systems, Man and Cybernetics, San Antonio, TX, October, 2009, pp. 5041–5044.
29. 29)
  - 1. Krishnan, R., Jeena, K., Prasad, K.P.: ‘Time series forecasting of Covid-19 using deep learning models: India-USA comparative case study’, Chaos, Solitons Fractals, 2020, 140, (November), p. 110227.
30. 30)
  - 36. Reiss, A., Stricker, D.: ‘Introducing a modular activity monitoring system’. Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society, Boston, MA, August, 2011, pp. 5621–5624.
31. 31)
  - 49. Hinton, G.E.: ‘Training products of experts by minimizing contrastive divergence’, Neural Comput., 2002, 14, (8), pp. 1771–1800.
32. 32)
  - 28. Bidargaddi, N., Sarela, A., Klingbeil, L., et al: ‘Detecting walking activity in cardiac rehabilitation by using accelerometer’. Proc. of the 2007 Int. Conf. on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), Melbourne, Qld., 2007, pp. 555–560.
33. 33)
  - 26. UCI Machine Learning Repository.Irvine, C.: ‘Smartphone – based recognition of human activities and postural transitions data set’, available at http://archive.ics.uci.edu/ml/datasets/Smartphone-Based + Recognition + of + Human + Activities + and + Postural + Transitions, accessed May 2019.
34. 34)
  - 4. Baarnhielm, A.: ‘Multiple time-series forecasting on mobile network data using an RNN-RBM model’, Uppsala University, 2017.
35. 35)
  - 17. Avci, A., Bosch, S., Marin-perianu, M., et al: ‘Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: a survey’. 23th Int. Conf. on Architecture of Computing Systems, Hannover, Germany, January 2010, pp. 167–176.
36. 36)
  - 34. Olszewski, R.T.R., Maxion, R., Siewiorek, D., et al: ‘Generalized feature extraction for structural pattern recognition in time-series data’, PhD Thesis, Carnegie Mellon University, 2001.
37. 37)
  - 29. Krishnan, N.C., Colbry, D., Juillard, C.: ‘Real time human activity recognition using tri-axial accelerometers’. Sensors Signals and Information Processing Workshop, Sedona, AZ, 2008.
38. 38)
  - 27. Karantonis, D.M., Narayanan, M.R., Mathie, M., et al: ‘Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring’, IEEE Trans. Inf. Technol. Biomed., 2006, 10, (1), pp. 156–167.
39. 39)
  - 44. Jiang, W., Yin, Z.: ‘Human activity recognition using wearable sensors by deep convolutional neural networks’. MM ‘15: Proc. of the 23rd ACM Int. Conf. on Multimedia, Brisbane Australia, 2015, pp. 1307–1310.
40. 40)
  - 18. Lara, Ó.D., Labrador, M.A.: ‘A survey on human activity recognition using wearable sensors’, IEEE Commun. Surv. Tutorials, 2013, 15, (3), pp. 1192–1209.
41. 41)
  - 15. Raymond, R., Osogami, T., Dasgupta, S.: ‘Dynamic Boltzmann machines for second order moments and generalized Gaussian distributions’. NIPS Time Series Workshop 2017, Long Beach, CA, USA, December 2017.
42. 42)
  - 63. Giorgino, T.: ‘Computing and visualizing dynamic time warping alignments in R: the dtw package’, J. Stat. Softw., 2009, 31, (7), pp. 1–24.
43. 43)
  - 21. Thomas Plötz, N.Y.H.: ‘Feature learning for activity recognition in ubiquitous computing’. IJCAI 2011 – 22nd Int. Joint Conf. on Artificial Intelligence, Barcelona, Spain, 2011, pp. 1729–1734.
44. 44)
  - 60. Arjovsky, M., Chintala, S., Bottou, L.: ‘Wasserstein GAN’, arXiv, 2016, 50, (6), pp. 35–42.
45. 45)
  - 11. Osogami, T.: ‘Boltzmann machines for time-series’. ‘Technical Report RT0980’, Tokyo, IBM Research, 2017.
46. 46)
  - 46. Leng, W.Y., Shamsuddin, S.M.: ‘Writer identification for Chinese handwriting’, Int. J. Adv. Soft Comput. Appl., 2010, 2, (2), pp. 142–173.
47. 47)
  - 14. Längkvist, M., Karlsson, L., Loutfi, A.: ‘A review of unsupervised feature learning for time series classification’, Pattern Recognit. Lett., 2014, 42, pp. 11–24.
48. 48)
  - 32. Keogh, E., Chu, S., Hart, D., et al: ‘An online algorithm for segmenting time series’. Proc. – IEEE Int. Conf. Data Mining (ICDM), San Jose, CA, USA, 2001, pp. 289–296.
49. 49)
  - 38. Song, K., Wang, Y.: ‘Remote activity monitoring of the elderly using a two-axis accelerometer’. Proc. of the CACS Automatic Control Conf., Tainan, Taiwan, 2005, pp. 18–19.
50. 50)
  - 25. Anguita, D., Ghio, A., Oneto, L., et al: ‘A public domain dataset for human activity recognition using smartphones’. ESANN 2013 Proc., 21st European Symp. on Artificial Neural Networks, Computational Intelligence and Machine Learning’, Bruges, Belgium, April 2013, pp. 437–442.
51. 51)
  - 62. Li, X., Shen, H., Li, H., et al: ‘Patch matching-based multitemporal group sparse representation for the missing information reconstruction of remote-sensing images’, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 2016, 9, (8), pp. 3629–3641.
52. 52)
  - 22. Mitchell, E., Monaghan, D., O'Connor, N.E.: ‘Classification of sporting activities using smartphone accelerometers’, Sensors, 2013, 13, (4), pp. 5317–5337.
53. 53)
  - 3. Mishra, M., Nayak, J., Naik, B., et al: ‘Deep learning in electrical utility industry: a comprehensive review of a decade of research’, Eng. Appl. Artif. Intell., 2020, 96, (August), p. 104000.
54. 54)
  - 2. Sezer, O.B., Gudelek, M.U., Ozbayoglu, A.M.: ‘Financial time series forecasting with deep learning: a systematic literature review: 2005–2019’, Appl. Soft Comput. J., 2020, 90, p. 106181.
55. 55)
  - 47. Smolensky, P.: ‘Information processing in dynamical systems: foundations of harmony theory’, Parallel Distrib. Process. Explor. Microstruct. Cogn., 1986, 1, (1), pp. 194–281.
56. 56)
  - 39. Ahmadi, A., Mitchell, E., Destelle, F., et al: ‘Automatic activity classification and movement assessment during a sports training session using wearable inertial sensors’. Proc. – 11th Int. Conf. on Wearable and Implantable Body Sensor Networks (BSN 2014), Zurich, Switzerland, 2014, pp. 98–103.
57. 57)
  - 30. Sajjad Hossain, H.M., Al Hafiz Khan, M.A., Roy, N.: ‘Soccermate: a personal soccer attribute profiler using wearables’. 2017 IEEE Int. Conf. on Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, 2017, pp. 164–169.
58. 58)
  - 65. Urbano, J., Lima, H., Hanjalic, A.: ‘Statistical significance testing in information retrieval: an empirical analysis of type I, type II and type III errors’. SIGIR 2019 – Proc. 42nd Int. ACM SIGIR Conf. Research Development Information Retrieval, Paris, France, 2019, pp. 505–514.
59. 59)
  - 35. Fulcher, B.D., Jones, N.S.: ‘Highly comparative feature-based time-series classification’, IEEE Trans. Knowl. Data Eng., 2014, 26, (12), pp. 3026–3037.
60. 60)
  - 13. Taylor, G.W., Hinton, G.E.: ‘Factored conditional restricted Boltzmann machines for modeling motion style’. Proc. of the 26th Int. Conf. On Machine Learning (ICML 2009), Montreal Quebec Canada, 2009, pp. 1025–1032.
61. 61)
  - 6. Hu, Y., Liu, J.N.K., You, J., et al: ‘Continuous RBM based deep neural network for wind speed forecasting in Hong Kong’. Proc. 2015 Int. Conf. Image Processing Computer Vision, Pattern Recognition (IPCV 2015), Las Vegas, Nevada, USA, 2015, pp. 368–374.
62. 62)
  - 12. Jakšić, Z., Cadenelli, N., Prats, D.B., et al: ‘A highly parameterizable framework for conditional restricted Boltzmann machine based workloads accelerated with FPGAs and OpenCL’, Future Gener. Comput. Syst., 2020, 104, pp. 201–211.
63. 63)
  - 52. Ackley, D., Hinton, G., Sejnowski, T.: ‘A learning algorithm for Boltzmann machines’, Cogn. Sci., 1985, 9, (1), pp. 147–169.
64. 64)
  - 40. Huang, N.E., Shen, Z., Long, S.R., et al: ‘The empirical mode decomposition and the Hubert spectrum for nonlinear and non-stationary time series analysis’, Proc. R. Soc. A Math. Phys. Eng. Sci., 1998, 454, (1971), pp. 903–995.
65. 65)
  - 8. Vengertsev, D.: ‘Deep learning architecture for univariate time series forecasting’, ‘Cs229’, Stanford University, 2014, pp. 3–7.
66. 66)
  - 48. Freund, Y., Haussler, D.: ‘Unsupervised learning of distributions on binary vectors using two layer networks’, Adv. Neural Inf. Process. Syst., 1992, 4, pp. 912–919.

Conditional restricted Boltzmann machine as a generative model for body-worn sensor signals

References

Related content