Hand gesture recognition based-on three-branch CNN with fine-tuning using MIMO radar
Hand gesture recognition based-on three-branch CNN with fine-tuning using MIMO radar
- Author(s): X. Zheng 1 and Z. Yang 1
- DOI: 10.1049/icp.2021.0509
For access to this article, please select a purchase option:
Buy conference paper PDF
Buy Knowledge Pack
IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.
Thank you
Your recommendation has been sent to your librarian.
- Author(s): X. Zheng 1 and Z. Yang 1
-
-
View affiliations
-
Affiliations:
1:
Guangdong Key Laboratory of Intelligent Information Processing College of Electronics and Information Engineering Shenzhen University , Shenzhen, Guangdong , China
Source:
IET International Radar Conference (IET IRC 2020),
2021
p.
1650 – 1655
-
Affiliations:
1:
Guangdong Key Laboratory of Intelligent Information Processing College of Electronics and Information Engineering Shenzhen University , Shenzhen, Guangdong , China
- Conference: IET International Radar Conference (IET IRC 2020)
- DOI: 10.1049/icp.2021.0509
- ISBN: 978-1-83953-540-6
- Location: Online Conference
- Conference date: 04-06 November 2020
- Format: PDF
In this paper, we employ a 77GHz frequency modulated continuous wave (FMCW) multiple-input-multiple-output (MIMO) radar to achieve hand gesture recognition based on a three-branch convolutional neural network (CNN) with fine-tuning. Firstly, the FMCW MIMO radar is utilized to capture the hand gesture data that are the fast-time-slow-time-antenna 3 dimension (3D) data. Then, by applying the discretize Fourier transform (DFT) to the fast-time and slow-time, the multiple signal classification (MUSIC) approach to the antenna dimension, and the multi-frame accumulation, the range-Doppler-angle temporal signatures can be extracted from the captured data. In order to exploit the temporal and spatial correlations of the hand motions, we construct a three-branch CNN to self-learn the hand gesture signatures from the range, Doppler and angle dimension, respectively, and to recognize 9 hand gestures. The fine-tuning approach is proposed to improve the robustness of the proposed network in recognizing an untrained person's hand gestures. After the fine-tuning, the experiment results show that the proposed approach can recognize 9 hand gestures of all trained persons with an average accuracy over 96% and an untrained person with an average accuracy over 96%.
Inspec keywords: convolutional neural nets; FM radar; millimetre wave radar; gesture recognition; radar signal processing; MIMO radar; radar antennas; signal classification; discrete Fourier transforms; CW radar
Subjects: Image recognition; Neural nets; Computer vision and image processing techniques; Integral transforms in numerical analysis; Radar theory; Radar equipment, systems and applications; Integral transforms in numerical analysis