| Title |
Hand Gesture Classification Using Early Fusion Based Multimodal Deep Learning |
| Authors |
김익진(Ik-Jin Kim) ; 김수열(Su-Yeol Kim) ; 이용찬(Yong-Chan Lee) ; 이연정(Yun-Jung Lee) |
| DOI |
https://doi.org/10.5370/KIEE.2021.70.11.1714 |
| Keywords |
Hand Gesture Classification; Deep Learning; EMG; Multimodal Learning; Ninapro DB |
| Abstract |
In this paper, we propose a new hand gesture classification strategy using early fusion based multimodal deep learning. The structure and parameters of the state-of-the-art deep learning models such as ResNet152, DenseNet201, EfficientNetB0 for the source task of image classification are reused in the target task of hand gesture classification using surface electromyograph(EMG) and finger's kinematic data. The time-domain EMG and kinematic signals are normalized and then transformed into combined 2-D images for the early-fusion network. The experimental results support the superiority of the proposed method in terms of classification accuracy. The transfer learning model with the EfficientNetB0 shows the 93.94% accuracy for 40 gestures of 40 participants in the Ninapro DB2. |