GaitFi: Robust Device-Free Human Identification via WiFi and Vision Multimodal Learning

Abstract

As an important biomarker for human identification, human gait can be collected at a distance by passive sensors without subject cooperation, which plays an essential role in crime prevention, security detection and other human identification applications. At present, most research works are based on cameras and computer vision techniques to perform gait recognition. However, vision-based methods are not reliable when confronting poor illuminations, leading to degrading performances. In this paper, we propose a novel multimodal gait recognition method, namely GaitFi, which leverages WiFi signals and videos for human identification. In GaitFi, Channel State Information (CSI) that reflects the multi-path propagation of WiFi is collected to capture human gaits, while videos are captured by cameras. To learn robust gait information, we propose a Lightweight Residual Convolution Network (LRCN) as the backbone network, and further propose the two-stream GaitFi by integrating WiFi and vision features for the gait retrieval task. The GaitFi is trained by the triplet loss and classification loss on different levels of features. Extensive experiments are conducted in the real world, which demonstrates that the GaitFi outperforms state-of-the-art gait recognition methods based on single WiFi or camera, achieving 94.2% for human identification tasks of 12 subjects.

WiAudina: Multimodal Gait Sensing Using Wi-Fi Signal and Acoustic Information

“…In this study by Lang Deng et al [4], a system called GaitFi is proposed, which is a person identification system based on gait using CSI and video. That achieves a person identification accuracy of 94.2% by performing feature fusion after extracting features for each modality.…”

PGGait: Gait Recognition Based on Millimeter-Wave Radar Spatio-Temporal Sensing of Multidimensional Point Clouds

“…GaitWay [ 20 ] investigates non-contact gait recognition by recognizing gait speed through wall monitoring. GaitFi [ 21 ], on the other hand, employs CSI reflected by WiFi multipath propagation to capture human gait while incorporating camera-captured video for robust gait recognition. Nevertheless, as previously highlighted, CSI exhibits high sensitivity to noise from diverse environmental directions.…”

Preface: Celebrating the 70th Anniversary of School of Mechanical Science and Engineering, Huazhong University of Science and Technology

“…2) Comparisons of Different Models: To compare the RCNN model with other models, we conducted an experiment in a meeting room scenario with transmissive RIS. We compare our method with novel WiFi-based human sensing methods, including CNN-GRU [17], CNN-GRU-CNN [17], LSTM-CNN [18] and GaitFi [19], as shown in Table III.…”