Despite the recent success of deep learning in continuous sign language recognition (CSLR), deep models typically focus on the most discriminative features, ignoring other potentially non-trivial and informative contents. Such characteristic heavily constrains their capability to learn implicit visual grammars behind the collaboration of different visual cues (i,e., hand shape, facial expression and body posture). By injecting multi-cue learning into neural network design, we propose a spatial-temporal multi-cue (STMC) network to solve the vision-based sequence learning problem. Our STMC network consists of a spatial multi-cue (SMC) module and a temporal multi-cue (TMC) module. The SMC module is dedicated to spatial representation and explicitly decomposes visual features of different cues with the aid of a self-contained pose estimation branch. The TMC module models temporal correlations along two parallel paths, i.e., intra-cue and inter-cue, which aims to preserve the uniqueness and explore the collaboration of multiple cues. Finally, we design a joint optimization strategy to achieve the end-to-end sequence learning of the STMC network. To validate the effectiveness, we perform experiments on three large-scale CSLR benchmarks: PHOENIX-2014, CSL and PHOENIX-2014-T. Experimental results demonstrate that the proposed method achieves new state-of-the-art performance on all three benchmarks.

Falling, as one of the main harm threats to the elderly, has drawn researchers' attentions and has always been one of the most valuable research topics in the daily health-care for the elderly in last two decades. Before 2014, several researchers reviewed the development of fall detection, presented issues and challenges, and navigated the direction for the study in the future. With smart sensors and Internet of Things (IoT) developing rapidly, this field has made great progress. However, there is a lack of a review and discussion on novel sensors, technologies and algorithms introduced and employed from 2014, as well as the emerging challenges and new issues. To bridge this gap, we present an overview of fall detection research and discuss the core research questions on this topic. A total of 6830 related documents were collected and analyzed based on the key words. Among these documents, the twenty most influential and highly cited articles are selected and discussed profoundly from three perspectives: sensors, algorithms and performance. The findings would assist researchers in understanding current developments and barriers in the systems of fall detection. Although researchers achieve fruitful work and progress, this research domain still confronts challenges on theories and practice. In the near future, the new solutions based on advanced IoT will sustainably urge the development to prevent falling injuries.

No abstract

No abstract

No abstract