Applying Multiple Knowledge to Sussex-Huawei Locomotion Challenge

Gjoreski, Martin; Janko, Vito; Reščič, Nina; Mlakar, Miha; Luštrek, Mitja; Bizjak, Jani; Slapničar, Gašper; Marinko, Matej; Drobnič, Vid; Gams, Matjaž

doi:10.1145/3267305.3267515

Cited by 19 publications

(24 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The sound and vision classifiers are among the first works that are applied to transportation mode recognition. The motion classifier was used to benchmark the SHL Challenge 2018 [17] and performed slightly worse than the winner of that challenge [32]. Since this paper mainly focuses on multimodal fusion, we did not aim to maximize the performance of each mono-modal classifier.…”

Section: Discussionmentioning

confidence: 99%

“…While the classifiers show promising results on the same user, the performance drops significantly in case of user variation. In future, more techniques could be employed to tackle the over-fitting problem, including some techniques reported in the SHL challenge, such as augmented learning, transfer learning, and designing hand-crafted features [32], [35], [36]. In essence, multimodal fusion improves the recognition performance at the expense of increasing the sensor channels and also the computational complexity.…”

Section: Discussionmentioning

confidence: 99%

“…The Sussex-Huawei Locomotion-Transportation (SHL) dataset is one of the biggest multimodal dataset for transportation and locomotion mode recognition from mobile devices [10], [32]. The dataset was recorded over 7 months by 3 users engaging in 8 different transportation modes: Still, Walk, Run, Bike, Car, Bus, Train and Subway.…”

Section: Datasetmentioning

confidence: 99%

“…In this paper we conduct the first work that combines the motion, sound and vision modalities for transportation mode recognition. The state-of-the-art Sussex-Huawei Locomotion-Transportation dataset [10], [32] contains rich sensor modalities (including the three above mentioned sensors), which enable us to carry out this research, in order to recognize eight modes of transportation: being still, walking, running, cycling, being in a car, being on a bus, train or subway (Sec. II).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Transportation mode recognition fusing wearable motion, sound and vision sensors

et al. 2020

View full text Add to dashboard Cite

We present the first work that investigates the potential of improving the performance of transportation mode recognition through fusing multimodal data from wearable sensors: motion, sound and vision. We first train three independent deep neural network (DNN) classifiers, which work with the three types of sensors, respectively. We then propose two schemes that fuse the classification results from the three mono-modal classifiers. The first scheme makes an ensemble decision with fixed rules including Sum, Product, Majority Voting, and Borda Count. The second scheme is an adaptive fuser built as another classifier (including Naive Bayes, Decision Tree, Random Forest and Neural Network) that learns enhanced predictions by combining the outputs from the three mono-modal classifiers. We verify the advantage of the proposed method with the state-of-the-art Sussex-Huawei Locomotion and Transportation (SHL) dataset recognizing the eight transportation activities: Still, Walk, Run, Bike, Bus, Car, Train and Subway. We achieve F1 scores of 79.4%, 82.1% and 72.8% with the mono-modal motion, sound and vision classifiers, respectively. The F1 score is remarkably improved to 94.5% and 95.5% by the two data fusion schemes, respectively. The recognition performance can be further improved with a post-processing scheme that exploits the temporal continuity of transportation. When assessing generalization of the model to unseen data, we show that while performance is reduced-as expected-for each individual classifier, the benefits of fusion are retained with performance improved by 15 percentage points. Besides the actual performance increase, this work, most importantly, opens up the possibility for dynamically fusing modalities to achieve distinct power-performance trade-off at run time.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Transportation mode recognition fusing wearable motion, sound and vision sensors

et al. 2020

View full text Add to dashboard Cite

show abstract

“…In other words, there is often more "human optimization" involved in using DL effectively. However, the best DL performance (JSI-Deep [1], 93.9%) is only 1.5% higher than the best ML performance (JSI-Classic [2], 92.4%). While the approach used by JSI-Deep is categorized as a DL pipeline, it is actually an ensemble of deep and classical machine learning models.…”

Section: Summary Of Approachesmentioning

confidence: 97%

Summary of the Sussex-Huawei locomotion-transportation recognition challenge 2019

Wang

Gjoreski

Ciliberto

et al. 2019

Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of The

View full text Add to dashboard Cite

In this paper we summarize the contributions of participants to the Sussex-Huawei Transportation-Locomotion (SHL) Recognition Challenge organized at the HASCA Workshop of UbiComp 2018. The SHL challenge is a machine learning and data science competition, which aims to recognize eight transportation activities (Still, Walk, Run, Bike, Bus, Car, Train, Subway) from the inertial and pressure sensor data of a smartphone. We introduce the dataset used in the challenge and the protocol for the competition. We present a meta-analysis of the contributions from 19 submissions, their approaches, the software tools used, computational cost and the achieved results. Overall, two entries achieved F1 scores above 90%, eight with F1 scores between 80% and 90%, and nine between 50% and 80%.

show abstract