Human Activity Classification Incorporating Egocentric Video and Inertial Measurement Unit Data

Lu, Yantao; Velipasalar, Senem

doi:10.1109/globalsip.2018.8646367

Cited by 7 publications

(8 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our recognition result achieved 49.89% for action recognition using only IMUs data from left and right hands. This result is better than [25] by around 5% having in consideration that we are recognizing 9 actions from only 2 sensors while [25] method was used to recognize only 6 action with 4 IMUs sensors, and they are using LSTM for feature extractions and we are using statistical feature extraction method. On the other hand, [34] achieved 62% f-measure recognition score by using only Accelerometer data captured from IMUs sensors attached to non-human objects.…”

Section: Resultsmentioning

confidence: 91%

“…However, for real life scenarios, realistic and compromised number of modalities should be used. Lu and Velipasalar [25] used LSTM to classify actions using four IMUs sensors corresponding 36 components with egocentric video from CMU Multimodal Activity (CMU-MMAC) Database [41]. Visual and audio sensors was used by [5] for activity recognition.…”

Section: Related Workmentioning

confidence: 99%

“…etc. [25]. The main objective of this paper is to recognize complex actions that have similar but opposite nature for example (open, close).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Action Recognition Using Local Visual Descriptors and Inertial Data

Alhersh

Belhaouari

Stuckenschmidt

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Different body sensors and modalities can be used in human action recognition, either separately or simultaneously. Multi-modal data can be used in recognizing human action. In this work we are using inertial measurement units (IMUs) positioned at left and right hands with first person vision for human action recognition. A novel statistical feature extraction method was proposed based on curvature of the graph of a function and tracking left and right hand positions in space. Local visual descriptors have been used as features for egocentric vision. An intermediate fusion between IMUs and visual sensors has been performed. Despite of using only two IMUs sensors with egocentric vision, our classification result achieved is 99.61% for recognizing nine different actions. Feature extraction step could play a vital step in human action recognition with limited number of sensors, hence, our method might indeed be promising.

show abstract

Section: Resultsmentioning

confidence: 91%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Action Recognition Using Local Visual Descriptors and Inertial Data

Alhersh

Belhaouari

Stuckenschmidt

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…In real-world circumstances, however, it is preferable to have a reasonable and balanced number of modalities. Lu et al [31] used LSTM to categorize activities utilizing four IMU sensors and egocentric video from the CMU Multimodal Activity (CMU-MMAC) database [32]. For activity recognition, [33] used visual and audio sensors.…”

Section: Related Work and Contributionsmentioning

confidence: 99%

Learning Human Activity From Visual Data Using Deep Learning

et al. 2021

View full text Add to dashboard Cite

Advances in wearable technologies have the ability to revolutionize and improve people's lives. The gains go beyond the personal sphere, encompassing business and, by extension, the global economy. The technologies are incorporated in electronic devices that collect data from consumers' bodies and their immediate environment. Human activities recognition, which involves the use of various body sensors and modalities either separately or simultaneously, is one of the most important areas of wearable technology development. In real-life scenarios, the number of sensors deployed is dictated by practical and financial considerations. In the research for this article, we reviewed our earlier efforts and have accordingly reduced the number of required sensors, limiting ourselves to first-person vision data for activities recognition. Nonetheless, our results beat state of the art by more than 4% of F1 score.

show abstract

“…Twenty US states including California, Texas, and New York have recently passed legislation to enable testing and deployment of autonomous vehicles. Since the control successors are highly depended on the outputs from the object detector and neural networks [1][2] [3], the performance of object detection is significant to the security of autonomous driving. However, neural network based object detectors are shown to be vulnerable to adversarial examples by recent researches, inputs that are carefully crafted to fool the model [4] [5][6] [7].…”

Section: Introductionmentioning

confidence: 99%

Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles

Velipasalar

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Recent works have shown that neural networks are vulnerable to carefully crafted adversarial examples (AE). By adding small perturbations to input images, AEs are able to make the victim model predicts incorrect outputs. Several research work in adversarial machine learning started to focus on the detection of AEs in autonomous driving. However, the existing studies either use preliminary assumption on outputs of detections or ignore the tracking system in the perception pipeline. In this paper, we firstly propose a novel distance metric for practical autonomous driving object detection outputs. Then, we bridge the gap between the current AE detection research and the real-world autonomous systems by providing a temporal detection algorithm, which takes the impact of tracking system into consideration. We preform evaluation on Berkeley Deep Drive (BDD) and CityScapes datasets to show how our approach outperforms existing single-frame-mAP based AE detections by increasing 17.76% accuracy of performance.

show abstract

Human Activity Classification Incorporating Egocentric Video and Inertial Measurement Unit Data

Cited by 7 publications

References 18 publications

Action Recognition Using Local Visual Descriptors and Inertial Data

Action Recognition Using Local Visual Descriptors and Inertial Data

Learning Human Activity From Visual Data Using Deep Learning

Weighted Average Precision: Adversarial Example Detection in the Visual Perception of Autonomous Vehicles

Contact Info

Product

Resources

About