Drinking Gesture Detection Using Wrist-Worn IMU Sensors with Multi-Stage Temporal Convolutional Network in Free-Living Environments

Wang, Chunzhuo; Kumar, Tapan; Raedt, W. De; Camps, Guido; Hallez, Hans; Vanrumste, Bart

doi:10.1109/embc48229.2022.9871817

Cited by 7 publications

(1 citation statement)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent research has also improved IMUs that are attached to the head or neck, combining sensors with glasses or necklaces so that they are less bulky and users are not aware that they are being worn. Besides IMUs, proximity sensors, piezoelectric sensors and radar sensors are also used to detect hand-to-mouth gestures or jawbone movements [21][22][23]. Pressure sensors are used to measure eating activity as well.…”

Section: Eating and Drinking Activity Detectionmentioning

confidence: 99%

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

Hiraguchi,

Perone,

Toet

et al. 2023

Sensors

View full text Add to dashboard Cite

To monitor adherence to diets and to design and evaluate nutritional interventions, it is essential to obtain objective knowledge about eating behavior. In most research, measures of eating behavior are based on self-reporting, such as 24-h recalls, food records (food diaries) and food frequency questionnaires. Self-reporting is prone to inaccuracies due to inaccurate and subjective recall and other biases. Recording behavior using nonobtrusive technology in daily life would overcome this. Here, we provide an up-to-date systematic overview encompassing all (close-to) publicly or commercially available technologies to automatically record eating behavior in real-life settings. A total of 1328 studies were screened and, after applying defined inclusion and exclusion criteria, 122 studies were included for in-depth evaluation. Technologies in these studies were categorized by what type of eating behavior they measure and which type of sensor technology they use. In general, we found that relatively simple sensors are often used. Depending on the purpose, these are mainly motion sensors, microphones, weight sensors and photo cameras. While several of these technologies are commercially available, there is still a lack of publicly available algorithms that are needed to process and interpret the resulting data. We argue that future work should focus on developing robust algorithms and validating these technologies in real-life settings. Combining technologies (e.g., prompting individuals for self-reports at sensed, opportune moments) is a promising route toward ecologically valid studies of eating behavior.

show abstract

Section: Eating and Drinking Activity Detectionmentioning

confidence: 99%

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

Hiraguchi,

Perone,

Toet

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

Freezing of gait assessment with inertial measurement units and deep learning: effect of tasks, medication states, and stops

Yang

Filtjens

Ginis

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: Freezing of gait (FOG) is an episodic and highly disabling symptom of Parkinson's Disease (PD). Traditionally, FOG assessment relies on time-consuming visual inspection of camera footage. Therefore, previous studies have proposed portable and automated solutions to annotate FOG. However, automated FOG assessment is challenging due to gait variability caused by medication effects and varying FOG-provoking tasks. Moreover, whether automated approaches can differentiate FOG from typical everyday movements, such as volitional stops, remains to be determined. To address these questions, we evaluated an automated FOG assessment model with deep learning (DL) based on inertial measurement units (IMUs). We assessed its performance trained on all standardized FOG-provoking tasks and medication states, as well as on specific tasks and medication states. Furthermore, we examined the effect of adding stopping periods on FOG detection performance. Methods: Twelve PD patients with self-reported FOG (mean age 69.33 +/- 6.28 years) completed a FOG-provoking protocol, including timed-up-and-go and 360-degree turning-in-place tasks in On/Off dopaminergic medication states with/without volitional stopping. IMUs were attached to the pelvis and both sides of the tibia and talus. A multi-stage temporal convolutional network was developed to detect FOG episodes. FOG severity was quantified by the percentage of time frozen (%TF) and the number of freezing episodes (#FOG). The agreement between the model-generated outcomes and the gold standard experts' video annotation was assessed by the intra-class correlation coefficient (ICC). Results: For FOG assessment in trials without stopping, the agreement of our model was strong (ICC(%TF) = 0.92 [0.68, 0.98]; ICC(#FOG) = 0.95 [0.72, 0.99]). Models trained on a specific FOG-provoking task could not generalize to unseen tasks, while models trained on a specific medication state could generalize to unseen states. For assessment in trials with stopping, the model trained on stopping trials made fewer false positives than the model trained without stopping (ICC(%TF) = 0.95 [0.73, 0.99]; ICC(#FOG) = 0.79 [0.46, 0.94]). Conclusion: A DL model trained on IMU signals allows valid FOG assessment in trials with/without stops containing different medication states and FOG-provoking tasks. These results are encouraging and enable future work investigating automated FOG assessment during everyday life.

show abstract

Automatic Detection and Assessment of Freezing of Gait Manifestations

Yang

Filtjens

Ginis

et al. 2023

Preprint

View full text Add to dashboard Cite

Freezing of gait (FOG) is an episodic and highly disabling symptom of Parkinson's disease (PD). Although described as a single phenomenon, FOG is not univocal and can express as different manifestations, such as trembling in place or complete akinesia. We aimed to analyze the utility of deep learning trained on inertial measurement unit data to classify FOG into both manifestations. We developed a temporal convolutional neural network, which we compared to three state-of-the-art FOG detection algorithms that were adapted to the FOG manifestation detection task. Next, we investigated its performance in distinguishing between the two manifestations and other forms of movement cessation (e.g., volitional stopping and sitting) based on gold-standard video annotations. Experiments were conducted on a dataset of twelve PD patients with FOG that completed a FOG-provoking protocol, including the timed-up-and-go and 360-degree turning-in-place tasks during ON and OFF anti-Parkinsonian medication. The results showed that our model enables accurate detection of FOG manifestations with an 11.43% higher F1 score than the second-best model. Assessment of FOG manifestation severity was moderately strong for trembling in place (Intra-class Correlation Coefficient (ICC)=0.64, [0.16,0.88]) and strong for complete akinesia (ICC=0.87, [0.63,0.96]). Remarkably, our results show that complete akinesia can be distinguished from volitional stopping. In conclusion, we established that FOG manifestations could be accurately detected and assessed with deep learning. Future work should establish whether these results hold firm for a more extensive and varied verification cohort.

show abstract

Drinking Gesture Detection Using Wrist-Worn IMU Sensors with Multi-Stage Temporal Convolutional Network in Free-Living Environments

Cited by 7 publications

References 50 publications

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review

Freezing of gait assessment with inertial measurement units and deep learning: effect of tasks, medication states, and stops

Automatic Detection and Assessment of Freezing of Gait Manifestations

Contact Info

Product

Resources

About