The role of respiration audio in multimodal analysis of movement qualities

Lussu, Vincenzo; Niewiadomski, Radosław; Volpe, Gualtiero; Camurri, Antonio

doi:10.1007/s12193-019-00302-1

Cited by 5 publications

(4 citation statements)

References 56 publications

(68 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is unsurprising given that the brain is involved in motor control, processing of spatial and temporal information, and motor learning [36]. Another process relevant to movement is respiration, which has for instance been found informative for capturing differences in movement qualities [58]. Respiration has further been implicated in motor control itself, and it is additionally associated with emotional experiences which can influence movement [83].…”

Section: Beyondmentioning

confidence: 99%

Human Movement Datasets: An Interdisciplinary Scoping Review

Olugbade

Bieńkiewicz²,

Barbareschi

et al. 2022

ACM Comput. Surv.

Self Cite

View full text Add to dashboard Cite

Movement dataset reviews exist but are limited in coverage, both in terms of size and research discipline. While topic-specific reviews clearly have their merit, it is critical to have a comprehensive overview based on a systematic survey across disciplines. This enables higher visibility of datasets available to the research communities and can foster interdisciplinary collaborations. We present a catalogue of 704 open datasets described by 10 variables that can be valuable to researchers searching for secondary data: name and reference, creation purpose, data type, annotations, source, population groups, ordinal size of people captured simultaneously, URL, motion capture sensor, and funders. The catalogue is available in the supplementary materials. We provide an analysis of the datasets and further review them under the themes of human diversity, ecological validity, and data recorded. The resulting 12-dimension framework can guide researchers in planning the creation of open movement datasets. This work has been the interdisciplinary effort of researchers across affective computing, clinical psychology, disability innovation, ethnomusicology, human-computer interaction, machine learning, music cognition, music computing, and movement neuroscience.

show abstract

Section: Beyondmentioning

confidence: 99%

Human Movement Datasets: An Interdisciplinary Scoping Review

Olugbade

Bieńkiewicz²,

Barbareschi

et al. 2022

ACM Comput. Surv.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Within performing arts the physiological data of the performer can be also collected. For instance, Niewiadomski et al ( 2017 ) and Lussu et al ( 2019 ) explored whether the respiration signal captured by a standard microphone placed near to mouth can be used to classify the expressive movements. For this purpose, they collected the respiration data of dancers performing the sequences of expressive movements.…”

Section: Common Emotion Elicitation Techniques Used For Data Collementioning

confidence: 99%

Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection

et al. 2020

Self Cite

View full text Add to dashboard Cite

Emotion, mood, and stress recognition (EMSR) has been studied in laboratory settings for decades. In particular, physiological signals are widely used to detect and classify affective states in lab conditions. However, physiological reactions to emotional stimuli have been found to differ in laboratory and natural settings. Thanks to recent technological progress (e.g., in wearables) the creation of EMSR systems for a large number of consumers during their everyday activities is increasingly possible. Therefore, datasets created in the wild are needed to insure the validity and the exploitability of EMSR models for real-life applications. In this paper, we initially present common techniques used in laboratory settings to induce emotions for the purpose of physiological dataset creation. Next, advantages and challenges of data collection in the wild are discussed. To assess the applicability of existing datasets to real-life applications, we propose a set of categories to guide and compare at a glance different methodologies used by researchers to collect such data. For this purpose, we also introduce a visual tool called Graphical Assessment of Real-life Application-Focused Emotional Dataset (GARAFED). In the last part of the paper, we apply the proposed tool to compare existing physiological datasets for EMSR in the wild and to show possible improvements and future directions of research. We wish for this paper and GARAFED to be used as guidelines for researchers and developers who aim at collecting affect-related data for real-life EMSR-based applications.

show abstract

“…There have been multiple methods to extract the respiration pattern from a human subject including strain sensors [23], microphones [24] and video signals [25]. However, in this work, video-based respiration extraction is considered since it is contact-free and requires no specialized hardware other than a video camera.…”

Section: A Role Of Respiration In Speech Productionmentioning

confidence: 99%

RespVAD: Voice Activity Detection via Video-Extracted Respiration Patterns

Mondal

Prathosh

2020

IEEE Sens. Lett.

View full text Add to dashboard Cite

Voice Activity Detection (VAD) refers to the task of identification of regions of human speech in digital signals such as audio and video. While VAD is a necessary first step in many speech processing systems, it poses challenges when there are high levels of ambient noise during the audio recording. To improve the performance of VAD in such conditions, several methods utilizing the visual information extracted from the region surrounding the mouth/lip region of the speakers' video recording have been proposed. Even though these provide advantages over audio-only methods, they depend on faithful extraction of lip/mouth regions. Motivated by these, a new paradigm for VAD based on the fact that respiration forms the primary source of energy for speech production is proposed. Specifically, an audio-independent VAD technique using the respiration pattern extracted from the speakers' video is developed. The Respiration Pattern is first extracted from the video focusing on the abdominal-thoracic region of a speaker using an optical flow based method. Subsequently, voice activity is detected from the respiration pattern signal using neural sequence-to-sequence prediction models. The efficacy of the proposed method is demonstrated through experiments on a challenging dataset recorded in real acoustic environments and compared with four previous methods based on audio and visual cues. Data and the code for our implementation will be made available at https://github.com/arnabkmondal/RespVAD.

show abstract

The role of respiration audio in multimodal analysis of movement qualities

Cited by 5 publications

References 56 publications

Human Movement Datasets: An Interdisciplinary Scoping Review

Human Movement Datasets: An Interdisciplinary Scoping Review

Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection

RespVAD: Voice Activity Detection via Video-Extracted Respiration Patterns

Contact Info

Product

Resources

About