Breathing life into shape

Tsoli, Aggeliki; Mahmood, Naureen; Black, Michael J.

doi:10.1145/2601097.2601225

Cited by 39 publications

(3 citation statements)

References 34 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Models of tidal breathing that strive to be anatomically accurate include those developed by [28,32]. Recent models are usually based on motion data captured from participants [23,27]. Modeling work on breathing in conjunction with speaking is sparse, and the work of [12] on the development of infant speech production one of the few published works.…”

Section: Breathing In Virtual Human Charactersmentioning

confidence: 99%

See 1 more Smart Citation

Speech Breathing in Virtual Humans: An Interactive Model and Empirical Study

Bernardet

Kanq

Feng

et al. 2019

2019 IEEE Virtual Humans and Crowds for Immersive Environments (VHCIE)

View full text Add to dashboard Cite

Human speech production requires the dynamic regulation of air through the vocal system. While virtual character systems commonly are capable of speech output, they rarely take breathing during speaking-speech breathing-into account. We believe that integrating dynamic speech breathing systems in virtual characters can significantly contribute to augmenting their realism. Here, we present a novel control architecture aimed at generating speech breathing in virtual characters. This architecture is informed by behavioral, linguistic and anatomical knowledge of human speech breathing. Based on textual input and controlled by a set of lowand high-level parameters, the system produces dynamic signals in real-time that control the virtual character's anatomy (thorax, abdomen, head, nostrils, and mouth) and sound production (speech and breathing). In addition, we perform a study to determine the effects of including breathing-motivated speech movements, such as head tilts and chest expansions during dialogue on a virtual character, as well as breathing sounds. This study includes speech that is generated both from a text-to-speech engine as well as from recorded voice.

show abstract

Section: Breathing In Virtual Human Charactersmentioning

confidence: 99%

“…Future steps include the improvement of the breathing animation to a state of the art implementation as presented e.g. in [27]. At the conceptual level, the system will be extended to include the control of speech breathing parameters with the goal of generating the expression of abstract constructs such as personality and emotion.…”

Section: Future Stepsmentioning

confidence: 99%

Speech Breathing in Virtual Humans: An Interactive Model and Empirical Study

Bernardet

Kanq

Feng

et al. 2019

2019 IEEE Virtual Humans and Crowds for Immersive Environments (VHCIE)

View full text Add to dashboard Cite

show abstract

“…3D human pose estimation aims to estimate the coordinates of human joints in 3D space by inputting an image or video. It is an important task in the field of computer vision and artificial intelligence, and is widely used in areas such as action recognition [1][2][3], robotics [4][5][6], and animation [7][8][9], and can also provide information for other computer vision tasks. Currently, 3D human pose estimation methods can be broadly divided into two categories: two-stage 3D human pose estimation method and end-to-end 3D human pose estimation method.…”

Section: Introductionmentioning

confidence: 99%

End-to-end 3D Human Pose Estimation using Dual Decoders

Wang¹,

Wang²,

Song³

et al. 2023

Preprint

View full text Add to dashboard Cite

Existing methods for 3D human pose estimation mainly divide the task into two stages. The first stage identifies the 2D coordinates of the human joints in the input image, namely the 2D human joint coordinates. The second stage uses the results from the first stage as input to recover the depth information of human joints from the 2D human joint coordinates to achieve 3D human pose estimation. However, the recognition accuracy of the two-stage method relies heavily on the results of the first stage and includes too many redundant processing steps, which reduces the inference efficiency of the network. To address these issues, we propose the EDD, a fully End-to-end 3D human pose estimation method based on transformer architecture with Dual Decoders. By learning multiple human poses, the model can directly infer all 3D human poses in the image using a pose decoder, and then further optimize the recognition result using a joint decoder based on the kinematic relations between joints. With the attention mechanism, this method can adaptively focus on the most relevant features to the target joint, effectively overcoming the feature misalignment problem in the human pose estimation task and greatly improving the model performance. Any complex post-processing step, such as non-maximum suppression, is eliminated, further improving the efficiency of the model. The results show that the method achieves an accuracy of 87.4% on the MuPoTS-3D dataset, significantly improving the accuracy of end-to-end 3D human pose estimation methods.

show abstract

Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors

Zhao

Younes

et al. 2018

Computer Animation & Virtual

View full text Add to dashboard Cite

In the last decade, 3D modeling techniques enjoyed a booming development in both hardware and software. High-end hardware generates high fidelity results, but the cost is prohibitive, whereas consumer-level devices generate plausible results for entertainment purposes but are not appropriate for medical uses. We present a cost-effective and easy-to-use 3D body reconstruction system using consumer-grade depth sensors, which provides reconstructed body shapes with a high degree of accuracy and reliability appropriate for medical applications. Our surface registration framework integrates the articulated motion assumption, global loop closure constraint, and a general as-rigid-as-possible deformation model. To enhance the reconstruction quality, we propose a novel approach to accurately infer skeletal joints from anatomical data using multimodality registration. We further propose a supervised predictive model to infer the skeletal joints for arbitrary subjects independent from anatomical data reference. A rigorous validation test has been conducted on real subjects to evaluate the reconstruction accuracy and repeatability. Our system has the potential to make accurate body surface scanning systems readily available for medical professionals and the general public. The system can be used to obtain additional health data derived from 3D body shapes, such as the percentage of body fat.

show abstract

Breathing life into shape

Cited by 39 publications

References 34 publications

Speech Breathing in Virtual Humans: An Interactive Model and Empirical Study

Speech Breathing in Virtual Humans: An Interactive Model and Empirical Study

End-to-end 3D Human Pose Estimation using Dual Decoders

Accurate nonrigid 3D human body surface reconstruction using commodity depth sensors

Contact Info

Product

Resources

About