Reconstructing head models from photographs for individualized 3D‐audio processing

Abstract

Visual fidelity and interactivity are the main goals in Computer Graphics research, but recently also audio is assuming an important role. Binaural rendering can provide extremely pleasing and realistic three-dimensional sound, but to achieve best results it's necessary either to measure or to estimate individual Head Related Transfer Function (HRTF). This function is strictly related to the peculiar features of ears and face of the listener. Recent sound scattering simulation techniques can calculate HRTF starting from an accurate 3D model of a human head. Hence, the use of binaural rendering on large scale (i.e. video games, entertainment) could depend on the possibility to produce a sufficiently accurate 3D model of a human head, starting from the smallest possible input. In this paper we present a completely automatic system, which produces a 3D model of a head starting from simple input data (five photos and some key-points indicated by user). The geometry is generated by extracting information from images and accordingly deforming a 3D dummy to reproduce user head features. The system proves to be fast, automatic, robust and reliable: geometric validation and preliminary assessments show that it can be accurate enough for HRTF calculation.

Automatic extraction of pinna edges for binaural audio customization

“…Jeges' edge extraction algorithm was also a critical component of the reconstruction method by Dellepiane et al [20] which interactively adapted a 3D head model to a specific user starting from a set of pictures of the head and pinna. Following a complementary approach to our anthropometrybased HRTF customization techniques, the resulting model was fed to a simplified boundary element method solver in order to simulate custom HRTFs for that user.…”

Markerless reconstruction of dynamic facial expressions

“…In an image based approach, they optimize these parameters to extract the geometry and texture of a human head from a photo. In a related approach Dellepiane et al [6] deform a dummy head to reconstruct the shapes of human heads from images and used them for binaural rendering. Active Appearance Models (AAMs), as in [20,15], are used to track motion through (multiview) image sequences.…”

Estimation of the low-frequency components of the head-related transfer functions of animals from photographs

“…The morphological parameters needed to feed the geometrical model can be easily estimated, for example, from photographs (Pernaux, 2003;Zotkin et al, 2003;Dellepiane et al, 2008). However, the resulting estimated HRTFs are only approximate and have generally been limited to humans.…”

“…Simplified geometrical models such as spherical models (Brown and Duda, 1998;Duda et al, 1999;Algazi et al, 2001b), head and torso models (Algazi et al, 2002;Pernaux, 2003;Zotkin et al, 2003), or dummy head models (Dellepiane et al, 2008) have also been proposed to estimate human listener's HRTFs. The morphological parameters needed to feed the geometrical model can be easily estimated, for example, from photographs (Pernaux, 2003;Zotkin et al, 2003;Dellepiane et al, 2008).…”