Facial expression recognition using ear canal transfer function

EarRumble: Discreet Hands- and Eyes-Free Input by Voluntary Tensor Tympani Muscle Contraction

“…Existing, hand-based interaction techniques detect tap gestures on [9,34,56], mid-air interactions in front of [31,39], and touch gestures around [31,56] the ear. Audio-based principles pointing in the ear were applied to detect changes in facial expression [1], to classify eating episodes [2], and for authentication [5]. Electric-feld and proximity sensing were also used to detect movement of the face [7,36,47].…”

“…The tensor's subconscious contraction accompanies vocalization and swallowing [29], or expecting a startling sound [17], and the ability to voluntarily contract the tensor tympani muscle has been discussed for over one hundred years [44]. Due to the vibrations induced by the tensing muscle, a rumbling sound can be heard during contraction [44], and those who can voluntarily contract the muscle often describe it as fexing, activating, or moving a muscle inside their ear 1 . The sound might be imitated by frmly tensing one's fst and pressing it on the ear which creates a comparable dull rumbling sound.…”

“…by an in-ear camera or acoustic impedance measurements [40,41]. The latter might be even realized with of-the-shelf hardware with noise canceling earphones as they have an in-ear microphone to for the noise canceling feedback loop [1].…”

SonicASL

“…EarEcho [20] utilized the built-in speaker and microphone of the earbud to capture the acoustic profile obtained from the user's ear canal for authentication. Similarly, Amesaka et al [1] developed a facial expression recognition system utilizing the user's ear canal transfer functions corresponding to different facial muscle movements. Bui et al [9] proposed a device called "eBP" to monitor the blood pressure inside the user's ear canal, which brings a positive impact on users' daily health monitoring.…”

“…Another finding is that SonicASL achieves the best recognition performance in the apartment setting, as it is a quiet environment with the least noise interference. 1.03 1.07 1 The battery size for Pixel 3 XL and Pixel 3, is 3430 and 2915. 2 Results are tested under the same data sample (single word gesture) and averaged for 10 runs.…”