SilentTalk: Lip reading through ultrasonic sensing on mobile phones

Tan, Jiayao; Nguyen, Cam-Tu; Wang, Xiaoliang

doi:10.1109/infocom.2017.8057099

Cited by 45 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The least mean squares (LMS) algorithm was used here because it requires fewer hardware resources and has good traceability [ 28 ]. The step size of the LMS is fixed at 0.5, to balance the errors and fast convergences [ 29 ]. The effect of interference elimination after applying the Butterworth bandpass filter and LMS filter in the time-domain frequency is shown in Figure 5 , and the spectrograms are shown in Figure 6 .…”

Section: Methodsmentioning

confidence: 99%

ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation

Wong

Huang

Chen

et al. 2023

Sensors

View full text Add to dashboard Cite

Although voice authentication is generally secure, voiceprint-based authentication methods have the drawback of being affected by environmental noise, long passphrases, and large registered samples. Therefore, we present a breakthrough idea for smartphone user authentication by analyzing articulation and integrating the physiology and behavior of the vocal tract, tongue position, and lip movement to expose the uniqueness of individuals while making utterances. The key idea is to leverage the smartphone speaker and microphone to simultaneously transmit and receive speech and ultrasonic signals, construct identity-related features, and determine whether a single utterance is a legitimate user or an attacker. Physiological authentication methods prevent other users from copying or reproducing passwords. Compared to other types of behavioral authentication, the system is more accurately able to recognize the user’s identity and adapt accordingly to environmental variations. The proposed system requires a smaller number of samples because single utterances are utilized, resulting in a user-friendly system that resists mimicry attacks with an average accuracy of 99% and an equal error rate of 0.5% under the three different surroundings.

show abstract

Section: Methodsmentioning

confidence: 99%

ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation

Wong

Huang

Chen

et al. 2023

Sensors

View full text Add to dashboard Cite

show abstract

“…Siri is able to answer questions or carry out tasks depending on the context. Google announced the availability of its voice search capabilities for iOS devices [19]. Since 2014 and continuing to the present day, the demand for virtual assistant services such as giving information and carry-ing out tasks has increased significantly.…”

Section: Survey Of Related Workmentioning

confidence: 99%

IoT‐Based Voice‐Controlled Smart Homes with Source Separation Based on Deep Learning

Alshammri

2023

Journal of Sensors

View full text Add to dashboard Cite

The widespread availability of cutting-edge computer technologies has shed light on the relevance of artificial intelligence (AI) applications in almost all sectors of the economy. As a result of the incorporation of voice control processing into many Internet of Things (IoT) devices, many of these IoT devices may be operated using spoken commands. The environment that is controlled by speech may include several devices, each of which may be used for a separate activity; yet, all of the devices may collect and process the same command at the same time. This may be the case if the devices can communicate with one another. Because other devices may choose to ignore orders that are intended for particular devices if those devices are not equipped to deal with those orders, only the device that is designed to carry out the activity and process the command will be able to carry out the activity. This is because only the device that is designed to carry out the activity and process the command will be able to carry out the activity. On the other hand, when all of the voice-controlled devices capture the command through the microphone, there is a greater chance that it will mix with other sounds coming from a variety of sources. This is because the microphone is being used to capture the command from all of the voice-controlled devices. These noises may include those that are emanating from the television, music systems, and other sounds that are created by activities taking on inside the family, among other things. During the identification of instructions via processing, any blending of other sounds that are not the primary command is regarded as noise and has to be deleted. This is because any such blending is deemed to be noise. The direction of arrival (also known as DOA) of the sound waves is given primary consideration by this approach. This is done at the same time as the performance of the system, and the proposal for it are being evaluated. Based on the angle of arrival estimate, a specific room impulse response (RIR) from a collection of defined RIR is identified as a room acoustic characteristic, and source separation is carried out using the technique of independent component analysis (ICA). Following the completion of the analysis of the signals produced by the split command speech, the characteristics of the speech are retrieved from the signals. The Mel-frequency cepstral coefficients (MFCC) approach is used so that the operation of feature extraction may be carried out. This is the goal of the technique. Following that, a support vector machine classifier is used to the data in order to further split these characteristics into a large range of distinct groups. Comparisons are made between the performance of the SVM classifier and the performance of a large number of different classifiers, including decision trees, which are often used in applications that incorporate machine learning (DT). After analyzing its performance, the multiclass SVM classifier is found to have an accuracy of 91%, according to the conclusions of the study. Utilizing a classifier that is based on a probabilistic neural network, which is sometimes referred to as a PNN, is one way in which the accuracy of future classifications may be enhanced. This particular classifier is made up of three layers: one layer of gated recurrent units (GRU), one layer of long short-term memory (LSTM), and one layer that integrates the two of those different kinds of memory. This classification seems to have obtained an accuracy of 94.5 percent, which is higher than the classification accuracy attained by the multiclass SVM classifier.

show abstract

“…Then the I component of the baseband signals is calculated as In most scenarios, nearby moving objects are the human bodies, which usually cause a frequency shift in the range of 50 ∼ 200Hz [49]. While the maximum frequency shift induced by lip motions usually does not exceed 40Hz [19], [50]. Therefore, FaceLip can leverage the low-pass filter to eliminate the dynamic interference.…”

Section: Motion Verificationmentioning

confidence: 99%

Securing Face Liveness Detection Using Unforgeable Lip Motion Patterns

Zhou,

Wang,

et al. 2021

Preprint

View full text Add to dashboard Cite

Face authentication usually utilizes deep learning models to verify users with high recognition accuracy. However, face authentication systems are vulnerable to various attacks that cheat the models by manipulating the digital counterparts of human faces. So far, lots of liveness detection schemes have been developed to prevent such attacks. Unfortunately, the attacker can still bypass these schemes by constructing wide-ranging sophisticated attacks. We study the security of existing face authentication services (e.g., Microsoft, Amazon, and Face++) and typical liveness detection approaches. Particularly, we develop a new type of attack, i.e., the low-cost 3D projection attack that projects manipulated face videos on a 3D face model, which can easily evade these face authentication services and liveness detection approaches. To this end, we propose FaceLip, a novel liveness detection scheme for face authentication, which utilizes unforgeable lip motion patterns built upon well-designed acoustic signals to enable a strong security guarantee. The unique lip motion patterns for each user are unforgeable because FaceLip verifies the patterns by capturing and analyzing the acoustic signals that are dynamically generated according to random challenges, which ensures that our signals for liveness detection cannot be manipulated. Specially, we develop robust algorithms for FaceLip to eliminate the impact of noisy signals in the environment and thus can accurately infer the lip motions at larger distances. We prototype FaceLip on off-the-shelf smartphones and conduct extensive experiments under different settings. Our evaluation with 44 participants validates the effectiveness and robustness of FaceLip.

show abstract

SilentTalk: Lip reading through ultrasonic sensing on mobile phones

Cited by 45 publications

References 23 publications

ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation

ArtiLock: Smartphone User Identification Based on Physiological and Behavioral Features of Monosyllable Articulation

IoT‐Based Voice‐Controlled Smart Homes with Source Separation Based on Deep Learning

Securing Face Liveness Detection Using Unforgeable Lip Motion Patterns

Contact Info

Product

Resources

About