Real-time hidden acoustic signal capture with smartphones

Wang, Qian; Ren, Kui; Zhou, Man; Lei, Tao; Koutsonikolas, Dimitrios; Lu, Shan

doi:10.1145/2973750.2985618

Cited by 12 publications

(11 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the gap between home and noisy environments becomes less significant at higher frequencies, which results in quite low PSD at the high frequency regardless of the environments. This observation is supported by the similar results in [12]. Therefore, if we use the high-frequency band to avoid human perception, ambient noise does not seem to be a serious problem.…”

Section: Ambient Noisesupporting

confidence: 78%

“…Dolphin [12] uses amplitude shift keying (ASK) and energy difference keying (EDK) depending on frequency ranges and achieves average data rates of up to 500 bps using COTS devices. Similarly, the authors in [13] propose a system for transmitting 900 bps data within music through a loudspeaker and a smartphone, where suitable frequency regions for OFDM subcarriers are determined by exploiting tonal harmonics of existing sound sources.…”

Section: Related Workmentioning

confidence: 99%

“…Audible signals may be annoying to AAC users [2], [6]- [9]. Even if audible signals are embedded into existing audible sound sources to avoid human perception [10], [12]- [14], audible sound sources should be played, and thus it is impossible to transmit signals only. Since ambient noise has higher power in the low-frequency band than in the highfrequency band, it significantly interfere with audible signals.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Reliable and Low-Complexity Chirp Spread Spectrum-Based Aerial Acoustic Communication

Lee

Kwak

Kim³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Aerial acoustic communication (AAC) has been developed for a variety of Internet-of-things (IoT) applications, thanks to the merit of embedding small amounts of data directly into audio contents and transmitting them through commercial off-the-shelf (COTS) devices without additional infrastructure. However, AAC has clear limitations due to ambient noise, severely frequency-selective acoustic channels, and high power consumption of COTS devices. We adopt chirp spread spectrum (CSS) modulation and design new quaternary symbols to overcome frequency selectivity of audio interfaces of COTS devices. We also develop a computation-efficient method to demodulate the proposed symbols, aiming to reduce the power consumption of COTS devices. We employ a frame combining technique without increasing computational complexity, to mitigate the effects of multipath fading and ambient noise in acoustic channels. To evaluate the proposed methods, we conduct extensive experiments with several smartphones in various environments. The experimental results and evaluation demonstrate that the proposed symbols and frame combining method contribute to improving the frame reception ratio by up to 59.8%p (267.9%) and 14%p (84.3%), respectively. The frame combining increases the possibility of receiving a frame within two attempts at extremely low SNR, by up to 107.9%, resulting in reduced excessive delay. The method to demodulate the proposed symbols lowers power consumption by tens to hundreds of milliwatts, depending on the device. Our proposal significantly helps to achieve high reliability and low power consumption of COTS devices in AAC. INDEX TERMS aerial acoustic communication, ambient noise, chirp spread spectrum, correlator, frame combining, software-based digital modem.

show abstract

Section: Ambient Noisesupporting

confidence: 78%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Reliable and Low-Complexity Chirp Spread Spectrum-Based Aerial Acoustic Communication

Lee

Kwak

Kim³

et al. 2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…These works mainly focus on the security-related application designs. There are also some prior works that propose more general methods to embed bits into sounds to achieve a side-channel information delivery [35], [40], [52]. These designs mainly embed bits into the high-frequency band, such as 18 kHz -20 kHz, to minimize the perception of human.…”

Section: Related Workmentioning

confidence: 99%

Metamorph: Injecting Inaudible Commands into Over-the-air Voice Controlled Systems

Chen

Shangguan

et al. 2020

Proceedings 2020 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

This paper presents Metamorph, a system that generates imperceptible audio that can survive over-the-air transmission to attack the neural network of a speech recognition system. The key challenge stems from how to ensure the added perturbation of the original audio in advance at the sender side is immune to unknown signal distortions during the transmission process. Our empirical study reveals that signal distortion is mainly due to device and channel frequency selectivity but with different characteristics. This brings a chance to capture and further pre-code this impact to generate adversarial examples that are robust to the over-the-air transmission. We leverage this opportunity in Metamorph and obtain an initial perturbation that captures the core distortion's impact from only a small set of prior measurements, and then take advantage of a domain adaptation algorithm to refine the perturbation to further improve the attack distance and reliability. Moreover, we consider also reducing human perceptibility of the added perturbation. Evaluation achieves a high attack success rate (90%) over the attack distance of up to 6 m. Within a moderate distance, e.g., 3 m, Metamorph maintains this high success rate, yet can be further adapted to largely improve the audio quality, confirmed by a human perceptibility study. Recent studies [17], [46] have investigated the first step of this attack, i.e., generating an adversarial example I + δ to directly fool a SR without actual over-the-air audio transmission. As Figure 1 depicts, the transcript T ("this is for you") of the input audio I can be recognized as T ("power off") after adding a small perturbation δ. However, these works also find that the proposed technique fail after over-the-air transmission (e.g., the recognized transcript becomes "this is

show abstract

“…A novel form of real-time sound communication method called Dolphin is proposed in [31]. Experiments on ambient noise measurement show that noise can be ignored at frequencies higher than 8 kHz.…”

Section: Related Workmentioning

confidence: 99%

Event-Triggered Ephemeral Group Communication and Coordination over Sound for Smart Consumer Devices

Song

Yeom

Woo

2019

Sensors

View full text Add to dashboard Cite

Voice-based interfaces have become one of the most popular device capabilities, recently being regarded as one flagship user experience of smart consumer devices. However, the lack of common coordination mechanisms might often degrade the user experience, especially when interacting with multiple voice-enabled devices located closely. For example, a hotword or wake-up utterance such as “hi Bixby” or “ok Google” frequently triggers redundant responses by several nearby smartphones. Motivated by the problem of uncoordinated react of voice-enabled devices especially in a multiple device environment, in this paper, we discuss the notion of an ephemeral group of consumer devices in which the member devices and the transient lifetime are implicitly determined by an external event (e.g., hotword detection) without any provisioned group structure, and specifically we concentrate on the time-constrained leader election process in such an ephemeral group. To do so: (i) We first present the sound-based multiple device communication framework, namely tailtag, that leverages the isomorphic capability of consumer devices for the tasks of processing hotword events and transmitting data over sound, and thus renders both the tasks confined to the same room area and enables the spontaneous leader election process in a unstructured group upon a hotword event. (ii) To improve the success rate of the leader election with a given time constraint, we then develop the adaptive messaging scheme especially tailored for sound-based data communication that inherently has low data rate. Our adaptive scheme utilizes an application-specific score that is individually calculated by a member device for each event detection, and employs score-based scheduling by which messages of a high score are scheduled first and so unnecessary message transmission can be suppressed during the election process. (iii) Through experiments, we also demonstrate that, when a hotword is detected by multiple smartphones in a room, the framework with the adaptive messaging scheme enables them to successfully achieve a coordinated response under the given latency bound, yielding an insignificant non-consensus probability, no more than 2%.

show abstract

Real-time hidden acoustic signal capture with smartphones

Cited by 12 publications

References 3 publications

Reliable and Low-Complexity Chirp Spread Spectrum-Based Aerial Acoustic Communication

Reliable and Low-Complexity Chirp Spread Spectrum-Based Aerial Acoustic Communication

Metamorph: Injecting Inaudible Commands into Over-the-air Voice Controlled Systems

Event-Triggered Ephemeral Group Communication and Coordination over Sound for Smart Consumer Devices

Contact Info

Product

Resources

About