Ubiquitous keyboard for small mobile devices

Wang, Junjue; Zhao, Kaichen; Zhang, Xinyu; Peng, Chunyi

doi:10.1145/2594368.2594384

Cited by 125 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sound Localization. Sound localization has been studied [5], [65]. Some systems used a microphone array pointing 360 • around to sense the magnitude of incoming sounds and infer the azimuth of the sound source [27].…”

Section: Related Workmentioning

confidence: 99%

Eavesdropping on Controller Acoustic Emanation for Keystroke Inference Attack in Virtual Reality

Luo,

Nguyen,

Farooq

et al. 2024

Proceedings 2024 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

Understanding the vulnerability of virtual reality (VR) is crucial for protecting sensitive data and building user trust in VR ecosystems. Previous attacks have demonstrated the feasibility of inferring VR keystrokes inside head-mounted displays (HMDs) by recording side-channel signals generated during user-HMD interactions. However, these attacks are heavily constrained by the physical layout or victim pose in the attack scenario since the recording device must be strictly positioned and oriented in a particular way with respect to the victim. In this paper, we unveil a placement-flexible keystroke inference attack in VR by eavesdropping the clicking sounds of the moving hand controller during keystrokes. The malicious recording smartphone can be placed anywhere surrounding the victim, making the attack more flexible and practical to deploy in VR environments. As the first acoustic attack in VR, our system, Heimdall, overcomes unique challenges unaddressed by previous acoustic attacks on physical keyboards and touchscreens. These challenges include differentiating sounds in a 3D space, adaptive mapping between keystroke sound and key in varying recording placement, and handling occasional hand rotations. Experiments with 30 participants show that Heimdall achieves key inference accuracy of 96.51% and top-5 accuracy of 85.14%-91.22% for inferring passwords with 4-8 characters. Heimdall is also robust under various practical impacts such as smartphone-user placement, attack environments, hardware models, and victim conditions.

show abstract

Section: Related Workmentioning

confidence: 99%

Eavesdropping on Controller Acoustic Emanation for Keystroke Inference Attack in Virtual Reality

Luo,

Nguyen,

Farooq

et al. 2024

Proceedings 2024 Network and Distributed System Security Symposium

View full text Add to dashboard Cite

show abstract

“…[97] Kafle  . [98] BreathPrint [101] DolphinAttack [102] PinDrop [109] TeethPass [110] ToothSonic [111] CommanderSong [103] EarEcho [104] LipPass [105] SilentSign [106] EarGate [107] Proximity Echo [108] Apnoea-Pi [92] Fig. 4.…”

Section: Orthogonal Frequency Division Multiplexing (Ofdm)mentioning

confidence: 99%

“…This method can provide a more accurate solution for the AE source location of complex structures including unknown void areas such as tunnels, bridges, railways and caves in practical engineering. [98] uses the active acoustic wave method to locate cracks in water supply pipes, and proposes an active detection and location method based on low-frequency acoustic wave propagation in water pipes to detect and locate leaks in water supply systems. To overcome the major difficulty of statistical processing of time delays associated with multiple sound paths in reverb environments, two approaches are used in this paper: 1) the classical signal decomposition technique (Prony's method) and 2) a clustering pre-processing approach called Spectral Mean-Shift Clustering.…”

Section: Indoor Localizationmentioning

confidence: 99%

“…After obtaining the WiFi or acoustic signal, the next step is to characterize it using various sensing techniques. Similar to WiFi sensing, typical applications based on acoustic sensing include daily actions monitoring [73][74][75][76][77][78] , gesture and hand movements recognition [79][80][81][82][83][84][85][86][87] , health caring [88][89][90][91][92] , localization and navigation [93][94][95][96][97][98] and privacy and security [99][100][101][102][103][104][105][106][107][108][109][110][111] . In the following, we will also introduce the basic content of signals and other characterization methods.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Ubiquitous WiFi and Acoustic Sensing: Principles, Technologies, and Applications

Huang

Wang

Zou

et al. 2023

J. Comput. Sci. Technol.

View full text Add to dashboard Cite

With the increasing pervasiveness of mobile devices such as smartphones, smart TVs, and wearables, smart sensing, transforming the physical world into digital information based on various sensing medias, has drawn researchers' great attention. Among different sensing medias, WiFi and acoustic signals stand out due to their ubiquity and zero hardware cost. Based on different basic principles, researchers have proposed different technologies for sensing applications with WiFi and acoustic signals covering human activity recognition, motion tracking, indoor localization, health monitoring, and the like. To enable readers to get a comprehensive understanding of ubiquitous wireless sensing, we conduct a survey of existing work to introduce their underlying principles, proposed technologies, and practical applications. Besides we also discuss some open issues of this research area. Our survey reals that as a promising research direction, WiFi and acoustic sensing technologies can bring about fancy applications, but still have limitations in hardware restriction, robustness, and applicability.

show abstract

“…For example, finger-tapping on a conventional surface (e.g. wooden table) produces enough audio information for 'keystroke' localization, thus enabling ADI-based text entry [5].…”

Section: Introductionmentioning

confidence: 99%

Contact-free Interaction with Mobile Devices using Magnetic, Lighting and Infrared Sources

Ljubić

Arbula

2017

Int. J. Interact. Mob. Technol.

View full text Add to dashboard Cite

Abstract-In the area of Human-Computer Interaction (HCI), the concept of Around Device Interaction (ADI) is gaining special attention nowadays. It provides the possibility of interacting with the device by making use of gestures made in the surrounding space, as well as by detecting certain ambient changes, thus eliminating the need of direct physical contact. Contact-free interaction can be useful for wearable and smart devices, especially in cases where operating with small buttons and touchscreens appears to be particularly inconvenient. This paper presents alternative techniques for smartphone interaction based on different sensory inputs. Namely, external sources of magnetic field, light, and infrared (IR) signals are used to manipulate device sensors' readings which are in turn interpreted as the corresponding interaction commands. Proof-ofconcept mobile applications are developed and here presented, demonstrating both the potentials and utility of the contact-free interaction. Specifically, small neodymium magnet was used for smartphone text entry, typical battery flashlight served for sending information to device via Morse code, as well as for alarm triggering, and voice-enabled control device was utilized for invoking smartphone calls from a distance. The presented interaction techniques do not require smartphone hardware alterations. While the usage of built-in magnetic field and ambient light sensors is assumed, peripheral infrared receiver can be used in order to provide IR-based support.

show abstract

Ubiquitous keyboard for small mobile devices

Cited by 125 publications

References 23 publications

Eavesdropping on Controller Acoustic Emanation for Keystroke Inference Attack in Virtual Reality

Eavesdropping on Controller Acoustic Emanation for Keystroke Inference Attack in Virtual Reality

Ubiquitous WiFi and Acoustic Sensing: Principles, Technologies, and Applications

Contact-free Interaction with Mobile Devices using Magnetic, Lighting and Infrared Sources

Contact Info

Product

Resources

About