Convolutional Neural Networks for User Identification Based on Motion Sensors Represented as Images

Benegui, Cezara; Ionescu, Radu Tudor

doi:10.1109/access.2020.2984214

Cited by 30 publications

(38 citation statements)

References 44 publications

(91 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within the wide range of explored approaches, there are studies that perform user recognition based on voice and accelerometer signals [34], as well as studies that perform human movement tracking based on motion sensors [32]. Regarding the considered approach, it is clear that the newest and best-performing methods belong to the category of deep learning approaches [30,36]. Until now, researchers studied recurrent neural networks [30] and convolutional neural networks [36].…”

Section: Authentication Based On Motion Sensorsmentioning

confidence: 99%

“…Regarding the considered approach, it is clear that the newest and best-performing methods belong to the category of deep learning approaches [30,36]. Until now, researchers studied recurrent neural networks [30] and convolutional neural networks [36]. To our knowledge, none of the previous works investigated ensemble methods that combine recurrent and convolutional neural networks.…”

Section: Authentication Based On Motion Sensorsmentioning

confidence: 99%

“…Due to the small set of motion signals in our multi-modal data set, we pre-train our CNN and ConvLSTM models on a subset of 50 users from the HMOG data set [32]. Following the experimental settings described in [36], we collect motion signals for 200 tap events per user, generating a data set of 10.000 data samples in total. Employing an 80-20% split of the data, we utilize 8000 samples for training and 2000 for validation.…”

Section: Data Setsmentioning

confidence: 99%

See 2 more Smart Citations

Improving the Authentication with Built-In Camera Protocol Using Built-In Motion Sensors: A Deep Learning Solution

Benegui

Ionescu

2021

Mathematics

Self Cite

View full text Add to dashboard Cite

In this paper, we propose an enhanced version of the Authentication with Built-in Camera (ABC) protocol by employing a deep learning solution based on built-in motion sensors. The standard ABC protocol identifies mobile devices based on the photo-response non-uniformity (PRNU) of the camera sensor, while also considering QR-code-based meta-information. During registration, users are required to capture photos using their smartphone camera. The photos are sent to a server that computes the camera fingerprint, storing it as an authentication trait. During authentication, the user is required to take two photos that contain two QR codes presented on a screen. The presented QR code images also contain a unique probe signal, similar to a camera fingerprint, generated by the protocol. During verification, the server computes the fingerprint of the received photos and authenticates the user if (i) the probe signal is present, (ii) the metadata embedded in the QR codes is correct and (iii) the camera fingerprint is identified correctly. However, the protocol is vulnerable to forgery attacks when the attacker can compute the camera fingerprint from external photos, as shown in our preliminary work. Hence, attackers can easily remove their PRNU from the authentication photos without completely altering the probe signal, resulting in attacks that bypass the defense systems of the ABC protocol. In this context, we propose an enhancement to the ABC protocol, using motion sensor data as an additional and passive authentication layer. Smartphones can be identified through their motion sensor data, which, unlike photos, is never posted by users on social media platforms, thus being more secure than using photographs alone. To this end, we transform motion signals into embedding vectors produced by deep neural networks, applying Support Vector Machines for the smartphone identification task. Our change to the ABC protocol results in a multi-modal protocol that lowers the false acceptance rate for the attack proposed in our previous work to a percentage as low as 0.07%. In this paper, we present the attack that makes ABC vulnerable, as well as our multi-modal ABC protocol along with relevant experiments and results.

show abstract

Section: Authentication Based On Motion Sensorsmentioning

confidence: 99%

Section: Authentication Based On Motion Sensorsmentioning

confidence: 99%

Section: Data Setsmentioning

confidence: 99%

See 1 more Smart Citation

Improving the Authentication with Built-In Camera Protocol Using Built-In Motion Sensors: A Deep Learning Solution

Benegui

Ionescu

2021

Mathematics

Self Cite

View full text Add to dashboard Cite

show abstract

“…A better solution might require more than five training samples and end-to-end training, e.g. by employing deep neural networks [7].…”

Section: Authentication Systemmentioning

confidence: 99%

A breach into the Authentication with Built-in Camera (ABC) Protocol

Benegui¹,

Ionescu²

2020

Preprint

Self Cite

View full text Add to dashboard Cite

<div>[Paper accepted at ACNS 2020]</div><div>In this paper, we propose a simple and effective attack on the recently introduced Smartphone Authentication with Built-in Camera Protocol, called ABC. The ABC protocol uses the photo-response non-uniformity (PRNU) as the main authentication factor in combination with anti-forgery detection systems. The ABC protocol interprets the PRNU as a fingerprint of the camera sensor built-in a smartphone device. The protocol works as follows: during the authentication process, the user is challenged with two QR codes (sent by the server) that need to be photographed with a pre-registered device. In each QR code, the server embeds a unique pattern noise (not visible to the naked eye), called probe signal, that is used to identify potential forgeries. The inserted probe signal is very similar to a genuine fingerprint. The photos of QR codes taken by the user are then sent to the server for verification. The server checks (i) if the photos contain the user's camera fingerprint (used to authenticate the pre-registered device) and (ii) if the photos contain the embedded probe signal. If an adversary tries to remove (subtract) his own camera fingerprint and replace it with the victim's camera fingerprint (computed from photos shared on social media), then he will implicitly remove the embedded probe signal and the attack will fail. The ABC protocol is able to detect these attacks with a false acceptance rate (FAR) of 0.5%. However, the ABC protocol wrongly assumes that the attacker can only determine his own camera fingerprint from the photos of the presented QR codes. The attack proposed in our work is able to get past the anti-forgery detection system with a FAR of 54.1%, simply by estimating the attacker's camera fingerprint from a different set of photos (e.g. five photos) owned by the attacker. This set of photos can be trivially obtained before the attack, allowing the adversary to compute his camera fingerprint independently of the attack. The key to the success of our attack is that the independently computed adversary's camera fingerprint does not contain the probe signal embedded in the QR codes. Therefore, when we subtract the adversary's camera fingerprint and add the victim's camera fingerprint, the embedded probe signal will remain in place. For this reason, the proposed attack can successfully pass through the anti-forgery detection system of the ABC protocol. In this paper, we also propose a potential fix based on analyzing signals from built-in motion sensors, which are not typically shared on social media.</div>

show abstract

“…Some previous works have proposed to encode time series data into image representations, which are then fed into a Convolutional Neural Network (CNN) to perform the classification (e.g., [3,[11][12][13]). For the case of motion sensors data, a number of state-ofthe-art approaches have proposed to form a 2D-image from the input signals, either for user identification (e.g., [1]) or for activity recognition (e.g., [4][5][6]). The main difference with our approach is that in our case the images are purposely built in a way that is particularly suited to be processed by CNNs, according to two widely recognized strengths, i.e., locality and edge detection [10].…”

Section: Introductionmentioning

confidence: 99%

VIDENS: Vision-based User Identification from Inertial Sensors

Guinea

Heinrich

Mühlhäuser

2021

2021 International Symposium on Wearable Computers

View full text Add to dashboard Cite

In this paper we propose the VIDENS (vision-based user identification from inertial sensors) approach, which transforms inertial sensors time-series data into images that represent in pixel form patterns found over time, allowing even a simple CNN to outperform complex ad-hoc deep learning models that combine RNNs and CNNs for user identification. Our evaluation shows promising results when comparing our approach to some relevant existing methods. CCS CONCEPTS• Human-centered computing → Ubiquitous and mobile computing systems and tools; Empirical studies in ubiquitous and mobile computing.

show abstract

Convolutional Neural Networks for User Identification Based on Motion Sensors Represented as Images

Cited by 30 publications

References 44 publications

Improving the Authentication with Built-In Camera Protocol Using Built-In Motion Sensors: A Deep Learning Solution

Improving the Authentication with Built-In Camera Protocol Using Built-In Motion Sensors: A Deep Learning Solution

A breach into the Authentication with Built-in Camera (ABC) Protocol

VIDENS: Vision-based User Identification from Inertial Sensors

Contact Info

Product

Resources

About