Eye-based keystroke prediction for natural texts – a feasibility analysis

Abstract

The use of videoconferencing is on the rise after COVID-19, being common to look at the screen and see someone typing. A side-channel attack may be launched to infer the text written from the face image. In this paper, we analyse the feasibility of such an attack, being the first proposal which work with a complete keyset (50 keys) and natural texts. We use different scenarios, lighting conditions and natural texts to increase realism. Our study involves 30 participants, who typed 49,365 keystrokes. We characterize the effect of lighting, gender, age and use of glasses. Our results show that on average 13.71% of keystrokes are revealed without error, and up to 31.8%, 52.5% and 61.2% are guessed with a maximum error of 1, 2 and 3 keys, respectively.