Blink and wink detection as a control tool in multimodal interaction

Abstract

The problem of multimodal interaction is discussed. The use of blinking and winking, interpreted as Beye gestures,^is considered. The main aim of this study is to propose a simple method that allows the recognition of the state of the eye: open or closed; and to distinguish between blinking and winking. Wearable technology has been used in the introduced solution. Placing the camera close to the eye allows us to simplify the complicated image analysis. The proposed method works irrespective of the user's location and his/her gaze direction. Further, the use of infrared radiation limits the influence of external disturbing factors such as lighting conditions and pollution. The new solution is tested in two types of experiments: with 2 × 5000 pictures of open and closed eyes and with a group of 30 participants. The total correctness of the eye state recognition is 99.68% (99.94% for open eyes and 99.42% for closed ones). This result implies that the proposed solution can be effectively applied to real-world scenarios. Two applications are considered. In the first one, blinking recognition allows us to check whether safety glasses are used appropriately. In the second application, the control of mouse keys is replaced with eye gestures interpreted from a winking analysis. The introduced solution allows for effective and correct eye state recognition. The new method of control by eye gestures was accepted by the participants of this study.

Toward Gaze-Based Map Interactions: Determining the Dwell Time and Buffer Size for the Gaze-Based Selection of Map Features

“…Multiple solutions have been proposed for eye movement-based HCIs, with each having unique benefits and drawbacks. Blinking eyes (i.e., the opening and closing of individual or both eyes) to select objects is one possible approach [24][25][26]. However, blinking is subject mostly to unconscious control and is thus unnatural for interactions.…”

Gaze+Hold: Eyes-only Direct Manipulation with Continuous Gaze Modulated by Closure of One Eye

“…Historically, much of the work on eyes-only interfaces is framed as mouse emulation [Lankford 2000]. Blinks and winks have been explored to mimic button events in order to make techniques designed for the mouse accessible [Kowalczyk and Sawicki 2019;Missimer and Betke 2010], whereas the focus of this work is on design for eyes-only direct manipulation.…”

“…The key idea underpinning our work is to use closing and opening of one eye, to modulate gaze input performed with the other. The literature refers to the closing/opening of one eye as wink, contrasting blinks performed with both eyes [Kaufman et al 1993;Kowalczyk and Sawicki 2019;Missimer and Betke 2010;Singh and Singh 2018]. A wink is a holistic gesture whereas we build specifically on eye-closing and eye-opening events to trigger and delimit direct manipulation.…”

“…Some of the earliest eye control interfaces were based entirely on blinks and winks, highlighting the utility of eye closure as input [Shaw et al 1990]. Single-eye closure has the advantage that it rarely occurs involuntarily and is less prone to false positive input [Kowalczyk and Sawicki 2019]. Most people can use either eye to wink but some tend to close both eyes when attempting to wink with the dominant eye [Palmer 1976].…”

Big data analytics of crime prevention and control based on image processing upon cloud computing

“…The acceleration vector of this frame image is retained if the improved acceleration value is greater than the threshold; (3) according to the obtained acceleration vector A(a x , a y ) of the image, the corresponding onedimensional linear equation parameter is calculated. When ignoring special cases, the parameter expression is as follows [17] :…”