Eyelid and Pupil Landmark Detection and Blink Estimation Based on Deformable Shape Models for Near-Field Infrared Video

Abstract

The eyelid contour, pupil contour, and blink event are important features of eye activity, and their estimation is a crucial research area for emerging wearable camera-based eyewear in a wide range of applications e.g., mental state estimation. Current approaches often estimate a single eye activity, such as blink or pupil center, from far-field and non-infrared (IR) eye images, and often depend on the knowledge of other eye components. This paper presents a unified approach to simultaneously estimate the landmarks for the eyelids, the iris and the pupil, and detect blink from near-field IR eye images based on a statistically learned deformable shape model and local appearance. Unlike the facial landmark estimation problem, by comparison, different shape models are applied to all eye states-closed eye, open eye with iris visible, and open eye with iris and pupil visible-to deal with the self-occluding interactions among the eye components. The most likely eye state is determined based on the learned local appearance. Evaluation on three different realistic datasets demonstrates that the proposed three-state deformable shape model achieves state-of-the-art performance for the open eye with iris and pupil state, where the normalized error was lower than 0.04. Blink detection can be as high as 90% in recall performance, without direct use of pupil detection. Cross-corpus evaluation results show that the proposed method improves on the state-of-the-art eyelid detection algorithm. This unified approach greatly facilitates eye activity analysis for research and practice when different types of eye activity are required rather than employ different techniques for each type. Our work is the first study proposing a unified approach for eye activity estimation from near-field IR eye images and achieved the state-of-the-art eyelid estimation and blink detection performance.

Clinical utility of maximum blink interval measured by smartphone application DryEyeRhythm to support dry eye disease diagnosis

“…Additionally, as the blink recognition function of DryEyeRhythm can be hindered for users wearing a mask, the app-based MBI was measured with masks removed. Another factor that may affect the blink recognition function of DryEyeRhythm is the narrow palpebral fissure width of the participants 33 , due to which, 10 participants were unable to undergo app-based MBI measurements in this study. Future studies and updates of the app should focus on enhancing the recognition algorithm, aiming to eliminate the necessity for users to remove masks and adjusting for narrow palpebral fissure width.…”

Pupillometric and blink measures of diverse task loads: Implications for working memory models

“…To obtain pupil size, we firstly used a machine learning algorithm (Chen & Epps, 2019b) to learn and predict eye state (blink or not) and eight landmarks on the pupil boundary. We then visually checked and corrected each frame for the correct eye state and landmark positions.…”