PupilNet, Measuring Task Evoked Pupillary Response using Commodity RGB Tablet Cameras

Abstract

Pupillary diameter monitoring has been proven successful at objectively measuring cognitive load that might otherwise be unobservable. This paper compares three different algorithms for measuring cognitive load using commodity cameras. We compare the performance of modified starburst algorithm (from previous work) and propose two new algorithms: 2 Level Snakuscules and a convolutional neural network which we call PupilNet. In a user study with eleven participants, our comparisons show PupilNet outperforms other algorithms in measuring pupil dilation, is robust to various lighting conditions, and robust to different eye colors. We show that the difference between PupilNet and a gold standard head-mounted gaze tracker varies only from -2.6% to 2.8%. Finally, we also show that PupilNet gives similar conclusions about cognitive load during a longer duration typing task.

Can a Robot Catch You Lying? A Machine Learning System to Detect Lies During Interactions

“…A recent research done by Wangwiwattana et al ( 2018 ) on eye dilation estimation, using RGB cameras, achieved a level of precision as high as the one using standard eye tracker devices. Therefore, it could be used instead of the Tobii device.…”

Side-Channel Sensing: Exploiting Side-Channels to Extract Information for Medical Diagnostics and Monitoring

“…As an alternative to context specific signal processing pathways, machine learning following a feature extraction step can be applied where there is a dataset available and known patterns need to be extracted [61] or to perform classification [53] , [67] , [68] , [70] (Ṁ, ③). Machine learning can be a powerful tool in the toolbox of target information extraction techniques, but a thorough analysis of the myriad of machine learning techniques is beyond the scope of this article.…”

“…Multivariate solutions are particularly effective when paired with machine learning due to the wealth of data collected [24], [25]. Deep learning can further extend this concept, automating the side-channel discovery and target information extraction process [53].…”

“…Smartphone cameras offer impressive clarity and magnification for quantifying externally visible biomarkers and performing photoplethysmography (PPG). Achievable with a standard camera is the determining of heart rates [16] , [41] , breathing rates [16] , blood pressure [50] , atrial fibrillation [12] , oxygen saturation [49] , classifying the severity of jaundice in newborns via the yellow discolouration of the skin or sclera [11] , [51] , [52] , or even inferring of cognitive loads through visual analysis of biomarkers within pupillary dilation [53] .…”

“…Target information in the context of MDM is the source information that is indirectly sensed through a side-channel, and typically quantifies physical or physiological parameters within the human body (MDM’s target system), for the purpose of a diagnosis or monitoring of a medical condition. Any condition that has associated physiological manifestations (including psychological conditions with physiological biomarkers, such as cognitive load via pupillary response [53] ) has the potential to be accessible via side-channels, which can be challenging to measure directly without disturbing or invading the system.…”