TensorFlow-Based Automatic Personality Recognition Used in Asynchronous Video Interviews

Abstract

With the development of artificial intelligence (AI), the automatic analysis of video interviews to recognize individual personality traits has become an active area of research and has applications in personality computing, human-computer interaction, and psychological assessment. Advances in computer vision and pattern recognition based on deep learning (DL) techniques have led to the establishment of convolutional neural network models that can successfully recognize human nonverbal cues and attribute their personality traits with the use of a camera. In this paper, an end-to-end AI interviewing system was developed using asynchronous video interview (AVI) processing and a TensorFlow AI engine to perform automatic personality recognition (APR) based on the features extracted from the AVIs and the true personality scores from the facial expressions and self-reported questionnaires of 120 real job applicants. The experimental results show that our AI-based interview agent can successfully recognize the ''big five'' traits of an interviewee at an accuracy between 90.9% and 97.4%. Our experiment also indicates that although the machine learning was conducted without large-scale data, the semisupervised DL approach performed surprisingly well with regard to APR despite the lack of labor-intensive manual annotation and labeling. The AI-based interview agent can supplement or replace existing self-reported personality assessment methods that job applicants may distort to achieve socially desirable effects.INDEX TERMS Big five, convolutional neural network (CNN), personality computing, TensorFlow.

Intelligent video interview agent used to predict communication skill and perceived personality traits

“…Although unsupervised DL can be adopted to automatically learn the correct patterns without requiring predefined labels, this approach requires huge quantities of data to learn the patterns [34]. Semisupervised DL can reduce the required labeling effort while maintaining high accuracy [17]. Since CNN can be used to effectively classify patterns from AVI image records [13] and the TensorFlow engine can be used to increase prediction accuracy [35], a CNN with a TensorFlow engine would be the ideal learning model to predict an interviewee's attributes based on his/her facial expressions [17].…”

“…Semisupervised DL can reduce the required labeling effort while maintaining high accuracy [17]. Since CNN can be used to effectively classify patterns from AVI image records [13] and the TensorFlow engine can be used to increase prediction accuracy [35], a CNN with a TensorFlow engine would be the ideal learning model to predict an interviewee's attributes based on his/her facial expressions [17]. In line with previous works, our study aims to develop an intelligent video interview agent based on AVI and semisupervised DL using a CNN with TensorFlow to extract the facial expression features, learn the patterns between the interviewees' facial expression and their communication skills and personality traits, and build a model to automatically predict an interviewees' personality based on his/her AVI records without assessing personality traits with any assessment tool.…”

“…Advanced biometrics recognition [38] and facial detection [41] techniques have been developed for fast extracting multiple patterns from near-infrared images accurately with limited computing resource [40]. With the emergence of artificial intelligence (AI), many computer scientists apply AI-based decision agent enabled with biometrics and facial recognition techniques to develop automatic interview platform based on AVI (called AVI-AI) [17]. The AVI-AI technologies have attracted considerable attention in both fields of computer sciences and human resources, particularly for automatically assessing communication skills [12] and personality traits [13].…”

“…Three major approaches to DL exist: supervised learning, unsupervised learning, and semisupervised learning [14]. Research has shown that semisupervised learning can be achieved with relatively small quantities of unlabeled data plus some labeled data for pattern recognition [17]. Convolutional neural networks (CNNs) have been proven to be high-performing models that can automatically classify patterns in image records [13], and CNNs are the most commonly used classifier that can be trained to accurately detect and recognize facial impressions without manual feature extraction, according to Sun et al [18].…”

“…TensorFlow is a popular open source DL framework that can be transplanted onto different heterogeneous systems across a variety of devices and platforms, including mobile and desktop [19]. Accordingly, the Ten-sorFlow-based CNN framework is expected to achieve a good face recognition effect in the context of video interviewing [17]. This study tested the validity and accuracy of assessing interpersonal communication skills and perceived big five personality traits [20] using AVI-AI.…”