Design of an explainable machine learning challenge for video interviews

Abstract

International audienceThis paper reviews and discusses research advances on “explainable machine learning” in computer vision. We focus on a particular area of the “Looking at People” (LAP) thematic domain: first impressions and personality analysis. Our aim is to make the computational intelligence and computer vision communities aware of the importance of developing explanatory mechanisms for computer-assisted decision making applications, such as automating recruitment. Judgments based on personality traits are being made routinely by human resource departments to evaluate the candidates' capacity of social insertion and their potential of career growth. However, inferring personality traits and, in general, the process by which we humans form a first impression of people, is highly subjective and may be biased. Previous studies have demonstrated that learning machines can learn to mimic human decisions. In this paper, we go one step further and formulate the problem of explaining the decisions of the models as a means of identifying what visual aspects are important, understanding how they relate to decisions suggested, and possibly gaining insight into undesirable negative biases. We design a new challenge on explainability of learning machines for first impressions analysis. We describe the setting, scenario, evaluation metrics and preliminary outcomes of the competition. To the best of our knowledge this is the first effort in terms of challenges for explainability in computer vision. In addition our challenge design comprises several other quantitative and qualitative elements of novelty, including a “coopetition” setting, which combines competition and collaboration

First Impressions: A Survey on Vision-Based Apparent Personality Trait Analysis

“…Because of this, explainability and interpretability should deserve special attention. In fact, the interest on these topics are evidenced by the organization of dedicated events, such as thematic workshops [67], [68] and challenges [26], [38]. However, this kind of research is just on its infancy.…”

“…3), facilitates the comparison of different approaches. For instance, if we consider the databases [9] and [38] (shown in Table 1), they are composed by the same audio-visual data (see Table 4 for more details) and we can directly compare works evaluated on them. However, there is still a large venue to be explored, as some key-points affecting personality perception can be considered at their early stages of research (e.g., the influence of face attributes and context, spatio-temporal modeling, explainability, etc).…”

“…More recently, Escalante et al [26] analyzed the First Impression dataset [9] and top winner approaches used in the ChaLearn LAP Job Candidate Screening Challenge [38]. Part of the study focused on the existence of latent bias towards gender and ethnicity.…”

“…Results from the latter challenge motivated a new competition in a closely related topic involving personality traits, the so called Job Candidate Screening Coopetition [38]. In this challenge, an extended version of the ChaLearn First Impression dataset [9] was considered, where the extension consisted of the new variable to be predicted and manual transcriptions of audio in the videos.…”

Recent trends in deep learning based personality detection

“…The reliability of personality tests comes into question as it sometimes happens that when a person takes the test it gives a certain personality, however when the same person takes the same test again, it gives a different type of personality. [28,27] study the interpretability and explainability of deep learning models in the context of computer vision for looking at people tasks. The MBTI personality measure is the most popular personality measure used across the world right now.…”