Do They All Look the Same? Deciphering Chinese, Japanese and Koreans by Fine-Grained Deep Learning

Abstract

We study to what extend Chinese, Japanese and Korean faces can be classified and which facial attributes offer the most important cues. First, we propose a novel way of obtaining large numbers of facial images with nationality labels. Then we train state-of-the-art neural networks with these labeled images. We are able to achieve an accuracy of 75.03% in the classification task, with chances being 33.33% and human accuracy 38.89% . Further, we train multiple facial attribute classifiers to identify the most distinctive features for each group. We find that Chinese, Japanese and Koreans do exhibit substantial differences in certain attributes, such as bangs, smiling, and bushy eyebrows. Along the way, we uncover several gender-related cross-country patterns as well. Our work, which complements existing APIs such as Microsoft Cognitive Services and Face++, could find potential applications in tourism, e-commerce, social media marketing, criminal justice and even counter-terrorism.

How Polarized Have We Become? A Multimodal Classification of Trump Followers and Clinton Followers

“…Deep convolutional neural networks (CNN), on the other hand, have proven extremely powerful in image classification, usually reaching or surpassing human performance [29,11,36]. There exist quite a few studies that analyze selfies and other images posted on the social network [4,42].…”

“…With the recent success of deep convolutional neural networks (CNN), profile pictures and selfies have emerged as an integral data source for understanding individuals' demographics, sentiments and habits [4,42,36]. In particular, several studies of the 2016 presidential election have demonstrated the efficacy of using images for modeling political events.…”

AI and recruiting software: Ethical and legal implications

“…This idea leads us to mention the concept "soft biometrics", the analysis of body geometry, marks and tattoos that, unlike primary biometric traits, can be obtained at a distance without cooperation [7]. Several researchers [8][9][10][11] studied up to what point it is possible to infer and model race in face recognition. "How implicit, non-declarative racial category can be conceptually modelled and quantitatively inferred from the face?"…”

Are you from North or South India? A hard face-classification task reveals systematic representational differences between humans and machines

“…Understanding the underlying features can help endow a variety of artificial-intelligence applications with humanlike performance. While face detection itself has been extensively studied in computer vision (Viola & Jones, 2001; Barnouti, Al-Dabbagh, & Matti, 2016; M. Wang & Deng, 2018), face categorization has largely been studied using only coarse distinctions such as ethnicity (Caucasian/Black/Asian; Brooks & Gwinn, 2010; Fu, He, & Hou, 2014) and gender (Tariq, Hu, & Huang, 2009; Fu, He & Hou, 2014; Y. Wang, Liao, Feng, Xu, & Luo, 2016). Even in humans, only coarse distinctions such as Caucasian/Black have been studied (Brooks & Gwinn, 2010; Fu, He & Hou, 2014).…”

“…Alternatively, the categories themselves can be made more fine-grained, thereby increasing task difficulty. While there has been some work on discriminating between finer grained geographical origin, such as with Chinese/Japanese/Korean (Y. Wang et al, 2016), Chinese subethnicities (Duan et al, 2010), and Myanmar (Tin & Sein, 2011), these studies have not systematically characterized human performance. In fact, it is an open question whether and how well humans can discriminate fine-grained face attributes across various world populations.…”