Reliability of Online Surveys in Investigating Perceptions and Impressions of Faces

Abstract

Online experimental methods are used in psychological studies investigating the perceptions and impressions of facial photographs, even without substantial evidence supporting their reliability and validity. Although, the quality of visual stimuli is more difficult to control remotely, the methods might allow us to obtain a large amount of data. Then the statistical analysis of a larger volume of data may reduce errors and suggest significant difference in the stimuli. Therefore, we analyzed the reliability and validity of online surveys in investigating the perceptions (shine, red, and dark) and impressions (attractiveness, trustworthy, and so on) of facial photographs created from averaged faces with skin tones modified using computer graphics (CG). In this study, we conducted online (Online1) and laboratory experiments with well-controlled conditions (Control). For each experiment, 50 participants (men and women in Japan, age: 20–59years) completed the same questionnaire regarding their impressions of the same 28 CG facial photographs. The results showed significant correlations between the two experiments for all 19 items in the questionnaire. SD in the Online1 compared to the Control from the stimuli and individual differences were 56–84 and 88–104% in each questionnaire items, respectively. Moreover, the rates of mismatching perceptual evaluations to the corresponding physical features demonstrated in the photographs were 4.9–9.7% on average in an additional online survey of another 2,000 participants (Online2). These results suggest that online surveys can be applied to experiments to investigate impressions from CG facial photographs instead of general laboratory experiment by obtaining an appropriate number of participants to offset larger statistical errors that may result from the increased noise in the data from conducting the experiment online.

Impact of face masks and sunglasses on attractiveness, trustworthiness, and familiarity, and limited time effect: a Japanese sample

“…In contrast, the study by Kamatani et al [ 14 ] was conducted in a laboratory setting with screens of the same size. Even though some studies reported that online experiments are acceptable [ 39 – 41 ], it is possible that the inconsistencies in screen resolution underlie the failure to replicate the results of Kamatani et al [ 14 ]. Contrastingly, the mere exposure effect [ 38 ] was expected to increase perceived attractiveness in both HAF and LAF in Latter.…”

“…The first is the issue of perceived familiarity with online experiments. Studies have shown that online experiments can be used to measure perceived attractiveness and trustworthiness from faces [39][40][41]. However, even with the existence of the beauty-is-good bias, it is not yet clear whether online experiments can accurately measure perceived familiarity.…”

“…Laboratory experiments were prohibited by the author's department to prevent the spread of COVID-19 infection. However, research has reported that perceived attractiveness and trustworthiness can be measured by online experiments [ 39 ]. Studies have also reported high positive correlations in perceived attractiveness measured in face-to-face and Internet-based experiments [ 40 , 41 ].…”

Color Affects Recognition of Emoticon Expressions

“…We were aware of the downside of an online experiment, such as less control over the display and viewing conditions, and the stimulus color, and the subsequent increase in variation of individual judgments. However, we were encouraged by a recent study by Hirao et al (2021) , who compared the perception of face obtained online and under controlled lab conditions. The authors conclude that results in the tasks performed online were broadly consistent with lab-based results, provided that a participant sample is sufficiently large to overcome a statistical error.…”

Validation and optimization of the Japanese version of the Adult Executive Functioning Inventory

“…Similar demand also exists in basic psychology fields. Recently, we can collect participants' data online, which enables us to get a large sample size in spite of physical distancing, such as from COVID-19, while ensuring quality and reliable of data (Hirao et al, 2021). Thus, researchers may occasionally need to assess EFs as an additional variable.…”