Crowdsourcing visual perception experiments: a case of contrast threshold

Abstract

Crowdsourcing has commonly been used for psychological research but not for studies on sensory perception. A reason is that in online experiments, one cannot ensure that the rigorous settings required for the experimental environment are replicated. The present study examined the suitability of online experiments on basic visual perception, particularly the contrast threshold. We conducted similar visual experiments in the laboratory and online, employing three experimental conditions. The first was a laboratory experiment, where a small sample of participants (n = 24; laboratory condition) completed a task with 10 iterations. The other two conditions were online experiments: participants were either presented with a task without repetition of trials (n = 285; online non-repetition condition) or one with 10 iterations (n = 166; online repetition condition). The results showed significant equivalence in the contrast thresholds between the laboratory and online repetition conditions, although a substantial amount of data needed to be excluded from the analyses in the latter condition. The contrast threshold was significantly higher in the online non-repetition condition compared with the laboratory and online repetition conditions. To make crowdsourcing more suitable for investigating the contrast threshold, ways to reduce data wastage need to be formulated.

Stage 2 Registered Report: How subtle linguistic cues prevent unethical behaviors

“…There were advantages and disadvantages to conducting the cheating experiment online; because of anonymity, participants were free from social expectations and could cheat more easily ( Grym & Veronica, 2016 ). However, we did not have a true grasp of whether participants completed the experiment in good faith (e.g., Sasaki & Yamada, 2019 ).…”

“…Therefore, even if the result of the original experiment was completely reproduced in our Experiment 1, it would not fully support the finding of the original experiment, as the reason for the possible different dishonest behavior rates between the “cheating” and “cheater” conditions in our Experiment 1 might be that the participants in the “cheating” group paid relatively less attention to the instruction; for this reason, “cheating” might have worked weakly as a moral reminder in this condition. Because the experiments are conducted online, it was difficult to ensure that the participants have actually seen and understood the instruction; in addition, it was also possible that the participants ignored the instructions of Experiment 1 due to satisficing, (e.g., Chandler et al, 2014 ; Oppenheimer et al, 2009 ; Sasaki & Yamada, 2019 ), further diminishing the effect of the unattended reminder (i.e., “cheating”). In this Experiment 2 we addressed these attention-related effects.…”

Stage 2 Registered Report: How subtle linguistic cues prevent unethical behaviors

“…Therefore, even if the result of the original experiment was completely reproduced in our Experiment 1, it would not fully support the finding of the original experiment, as the reason for the possible different dishonest behavior rates between the "cheating" and "cheater" conditions in our Experiment 1 might be that the participants in the "cheating" group paid relatively less attention to the instruction; for this reason, "cheating" might have worked weakly as a moral reminder in this condition. Because the experiments are conducted online, it was difficult to ensure that the participants have actually seen and understood the instruction; in addition, it was also possible that the participants ignored the instructions of Experiment 1 due to satisficing, (e.g., Chandler et al, 2014;Oppenheimer et al, 2009;Sasaki & Yamada, 2019), further diminishing the effect of the unattended reminder (i.e., "cheating").…”

Micropublishing During and After the COVID-19 Era

“…For example, several large psychological multi-lab collaborations, such as the COVIDiSTRESS project (https://covidistress.github.io/) or the Psychological Science Accelerator's COVID-Rapid project (https://psysciacc.org/), are considering implementing online questionnaires. Although various limitations have to be taken into consideration (for example, prolonged experiments result in a substantial number of dropouts: Sasaki & Yamada, 2019), online experiments and surveys enable researchers to collect diverse and comprehensive data in a short period of time (Grootswagers, 2020). Therefore, some social and behavioral researchers, who are currently unable to conduct laboratory experiments, have been able to continue to collect data.…”