Values of Participants in Behavioral Accounting Research: A Comparison of the M-Turk Population to a Nationally Representative Sample

Abstract

The external validity of conclusions from behavioral accounting experiments is in part dependent upon the representativeness of the sample compared to the population of interest. Researchers are beginning to leverage the availability of workers via online labor markets, such as Amazon's Mechanical Turk (M-Turk), as proxies for the general population (e.g., investors, jurors, and taxpayers). Using over 200 values-based items from the World Values Survey (WVS), the purpose of the current study is to explore whether U.S. M-Turk workers' values are similar to those of the U.S. population. Results show for the majority of items collected, M-Turk participants' values are significantly different from the WVS participants (e.g., values related to trust, ethics, religious beliefs, and politics). We present select items and themes representing values shown to influence judgments in prior research and discuss how those values may affect inferences of behavioral accounting researchers. Data Availability: Data are available from the authors upon request.

Amazon’s MTurk: A Currently Underutilised Resource for Survey Researchers?

“…also state that workers tend to be younger (about 30 years old), overeducated, underemployed, less religious, and more liberal than the general population. Brink, Lee, et al (2019) further add that the MTurk population is more willing to justify unethical behaviour, more trusting in others, places lower importance on hard work, and has lower capitalist values. Goodman et al (2013) find that MTurk samples differ from traditional samples on several dimensions such as personality measures and attention span.…”

“…However, some studies have found differences between MTurk participants and traditional participants (e.g. Brink, Lee, et al, 2019;Goodman et al, 2013). MTurk has been used across different accounting research fields including, for example, financial accounting studies that examine investors' reliance on non-financial information disclosures (Dong, 2017) and the impact on investors' judgements of corporate social responsibility reports (Elliott et al, 2017); management accounting studies investigating the effect of performance reporting frequency on employee performance (Hecht et al, 2019) and motivations for people to report honestly (Murphy et al, 2019); auditing studies focusing on manager responses to internal audit and standards of care required by jurors when assessing auditor negligence (Maksymov & Nelson, 2017); and taxation studies addressing decision makers' willingness to evade taxes and the effect of consumer-directed tax credits on motivating purchasing behaviour (Stinson et al, 2018).…”

Examining attitudes on pay for low level workers: do consumers care?

“…Qualtrics was set to accept one response per IP address, to eliminate duplicate responses by individuals with multiple MTurk accounts (Buhrmester et al , 2018). Finally, while MTurk workers differ from the general population in specific ways (religious and political beliefs; Brink et al , 2018), they do respond similarly to the general population on tasks that relate to fairness (e.g. public good experiments; Arechar et al , 2017), suggesting this population is appropriate for this research question.…”

Mitigating stigma associated with recycled water

“…This paper also contributes to the methodological literature on how to recruit a reasonably representative sample of adult consumers in experimental studies (Boas et al, 2020;Brink et al, 2019;Coppock, 2019). Study II replicates the incentive-compatible design of Study I but converts the inperson field experiment into an online field experiment.…”