Psychological research in the internet age: The quality of web-based data

Ramsey, Sarah R.; Thompson, Kristen L.; McKenzie, Melissa; Rosenbaum, Alan

doi:10.1016/j.chb.2015.12.049

Cited by 161 publications

(127 citation statements)

References 20 publications

Supporting

Mentioning

119

Contrasting

Order By: Relevance

“…Online crowdsourcing platforms are becoming a popular alternative to undergraduate participant pools, as crowdsourcing platforms allow for large samples of demographically-diverse participants to be recruited easily and cost-effectively (Buhrmester, Kwang, & Gosling, 2011;Paolacci, Chandler, & Ipeirotis, 2010). Research shows that these online participants are at least as attentive to task instructions as participants from traditional university-based pools (Hauser & Schwarz, 2016;Ramsey, Thompson, McKenzie, & Rosenbaum, 2016).…”

Section: Participantsmentioning

confidence: 99%

Behavioral complexity of British gambling advertising

Newall

2017

Addiction Research & Theory

View full text Add to dashboard Cite

Background: The scale and complexity of British gambling advertising has increased in recent years. "Live-odds" TV gambling adverts broadcast the odds on very specific, complex, gambles during sporting events (e.g., in soccer, "Wayne Rooney to score the first goal, 5-to-1," or, "Chelsea to win 2-1, 10-to-1"). These gambles were analyzed from a behavioral scientific perspective (the intersection of economics and psychology). Method: A mixed methods design combining observational and experimental data. A content analysis showed that live-odds adverts from two months of televised English Premier League matches were biased towards complex, rather than simple, gambles. Complex gambles were also associated with high bookmaker profit margins. A series of experiments then quantified the rationality of participants' forecasts across key gambles from the content analysis (Total N = 1,467 participants across 5 Experiments). Results: Soccer fans rarely formed rational probability judgments for the complex events dominating gambling advertising, but were much better at estimating simple events. Conclusions: British gambling advertising is concentrated on the complex products that mislead consumers the most.Behavioral scientific findings are relevant to the active public debate about gambling.

show abstract

Section: Participantsmentioning

confidence: 99%

Behavioral complexity of British gambling advertising

Newall

2017

Addiction Research & Theory

View full text Add to dashboard Cite

show abstract

“…These results suggest that HCI research conducted using online platforms can still produce valid results. However, this finding is not replicated in all online experiments, as noted in Ramsey, Thompson, Mckenzie, & Rosenbaum, (2016), success rates when replicating lab experiments online are not consistent.…”

Section: Experiments Onlinementioning

confidence: 54%

“…If the payment is not related to the quality of data that is produced, there is no incentive to engage with the experiment -they will be paid regardless -a phenomenon noted by Mason & Watts, (2010). Whereas in the lab, participants are incentivized to complete the experiment to the best of their ability due to researcher presence perhaps or the "scientific aura" (Ramsey et al, 2016), online there is no such incentive meaning those in Online Reimbursed conditions can choose to complete the task without giving it their full attention.…”

Section: Data Qualitymentioning

confidence: 99%

Exploring the effects of non-monetary reimbursement for participants in HCI research

Wiseman

Cox

Gould

et al. 2017

HCj

View full text Add to dashboard Cite

When running experiments within the field of Human Computer Interaction (HCI) it is common practice to ask participants to come to a specified lab location, and reimburse them monetarily for their time and travel costs. This, however, is not the only means by which to encourage participation in scientific study. Citizen science projects, which encourage the public to become involved in scientific research, have had great success in getting people to act as sensors to collect data or to volunteer their idling computer or brain power to classify large data sets across a broad range of fields including biology, cosmology and physical and environmental science. This is often done without the expectation of payment. Additionally, data collection need not be done on behalf of an external researcher; the Quantified Self (QS) movement allows people to reflect on data they have collected about themselves. This too, then, is a form of non-reimbursed data collection. Here we investigate whether citizen HCI scientists and those interested in personal data produce reliable results compared to participants in more traditional lab-based studies. Through six studies, we explore how participation rates and data quality are affected by recruiting participants without monetary reimbursement: either by providing participants with data about themselves as reward (a QS approach), or by simply requesting help with no extrinsic reward (as in citizen science projects). We show that people are indeed willing to take part in online HCI research in the absence of extrinsic monetary reward, and that the data generated by participants who take part for selfless reasons, rather than for monetary reward, can be as high quality as data gathered in the lab and in addition may be of higher quality than data generated by participants given monetary reimbursement online. This suggests that large HCI experiments could be run online in the future, without having to incur the equally large reimbursement costs alongside the possibility of running experiments in environments outside of the lab.

show abstract

“…Our sample included N = 168 individuals recruited via Amazon Mechanical Turk (mTurk)-a service for recruiting volunteers to complete online tasks (for more on the use of mTurk in psychological research, see Buhrmester, Kwang, & Gosling, 2011;Crump, McDonnell, & Gureckis, 2013;Ramsey, Thompson, McKenzie, & Rosenbaum, 2016). In exchange for their participation, each participant received a $1.00 credit to their Amazon mTurk account.…”

Section: Participants and Designmentioning

confidence: 99%

Stigma against false confessors impacts post‐exoneration financial compensation

Kukucka

Evelo

2019

Behavioral Sci & The Law

View full text Add to dashboard Cite

False confessors are stigmatized more than other exonerees. Traditional theories of stigma suggest that this difference may result from confessors being seen as more responsible for their own wrongful conviction. In the current study, we examined an important tangible consequence of stigma against false confessors—namely, that it might impede their ability to win financial restitution in post‐exoneration civil lawsuits. Mock jurors (N = 129), recruited online, read a case summary in which an exoneree is seeking damages after being wrongly convicted due to a false confession or eyewitness misidentification, which either did or did not result from police misconduct. When the exoneree falsely confessed in the absence of police misconduct, mock jurors rated him as most responsible for his own conviction and expressed the most doubt over his actual innocence. Contrary to legal criteria, they also awarded him smaller compensatory and punitive damage awards. Notably, the false confessor was seen as more responsible than the misidentified exoneree even if his interrogation was highly coercive. In turn, false confessors who were seen as more responsible received smaller damage awards. Implications for trial procedure and exoneree compensation are discussed.

show abstract

Psychological research in the internet age: The quality of web-based data

Cited by 161 publications

References 20 publications

Behavioral complexity of British gambling advertising

Behavioral complexity of British gambling advertising

Exploring the effects of non-monetary reimbursement for participants in HCI research

Stigma against false confessors impacts post‐exoneration financial compensation

Contact Info

Product

Resources

About