LabintheWild

Reinecke, Katharina; Gajos, Krzysztof Z.

doi:10.1145/2675133.2675246

Cited by 97 publications

(36 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Instead of offering monetary compensation, these platforms motivate participation by promising participants that they will receive their results immediately at the end of the study and will be able to compare their own performance to others. Several validation studies demonstrated that data collected on those platforms are not statistically different from the data collected in conventional laboratory settings [18,19,29]. Despite the fact that participants self-select to take part in studies on these platforms, the samples appear more diverse (and thus more representative of the general population) than the samples that participate in studies in conventional academic laboratories or populations recruited via MTurk [29].…”

Section: Conducting Behavioral Research With Unpaid Online Volunteersmentioning

confidence: 99%

The Role of Explanations in Casual Observational Learning about Nutrition

Burgermaster

Gajos

Davidson

et al. 2017

Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

The ubiquity of internet-based nutrition information sharing indicates an opportunity to use social computing platforms to promote nutrition literacy and healthy nutritional choices. We conducted a series of experiments with unpaid volunteers using an online Nutrition Knowledge Test. The test asked participants to examine pairs of photographed meals and identify meals higher in a specific macronutrient (e.g., carbohydrate). After each answer, participants received no feedback on the accuracy of their answers, viewed proportions of peers choosing each response, received correctness feedback from an expert dietitian with or without expert-generated explanations, or received correctness feedback with crowd-generated explanations. The results showed that neither viewing peer responses nor correctness feedback alone improved learning. However, correctness feedback with explanations (i.e., modeling) led to significant learning gains, with no significant difference between explanations generated by experts or peers. This suggests the importance of explanations in social computing-based casual learning about nutrition and the potential for scaling this approach via crowdsourcing.

show abstract

Section: Conducting Behavioral Research With Unpaid Online Volunteersmentioning

confidence: 99%

The Role of Explanations in Casual Observational Learning about Nutrition

Burgermaster

Gajos

Davidson

et al. 2017

Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The success of previous initiatives aimed at recruiting people to experiments using data as a reward suggested that this would be possible (Reinecke et al, 2015) however, the current study shows that this is also possible even without a carefully designed "fun" sounding experiment. This suggests that QS-style recruitment might be possible for a wide range of experiments, not just those that will provide users with particularly novel data about themselves.…”

Section: Participation Levelsmentioning

confidence: 71%

“…The Test My Brain site specifically states that "the study should be fun for our participants" if any researchers wish to submit their own experiments. An investigation into participant motivation shows participants take part because they find the studies fun and enjoy comparing their results to others (Reinecke, Arbor, & Gajos, 2015). It appears that enjoyment is key for participation in online experiments such as these.…”

Section: Quantified Selfmentioning

confidence: 99%

Exploring the effects of non-monetary reimbursement for participants in HCI research

Wiseman

Cox

Gould

et al. 2017

HCj

View full text Add to dashboard Cite

When running experiments within the field of Human Computer Interaction (HCI) it is common practice to ask participants to come to a specified lab location, and reimburse them monetarily for their time and travel costs. This, however, is not the only means by which to encourage participation in scientific study. Citizen science projects, which encourage the public to become involved in scientific research, have had great success in getting people to act as sensors to collect data or to volunteer their idling computer or brain power to classify large data sets across a broad range of fields including biology, cosmology and physical and environmental science. This is often done without the expectation of payment. Additionally, data collection need not be done on behalf of an external researcher; the Quantified Self (QS) movement allows people to reflect on data they have collected about themselves. This too, then, is a form of non-reimbursed data collection. Here we investigate whether citizen HCI scientists and those interested in personal data produce reliable results compared to participants in more traditional lab-based studies. Through six studies, we explore how participation rates and data quality are affected by recruiting participants without monetary reimbursement: either by providing participants with data about themselves as reward (a QS approach), or by simply requesting help with no extrinsic reward (as in citizen science projects). We show that people are indeed willing to take part in online HCI research in the absence of extrinsic monetary reward, and that the data generated by participants who take part for selfless reasons, rather than for monetary reward, can be as high quality as data gathered in the lab and in addition may be of higher quality than data generated by participants given monetary reimbursement online. This suggests that large HCI experiments could be run online in the future, without having to incur the equally large reimbursement costs alongside the possibility of running experiments in environments outside of the lab.

show abstract

“…The number of available workers with VR devices continues to grow, and will allow the scale of such experiments to expand as well. Of course, expanding the experiments beyond AMT and even beyond crowdsourcing platforms using other online experiment recruiting methods could also be feasible and scalable [40].…”

Section: Discussionmentioning

confidence: 99%

“…Online experiments are not limited, of course, to crowdsourced environments, and have long been an acceptable tool in behavioral research [24,44]. Some recent innovations, however, involve new mechanisms of recruitment, distribution and data collection, for example by volunteers (in return for feedback on performance) [40]. While such mechanisms will be increasingly available for research in VR, our focus remains on online VR experiments via crowdsourcing on AMT.…”

Section: Running Online Experimentsmentioning

confidence: 99%

Web-Based VR Experiments Powered by the Crowd

Cackett

Park

et al. 2018

Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18

View full text Add to dashboard Cite

We build on the increasing availability of Virtual Reality (VR) devices and Web technologies to conduct behavioral experiments in VR using crowdsourcing techniques. A new recruiting and validation method allows us to create a panel of eligible experiment participants recruited from Amazon Mechanical Turk. Using this panel, we ran three different crowdsourced VR experiments, each reproducing one of three VR illusions: place illusion, embodiment illusion, and plausibility illusion. Our experience and worker feedback on these experiments show that conducting Web-based VR experiments using crowdsourcing is already feasible, though some challenges-including scale-remain. Such crowdsourced VR experiments on the Web have the potential to finally support replicable VR experiments with diverse populations at a low cost.

show abstract

LabintheWild

Cited by 97 publications

References 37 publications

The Role of Explanations in Casual Observational Learning about Nutrition

The Role of Explanations in Casual Observational Learning about Nutrition

Exploring the effects of non-monetary reimbursement for participants in HCI research

Web-Based VR Experiments Powered by the Crowd

Contact Info

Product

Resources

About