Volunteer Science as an Online Platform for Experiments in Organizations

Radford, Jason; Pilny, Andrew; Reichelmann, Ashley V.; Keegan, Brian; Welles, Brooke Foucault; Hoye, Jeff; Ognyanova, Katherine; Meleis, Waleed; Lazer, David

doi:10.5465/ambpp.2016.17008abstract

Cited by 6 publications

(9 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our experimental data analysis and modeling software pipelines are complementary to current efforts to build configurable software platforms to perform social science experiments. See [ 35 – 38 ]. Usually, these systems only focus on the design and running of online lab experiments.…”

Section: Solution Overview and Contributionsmentioning

confidence: 99%

“…While some teams, create web-based programs especially designed for their research [ 17 , 61 , 62 ], others use web-based experimental platforms that provide this service [ 60 , 63 ]. In [ 60 ] the online platform Volunteer Science [ 35 ] was used to implement a web-based public goods experiment, and to recruit participants around the world. In [ 63 ], a repeated public goods experiment was implemented in the free web-based platform for interactive online experiments, LIONESS [ 36 ], and participants were recruited via Amazon Mechanical Turk (MTurk).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Data analysis and modeling pipelines for controlled networked social science experiments

Cedeno-Mieles

Ren

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

There is large interest in networked social science experiments for understanding human behavior at-scale. Significant effort is required to perform data analytics on experimental outputs and for computational modeling of custom experiments. Moreover, experiments and modeling are often performed in a cycle, enabling iterative experimental refinement and data modeling to uncover interesting insights and to generate/refute hypotheses about social behaviors. The current practice for social analysts is to develop tailor-made computer programs and analytical scripts for experiments and modeling. This often leads to inefficiencies and duplication of effort. In this work, we propose a pipeline framework to take a significant step towards overcoming these challenges. Our contribution is to describe the design and implementation of a software system to automate many of the steps involved in analyzing social science experimental data, building models to capture the behavior of human subjects, and providing data to test hypotheses. The proposed pipeline framework consists of formal models, formal algorithms, and theoretical models as the basis for the design and implementation. We propose a formal data model, such that if an experiment can be described in terms of this model, then our pipeline software can be used to analyze data efficiently. The merits of the proposed pipeline framework is elaborated by several case studies of networked social science experiments.

show abstract

Section: Solution Overview and Contributionsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Data analysis and modeling pipelines for controlled networked social science experiments

Cedeno-Mieles

Ren

et al. 2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Careful comparisons whether they are cross-cultural, cross-national, or cross-level analysis are excellent ways to understand the limitations of original theoretical boundaries so that we can expand and move beyond them. We live in exciting times of increasing access to big data (Cheung and Jak 2016) and new opportunities for innovative field experiments (Parigi, Santana, and Cook 2017;Radford et al 2016) to test theories in a variety of new empirical settings. How can traditional group processes research conducted in in-person labs grapple with acknowledging their own unique structural and cultural characteristics?…”

Section: Aims Of This Special Issuementioning

confidence: 99%

Comparative Analysis in Group Processes: Expanding the Reach of Sociological Social Psychology

Chin

2017

International Journal of Sociology

View full text Add to dashboard Cite

Over the past 40 years, the social psychological field of group processes has achieved rapid theoretical development. However, this subfield's amazing growth has been recognized mainly by its own practitioners. The focus of this special issue is to spark new conversations. First, it aims to ignite an interest amongst regular readers of the journal to consider how theoretical research programs in group processes may be relevant to their own work. Second, it also aims to encourage group processes researchers to think more carefully about how different kinds of comparative analysis can be incorporated in their own research. Considering new forms of comparative analysis (cross-cultural, cross-national, cross-level of analysis, etc.) may help group processes scholars become more aware of the theoretical and cultural assumptions that undergird the scope conditions and the manner in which their traditional experimental protocols have been designed. While the four articles in this issue do not encompass the full range of theoretical research programs in group processes, they all grapple with the ups and downs of using comparative analysis to question the fundamental assumptions undergirding the theories they work with. More than resolving any questions, this special issue aims to inspire new lines of inquiry. Keywords Comparative analysis; experiments; group processes; social psychology; theory This special issue of the International Journal of Sociology (IJS) examines group processes, a sociological subfield in social psychology, through a comparative lens. Theoretical research programs (Berger and Zelditch 1993) in group processes are uniquely positioned in their ability to span the divide between societal and individual levels of analysis by focusing on how interactions shape social actors' thoughts, behaviors, and interpersonal relationships (Lawler,

show abstract

“…Online crowdsourcing has been proposed as a solution to these challenges; to date, development efforts have centred around two distinct paradigms: digital versions of traditional tasks and game-based assessment. Projects such as LabintheWild 3 , Volunteer Science 4 , and TestMyBrain 5 offer a broad suite of digitized tasks from cognitive and social science; researchers create and post their tasks online, to be completed by volunteers from the general public. These scientific platforms have proven immensely successful for crowdsourcing data from standardized and quickly customizable tasks as an alternative to both laboratory studies and generic crowdsourcing platforms such as Amazon Mechanical Turk (MTurk).…”

Section: Introductionmentioning

confidence: 99%

“…Thus, in contrast to most social science experiments, where participants' main benefit is monetary compensation, Skill Lab players' efforts are rewarded by personal feedback and an enjoyable experience 12 . Finally, as a first among big data projects in cognitive science [3][4][5]10,11,[14][15][16][19][20][21][22][23] , an anonymised version of the dataset will be openly accessible.…”

Section: Introductionmentioning

confidence: 99%

Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment

Pedersen¹,

Díaz²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Psychology and the social sciences are undergoing a revolution: It has become increasingly clear that traditional lab-based experiments fail to capture the full range of differences in cognitive abilities and behaviours across the general population. Some progress has been made toward devising measures that can be applied at scale across individuals and populations. What has been missing is a broad battery of validated tasks that can be easily deployed, used across different age ranges and social backgrounds, and employed in practical, clinical, and research contexts. Here, we present Skill Lab, a game-based approach allowing the efficient assessment of a suite of cognitive abilities. Skill Lab has been validated outside the lab in a crowdsourced population-size sample recruited in collaboration with the Danish Broadcast Company (Danmarks Radio, DR). Our game-based measures are five times faster to complete than the equivalent traditional measures and replicate previous findings on the decline of cognitive abilities with age in a large population sample. Furthermore, by combining the game data with an in-game survey, we demonstrate that this unique dataset has implication for key questions in social science, challenging the Jack-of-all-Trades theory of entrepreneurship and provide evidence for risk preference being independent of executive functioning.

show abstract

Volunteer Science as an Online Platform for Experiments in Organizations

Cited by 6 publications

References 0 publications

Data analysis and modeling pipelines for controlled networked social science experiments

Data analysis and modeling pipelines for controlled networked social science experiments

Comparative Analysis in Group Processes: Expanding the Reach of Sociological Social Psychology

Measuring Cognitive Abilities in the Wild: Validating a Population-Scale Game-Based Cognitive Assessment

Contact Info

Product

Resources

About