Dynamics of Content Quality in Collaborative Knowledge Production

Ferrara, Emilio; Alipourfard, Nazanin; Burghardt, Keith; Gopal, Chiranth; Lerman, Kristina

doi:10.1609/icwsm.v11i1.14907

Cited by 9 publications

(2 citation statements)

References 15 publications

(20 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This suggests that performance on these platforms has a non-trivial dynamic component that merits deeper investigation. Indeed, a study of Stack Exchange observed short term deterioration in performance (Ferrara et al 2017). Our study suggests that such an effect may be general.…”

Section: Discussionsupporting

confidence: 53%

Using Simpson’s Paradox to Discover Interesting Patterns in Behavioral Data

Alipourfard

Fennell

Lerman³

2018

ICWSM

View full text Add to dashboard Cite

We describe a data-driven discovery method that leverages Simpson's paradox to uncover interesting patterns in behavioral data. Our method systematically disaggregates data to identify subgroups within a population whose behavior deviates significantly from the rest of the population. Given an outcome of interest and a set of covariates, the method follows three steps. First, it disaggregates data into subgroups, by conditioning on a particular covariate, so as minimize the variation of the outcome within the subgroups. Next, it models the outcome as a linear function of another covariate, both in the subgroups and in the aggregate data. Finally, it compares trendsto identify disaggregations that produce subgroups with different behaviors from the aggregate.We illustrate the method by applying it to three real-world behavioral datasets, including Q\&A site Stack Exchange and online learning platforms Khan Academy and Duolingo.

show abstract

Section: Discussionsupporting

confidence: 53%

Using Simpson’s Paradox to Discover Interesting Patterns in Behavioral Data

Alipourfard

Fennell

Lerman³

2018

ICWSM

View full text Add to dashboard Cite

show abstract

“…This observation is consistent with an analysis of 18 interviews by Rieh et al (Rieh et al 2017), where they reported that as a result of providing answers in a CQA environment, users (teens) were able to expand their knowledge bases, improve inquisitiveness, embrace challenging questions, and increase self-confidence. Moreover, a recent study on quality of content in a CQA environment by Ferrara et al (Ferrara et al 2017), revealed a long-term trend in terms of improvement of quality of answers due to learning and skill acquisition by users over time.…”

Section: Questions In Pythonmentioning

confidence: 99%

QDEE: Question Difficulty and Expertise Estimation in Community Question Answering Sites

Sun

Moosavi

Ramnath

et al. 2018

ICWSM

View full text Add to dashboard Cite

In this paper, we present a framework for Question Difficulty and Expertise Estimation (QDEE) in Community Question Answering sites (CQAs) such as Yahoo! Answers and Stack Overflow, which tackles a fundamental challenge in crowdsourcing: how to appropriately route and assign questions to users with the suitable expertise. This problem domain has been the subject of much research and includes both language-agnostic as well as language conscious solutions. We bring to bear a key language-agnostic insight: that users gain expertise and therefore tend to ask as well as answer more difficult questions over time. We use this insight within the popular competition (directed) graph model to estimate question difficulty and user expertise by identifying key hierarchical structure within said model. An important and novel contribution here is the application of ``social agony'' to this problem domain. Difficulty levels of newly posted questions (the cold-start problem) are estimated by using our QDEE framework and additional textual features. We also propose a model to route newly posted questions to appropriate users based on the difficulty level of the question and the expertise of the user. Extensive experiments on real world CQAs such as Yahoo! Answers and Stack Overflow data demonstrate the improved efficacy of our approach over contemporary state-of-the-art models.

show abstract

Computational social scientist beware: Simpson’s paradox in behavioral data

Lerman¹

2017

J Comput Soc Sc

View full text Add to dashboard Cite

Observational data about human behavior are often heterogeneous, i.e., generated by subgroups within the population under study that vary in size and behavior. Heterogeneity predisposes analysis to Simpson's paradox, whereby the trends observed in data that have been aggregated over the entire population may be substantially different from those of the underlying subgroups. I illustrate Simpson's paradox with several examples coming from studies of online behavior and show that aggregate response leads to wrong conclusions about the underlying individual behavior. I then present a simple method to test whether Simpson's paradox is affecting results of analysis. The presence of Simpson's paradox in social data suggests that important behavioral differences exist within the population, and failure to take these differences into account can distort the studies' findings.

show abstract

Dynamics of Content Quality in Collaborative Knowledge Production

Cited by 9 publications

References 15 publications

Using Simpson’s Paradox to Discover Interesting Patterns in Behavioral Data

Using Simpson’s Paradox to Discover Interesting Patterns in Behavioral Data

QDEE: Question Difficulty and Expertise Estimation in Community Question Answering Sites

Computational social scientist beware: Simpson’s paradox in behavioral data

Contact Info

Product

Resources

About