The Impact of Student Opt-Out on Educational Predictive Models

Li, Warren; Brooks, Christopher; Schaub, Florian

doi:10.1145/3303772.3303809

Cited by 6 publications

(13 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, for the remaining analysis, we combine the opt-in and optout conditions and look only at the aggregate data, given that the differences are negligible. This confirms the result in Li et al (2019), which states that wording has no effect on participation rate, but contrasts with their findings regarding consent rate where a difference was found between conditions. Table 1 Response rates per condition.…”

Section: Response To Email Promptsupporting

confidence: 80%

“…The analysis methods are also shown along with the corresponding data that is needed for each approach in order to address our research questions. Li et al (2019) found that the use of either "opt-out" or "opt-in" wording leads to different response rates from participants. Thus, we prepared two variants of the email prompt, which were identical except for being framed as an "opt-in" or an "opt-out" decision.…”

Section: Measuring Privacy Perceptions Personal Traits and Decision To Consentmentioning

confidence: 92%

“…However, such agency comes at the risk of decreased data and impacts predictive model performance. Li et al (2019) showed in simulations the differential effect of student's opting out on the predictive power of learning analytics models for different student subpopulations; accuracy precipitously decreased for high simulated opt-out rates. A major limitation was the fact that students opting out of data sharing was simulated based on dropping a random sample, even though this is not likely the case in reality as there may be disparities in consent when considering demographics.…”

Section: Privacy Concerns In Learning Analyticsmentioning

confidence: 97%

“…This is of particular concern as an increasing number of technologies designed to help students rely on predictive models trained with educational records. These models have been shown to have bias, thereby performing poorly for populations which are not well represented (Li et al, 2019;Ocumpaugh et al, 2014). Coupled with the fact that minorities are less likely to consent in numerous contexts, it becomes crucial to understand these interactions in education to avoid developing models that inadvertently widen disparities.…”

Section: Disparities Between Demographic Groupsmentioning

confidence: 99%

“…Moreover, biases have been shown to exist in predictive models, partly due to non-representative samples acquired during data collection (Ocumpaugh et al, 2014). As the availability of data is restricted, machine-learned models may have a reduction in accuracy which can lead to less effective interventions for some (or all) students (Li et al, 2019). This is particularly concerning since demographic gaps already exist in educational achievement (Bainbridge and Lasley, 2002), which is es-pecially true for underrepresented minorities (Bensimon, 2005), those with a lower socioeconomic status (Duncan and Magnuson, 2005), and between genders in certain contexts such as STEM programs (Matz et al, 2017).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Disparities in Students’ Propensity to Consent to Learning Analytics

Li¹,

Sun²,

Schaub³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Use of university students’ educational data for learning analytics has spurred a debate about whether and how to provide students with agency regarding data collection and use. A concern is that students opting out of learning analytics may skew predictive models, in particular if certain student populations disproportionately opt out and biases are unintentionally introduced into predictive models. We investigated university students’ propensity to consent to learning analytics through an email prompt, and collected respondents’ perceived beneﬁts and privacy concerns regarding learning analytics in a subsequent online survey. In particular, we studied whether and why students’ consent propensity diﬀers among student subpopulations by sending our email prompt to a sample of 4,000 students at our institution stratiﬁed by ethnicity and gender. 272 students interacted with the email, of which 119 completed the survey. We identiﬁed that institutional trust, concerns with the amount of data collection versus perceived beneﬁts, and comfort with instructors’ data access were key determinants in students’ decision to participate in learning analytics. We ﬁnd that students identifying ethnically as Black were signiﬁcantly less likely to respond and self-reported lower levels of institutional trust. Female students reported concerns with data collection but were also more comfortable with use of their data by instructors . Students’ comments corroborate these ﬁndings and we discuss the implications of these concerns on educational data collection.

show abstract

Section: Response To Email Promptsupporting

confidence: 80%

Section: Measuring Privacy Perceptions Personal Traits and Decision To Consentmentioning

confidence: 92%

Section: Privacy Concerns In Learning Analyticsmentioning

confidence: 97%

Section: Disparities Between Demographic Groupsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Disparities in Students’ Propensity to Consent to Learning Analytics

Li¹,

Sun²,

Schaub³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Disparities in Students’ Propensity to Consent to Learning Analytics

Sun

Schaub

et al. 2021

Int J Artif Intell Educ

Self Cite

View full text Add to dashboard Cite

Use of university students’ educational data for learning analytics has spurred a debate about whether and how to provide students with agency regarding data collection and use. A concern is that students opting out of learning analytics may skew predictive models, in particular if certain student populations disproportionately opt out and biases are unintentionally introduced into predictive models. We investigated university students’ propensity to consent to learning analytics through an email prompt, and collected respondents’ perceived benefits and privacy concerns regarding learning analytics in a subsequent online survey. In particular, we studied whether and why students’ consent propensity differs among student subpopulations bysending our email prompt to a sample of 4,000 students at our institution stratified by ethnicity and gender. 272 students interacted with the email, of which 119 also completed the survey. We identified that institutional trust, concerns with the amount of data collection versus perceived benefits, and comfort with instructors’ data use for learning engagement were key determinants in students’ decision to participate in learning analytics. We find that students identifying ethnically as Black were significantly less likely to respond and self-reported lower levels of institutional trust. Female students reported concerns with data collection but were also more comfortable with use of their data by instructors for learning engagement purposes. Students’ comments corroborate these findings and suggest that agency alone is insufficient; institutional leaders and instructors also play a large role in alleviating the issue of bias.

show abstract