Bartleby: Procedural and Substantive Ethics in the Design of Research Ethics Systems

Abstract

The lack of consent or debriefing in online research has attracted widespread public distrust. How can designers create systems to earn and maintain public trust in large-scale online research? Procedural theories inform processes that enable individuals to make decisions about their participation. Substantive theories focus on the normative judgments that researchers and participants make about specific studies in context. Informed by these theories, we designed Bartleby, a system for debriefing participants and eliciting their views about studies that involved them. We evaluated this system by using it to debrief thousands of participants in a series of observational and experimental studies on Twitter and Reddit. We find that Bartleby addresses procedural concerns by creating new opportunities for study participants to exercise autonomy. We also find that participants use Bartleby to contribute to substantive, value-driven conversations about participant voice and power. We conclude with a critical reflection on the strengths and limitations of reusable software to satisfy values from both procedural and substantive ethical theories.

Null Compliance: NYC Local Law 144 and the challenges of algorithm accountability

“…Second, disclosures need to be comprehensible enough for a job seeker to make an informed choice [73]. Ongoing literature in bioethics, psychology, and design is exploring exactly what it means to make an informed decision [56,71,74,75]. At minimum, an informed decision is one in which people are able to reason about the possible consequences based on reliable information about the possible choices [71].…”

Consent: A Research and Design Lens for Human-Computer Interaction

“…people discovered online [25] and with sex robots [20], and consent to online research participation [24]. At the same time, consent has become a focal point in the tech industry [4] and amongst community advocacy groups [16].…”

“…Yet much of this literature has historically not recognized the role that computer-mediated communication does and could play in consent exchange, and how "best practices" for consent exchange should be updated to explicitly accommodate the role of technology. Computer-mediated communication adds an additional layer of complexity because computing systems act as examples of consent practices [13] and inadvertent influences over users' perceptions of consent [25], as well as direct intermediaries between parties consenting to an activity that occurs online [24]. HCI researchers have studied ways to design better interfaces and policies for individuals' consent, such as identifying dark patterns [12] or designing features for online safety, especially for marginalized populations [21].…”

“…HCI researchers have studied ways to design better interfaces and policies for individuals' consent, such as identifying dark patterns [12] or designing features for online safety, especially for marginalized populations [21]. Simultaneously however, researchers have also critiqued the limitations of focusing on individual consent [11,24]. Scholars taking on the standpoint of non-western data subjects have offered alternatives to western consent models, including informed refusal [3,23] and indigenous data sovereignty [9].…”