Double Binds and Double Blinds: Evaluation Tactics in Critically Oriented HCI

Abstract

Critically oriented researchers within Human-Computer Interaction (HCI) have fruitfully intersected design and critical analysis to engage users and designers in reflection on underlying values, assumptions and dominant practices in technology. To successfully integrate this work within the HCI community, critically oriented researchers have tactically engaged with dominant practices within HCI in the design and evaluation of their work. This paper draws attention to the ways that tactical engagement with aspects of HCI evaluation methodology shapes and bears consequences for critically oriented research. We reflect on three of our own experiences evaluating critically oriented designs and trace challenges that we faced to the ways that sensibilities about generalizable knowledge are manifested in HCI evaluation methodology. Drawing from our own experiences, as well as other influential critically oriented design projects in HCI, we articulate some of the trade-offs involved in consciously adopting or not adopting certain normative aspects of HCI evaluation. We argue that some forms of this engagement can hamstring researchers from pursuing their intended research goals and have consequences beyond specific research projects to affect the normative discourse in the field as a whole.

Infrastructural Speculations: Tactics for Designing and Interrogating Lifeworlds

“…Critically oriented speculative practices use the process of design to surface values, critique social issues, and present alternative visions of the future by creating conceptual proposals and artifacts [48]. These include practices such as critical design [2,26,73], speculative design [1,27,35,104], adversarial design [22], and design fiction [5,7,52,56].…”

Bringing Design to the Privacy Table

“…This work is often discussed under the broad rubric of critically oriented HCI. Rather than create design solutions that are deployable at scale, critically oriented HCI creates conceptual designs and design artifacts that subvert expectations, provoke, or exaggerate existing trends in order to surface, critique, and discuss values issues, and utilizes different evaluation criteria than performance, efficiency, or usability [28,65,93]. From our corpus, this approach has been used to probe privacy implications of systems by conceptually designing: a fictional drone regulatory system [74], a range of fictional human biosensing products deployed in a variety of contexts [122], and conceptual search engine technologies that embed alternate sets of values [68].…”

Eliciting Values Reflections by Engaging Privacy Futures Using Design Workbooks

“…We imagined that several audiences might generatively interact with the design proposals, including engineering students, regulators, developers, or other technology professionals. Given calls to increase the diversity of design research artifacts [50] and documented complexities in negotiations between interviewers and participants when presenting speculative designs [38] we wanted to create multiple forms of the workbook, as people might interact with them differently.…”