Addressing UX Practitioners’ Challenges in Designing ML Applications: an Interactive Machine Learning Approach

Abstract

UX practitioners face novel challenges when designing user interfaces for machine learning (ML)-enabled applications. Interactive ML paradigms, like AutoML and interactive machine teaching, lower the barrier for non-expert end users to create, understand, and use ML models, but their application to UX practice is largely unstudied. We conducted a task-based design study with 27 UX practitioners where we asked them to propose a proof-of-concept design for a new ML-enabled application. During the task, our participants were given opportunities to create, test, and modify ML models as part of their workflows. Through a qualitative analysis of our post-task interview, we found that direct, interactive experimentation with ML allowed UX practitioners to tie ML capabilities and underlying data to user goals, compose affordances to enhance end-user interactions with ML, and identify ML-related ethical risks and challenges. We discuss our findings in the context of previously established human-AI guidelines. We also identify some limitations of interactive ML in UX processes and propose research-informed machine teaching as a supplement to future design tools alongside interactive ML. CCS CONCEPTS• Human-centered computing → Empirical studies in interaction design.

Establishing Heuristics for Improving the Usability of GUI Machine Learning Tools for Novice Users

“…Context Relevance was also proposed in the context of providing Guidelines to Human-AI Interaction and maps to Show contextually relevant information in the study [3], with the difference that in Human-AI Interaction, the context is the user's inferred goals and attention during the interaction, while in our GUI ML Tools Heuristics, the context is the current dataset and problem. This becomes increasingly important when building Fair ML models as emphasized by practitioners [22]. To summarize, after conducting a usability evaluation on the prototype of the GUI ML tool that followed the newly proposed set of heuristics, it was found that fewer usability issues were present in the prototype.…”

“…The scaffolding approach followed in iML tools allowed the users to focus their mental efforts on integrating their domain knowledge into the models. For example, Teachable Machine [12] was recently evaluated in the context of aiding the development of prototypes by UX practitioners (UXP) to propose a proof-ofconcept design for a new ML-enabled application [21]. Participants found it valuable to interactively explore different combinations of model classes and visualize the data.…”

Mapping the Design Space of Teachable Social Media Feed Experiences

“…Interactive machine teaching (IMT) is an interaction framework by which subject matter experts-who are often not experts in machine learning-draw upon their personal expertise to train machine learning (ML) models that can operate effectively within their domain [32,48,79,90,111,119]. While conventional ML is primarily concerned with developing algorithms that automatically learn conceptual representations from training data, IMT argues that learnable representations should directly come from human knowledge [79,111].…”