User-Centred Design Actions for Lightweight Evaluation of an Interactive Machine Learning Toolkit

Abstract

Machine learning offers great potential to developers and end users in the creative industries. For example, it can support new sensor-based interactions, procedural content generation and enduser product customisation. However, designing machine learning toolkits for adoption by creative developers is still a nascent effort. This work focuses on the application of user-centred design with creative end-user developers for informing the design of an interactive machine learning toolkit. We introduce a framework for user-centred design actions that we developed within the context of an European Union innovation project, RAPID-MIX. We illustrate the application of the framework with two actions for lightweight formative evaluation of our toolkit-the JUCE Machine Learning Hackathon and the RAPID-MIX API workshop at eNTERFACE'17. We describe how we used these actions to uncover conceptual and technical limitations. We also discuss how these actions provided us with a better understanding of users, helped us to refine the scope of the design space, and informed improvements to the toolkit. We conclude with a reflection about the knowledge we obtained from applying user-centred design to creative technology, in the context of an innovation project in the creative industries.

Machine Learning Education for Artists, Musicians, and Other Creative Practitioners

“…Alternative toolchains could support different cohorts in achieving the same learning objectives, while presenting different tradeoffs. For instance, ml.lib [10], RAPID-MIX API [4], and scikit-learn [58] can be used by students who are proficient programmers to employ many of the same learning algorithms, without the need to run multiple programs communicating via OSC. Yet to support the same type of lab activities and creative projects, students or instructors would need to write significant additional code to implement functionalities such as control over iterative IML data collection, training and evaluation; interfacing with sensors and other data sources; and connecting to other creative software (e.g., game engines or music environments).…”

“…Two projects used RAPID-MIX [4] (a C++ library); all others used Wekinator. Students used a variety of algorithms, and several used multiple algorithms simultaneously.…”

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

“…Apart from applications that use AI in the back-end where the users are not exposed to the underlying technology or AI model, some tools allow users to interface with the AI algorithms [ 186 , 187 , 188 , 189 , 190 , 191 ]. These tools are often referred to as non-AI-expert tools or non-expert tools in general.…”

Designing and Evaluating the Usability of a Machine Learning API for Rapid Prototyping Music Technology

“…Design iterations were informed by lightweight formative evaluation actions (Bernardo et al, 2018 ) using techniques such as direct observation, interviews and group discussions in workshops and hackathons, and remote Q&A sessions between API designers and users. This work contributed to a better understanding of the needs, goals and values of the target users of the RAPID-MIX API, which spanned a breadth of software development skills, experience, motivation, and technical approach expected from creative and music technology developers.…”

“…C++ is intended for low-level audio and media developers, native mobile apps, and embedded processors. It has been tested in openFrameworks and JUCE, as well as on Raspberry Pi and Bela embedded hardware (Bernardo et al, 2018 ).…”