Experience evaluations for human–computer co-creative processes – planning and conducting an evaluation in practice

Abstract

In human-computer co-creativity, humans and creative computational algorithms create together. Too often, only the creative algorithms and their outcomes are evaluated when studying these co-creative processes, leaving the human participants to little attention. This paper presents a case study emphasising the human experiences when evaluating the use of a co-creative poetry writing system called the Poetry Machine. The co-creative process was evaluated using seven metrics: Fun, Enjoyment, Expressiveness, Outcome satisfaction, Collaboration, Ease of writing, and Ownership. The metrics were studied in a comparative setting using three co-creation processes: a human-computer, a human-human, and a human-human-computer co-creation process. Twelve pupils of age 10-11 attended the studies in six pairs trying out all the alternative writing processes. The study methods included observation in paired-user testing, questionnaires, and interview. The observations were complemented with analyses of the video recordings of the evaluation sessions. According to statistical analyses, Collaboration was the strongest in human-human-computer co-creation, and weakest in human-computer co-creation. Ownership was just the opposite: weakest in human-human-computer co-creation, and strongest in human-computer co-creation. Other metrics did not produce statistically significant results. In addition to the results, this paper presents the lessons learned in the evaluations with children using the selected methods. ARTICLE HISTORY

Evaluating Human-Computer Co-creative Processes in Music: A Case Study on the CHAMELEON Melodic Harmonizer

“…In both experimental phases, participants filled in a 7-point Likert scale post-task questionnaire for user-experience evaluation whose questions represented the following metrics as defined by Kantosalo and Riihiaho ( 2019 ): Enjoyment, Expressivity, Outcome satisfaction, Ease of use, Collaboration, Ownership, Exploration, Immersion, and Productivity. Table 1 presents the questions corresponding to each metric both for the simple harmonization and for the computationally assisted task.…”

“…The interest in the impact of technology on human creativity is more recent (Lubart, 2005 ; Shneiderman et al, 2006 ; Shneiderman, 2007 ) in comparison to research on the definition and evaluation of creativity itself (see, Cherry and Latulipe, 2014 ). However, creativity support has already been studied in the context of various creative human activities such as poetry writing (Kantosalo and Riihiaho, 2019 ), creative design (Albert and Runco, 1999 ; Bonnardel and Zenasni, 2010 ), 3D modeling (Chaudhuri and Koltun, 2010 ), or general problem solving (Massetti, 1996 ).…”

Creative geographies in the age of AI: Co‐creative spatiality and the emerging techno‐material relations between artists and artificial intelligence

“…One approach in computational creativity is so‐called human–computer co‐creativity (e.g., Davis, 2021; Feldman, 2017), with an aim to ‘facilitate human creativity via computationally creative means and vice versa’ (Kantosalo, 2019, p. 1). Whereas earlier computational agents were mostly seen as tools, in recent studies, creative AI has been considered more of a medium or partner to humans (Davis, 2021; Elgammal & Mazzone, 2020; Kantosalo & Toivonen, 2016). This is also the case with art‐generating AI algorithms that, according to Mazzone and Elgammal (2019), are closer to a medium than just inanimate objects or tools that artists use.…”

“…However, techno‐material perspectives on creativity have been less investigated within creative geographies (except for: Nordström, 2017; Rose, 2016; Woodward et al, 2015; Zebracki & Luger, 2019). We claim that AI expands the understandings of creativity due to its partly autonomous capacity to create novel outcomes together with humans (Davis, 2021; Feldman, 2017; Kantosalo & Toivonen, 2016). We ask what role AI plays in such creativity, what its spatial manifestations are, and what directions creative geographies can take in the age of AI.…”

AfriKI: Machine-in-the-Loop Afrikaans Poetry Generation

“…Machine-in-the-Loop: Human-machine collaboration for the enhancement of creative writing has been examined under automated assistance Gordon, 2015, 2018), co-authorship (Tucker, 2019), co-creativity (Manjavacas et al, 2017;Kantosalo and Riihiaho, 2019;Calderwood et al, 2020), interactive storytelling (Swanson and Gordon, 2012;Brahman et al, 2020) and machinein-the-loop (Clark et al, 2018;Akoury et al, 2020).…”