Evaluation of User Experience of Self-scheduling Software for Astronauts: Defining a Satisfaction Baseline

Collaborative System Usability in Spaceflight Analog Environments through Remote Observations

“…After using the voice logs to establish that HERA is a collaborative, social environment during self-scheduling, we analyzed the descriptive statistics of post-mission usability survey responses to infer UX in a spaceflight analog. Our findings indicate that Playbook in C6 scored extremely well relative to a benchmark provided by the developers of the UEQ [25] and demonstrated comparable scores to those from prior testing in our controlled in-lab experiments [18]. While it remains uncertain to what extent general improvements and new features in Playbook [17] contributed to shifts in UX between lab and analog testing, our results suggest that Playbook is a usable system for actual autonomous crews in space missions.…”

“…Self-scheduling with Playbook in an analog environment provides a research opportunity for conversation analysis, applying Clinkenbeard's [5] framework to augment our previous laboratory-based approaches [18] in validating the tool for future long-duration exploration missions.…”

“…These scores were then compared to a benchmark (v12) derived from thousands of UEQ ratings of various products, provided by Schrepp et al [25]. Additionally, we compared these scores to past scores from a recent, larger in-lab sample (n = 30; [18]) which serve as a satisfaction baseline specifically for Playbook. This comparative analysis enables an assessment of whether Playbook performs in the upper percentiles of software UX, even when in a complex, collaborative environment.…”

“…Scores for each of these subscales were computed using a provided calculator (www.ueq-online.org [accessed on 8 February 2024]). The goal with the UEQ was to quantify the usability of our tool in the analog and roughly compare it with prior UEQ scores from lab-based testing [18,22]. Because usability questionnaires are commonly used in human-systems integration research, we sought to supplement our proposed conversation analysis by presenting scores for Playbook in a simulated spaceflight environment.…”