The Human in the Infinite Loop: A Case Study on Revealing and Explaining Human-AI Interaction Loop Failures

The Impact of Expertise in the Loop for Exploring Machine Rationality

“…Additionally, in the 3D model simplification task, a user can rotate, zoom, and move the four models simultaneously to inspect and compare the quality of the models. In line with prior work by Ou et al [53], we use a listwise interface with four variants instead of two pairwise comparisons to increase the collected feedback in each iteration without increasing system processing and data transmission time. After the user submits the ranking information, the background system will utilize this information and then optimize and infer the next optimal set of variants.…”

“…This service allows five adjustable system hyper-parameters that are bounded between 0 and 1: 1) brightness, 2) contrast, 3) saturation, 4) temperature, and 5) tint. Lastly, we used a 3D mesh reducer [53] as a 3D mesh processing service, and it also contains five bounded system hyper-parameters: 1) simplification ratio, 2) border preservation, 3) hard edge preservation, 4) sharpness preservation, and 5) quadrilateral preservation.…”

“…However, the system might not always be effective in achieving user satisfaction. There are several known reasons for this, such as context [57], timing [28], trustworthiness [35], cognitive biases [12], and unstable and contradicting preferences [53]. Specifically, in cocreation, a user is not always satisfied with the results generated by the machine due to the lack of practical creativity in the system [50]; in a personalized recommendation, the machine may converge to some fixed recommended content and cannot bring fresh ideas for users [32]; even in the process of AI-assisted decision-making, users may not hold enough trust in the results provided by an algorithm [18].…”

“…To cover a spectrum of different application domains, we consider text summarization [64], photo color enhancement [39][40][41], and a 3D model simplification task [21,31,53] to evaluate the relation between user expertise, satisfaction, and system outcome quality when interacting with an intelligent system. Figure 1 shows one of our ranking interfaces for HITL optimization.…”