The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems

Abstract

Recent technological advances, especially in the field of machine learning, provide astonishing progress on the road towards artificial general intelligence. However, tasks in current real-world business applications cannot yet be solved by machines alone. We, therefore, identify the need for developing socio-technological ensembles of humans and machines. Such systems possess the ability to accomplish complex goals by combining human and artificial intelligence to collectively achieve superior results and continuously improve by learning from each other. Thus, the need for structured design knowledge for those systems arises. Following a taxonomy development method, this article provides three main contributions: First, we present a structured overview of interdisciplinary research on the role of humans in the machine learning pipeline. Second, we envision hybrid intelligence systems and conceptualize the relevant dimensions for system design for the first time. Finally, we offer useful guidance for system developers during the implementation of such applications.

Application of artificially intelligent systems for the identification of discrete fossiliferous levels

“…Considering the currently unquantifiable nature of numerous geological features that ML algorithms may be unable to detect, a human-in-the-loop collaborative strategy was proposed for this study (Kamar, 2016;Holzinger, 2016;Dellermann et al, 2019). The objectives of including this hybrid intelligence strategy meant that human interaction could complement the strengths of the pattern detection algorithms (Simard et al, 2017;Dellermann et al, 2019), whereby creating a bridge between both unsupervised and supervised techniques for model creation.…”

“…This was performed as objectively as possible through complimenting paleontological data routinely collected from the site during excavation and evaluating the nature of each of DBSCAN's groupings. EitL thus benefits through Machine Teaching (MT) in as much as the expert knowledge is used for troubleshooting and debugging (Dellermann et al, 2019; also known as a sense-making approach). Moreover, the amount of human input for models was monitored in a collective manner, using numerous domain experts to ensure accuracy (Dellermann et al, 2019).…”

“…EitL thus benefits through Machine Teaching (MT) in as much as the expert knowledge is used for troubleshooting and debugging (Dellermann et al, 2019; also known as a sense-making approach). Moreover, the amount of human input for models was monitored in a collective manner, using numerous domain experts to ensure accuracy (Dellermann et al, 2019). In cases where clusters seemed dubious or could not be clearly defined by either the EitL participants or the MT algorithm, these were stripped of their labels and included in the noise (cluster "0") group for objective classification by the trained ML algorithms in the supervised phase of the system.…”

Evolutions of Artificial Intelligence: What issues for human activities and Human-Machine relationships at work ?

“…Une première se réfère aux formes où l'humain « augmente » la machine en l'alimentant en données, en définissant la bonne stratégie d'apprentissage qui lui permettra d'atteindre le niveau de performance recherché, en lui faisant des retours sur ces performances, ou en expliquant ses actions (ceci ne va pas de soi comme nous le verrons dans la troisième partie). Cette idée d'augmentation de la machine par l'Homme, qui est relativement nouvelle et qui est de plus en plus envisagée (voir égalementDellermann, Calma, Lipusch, Weber, Weigel, & Ebel, 2019) laisse cependant une question ouverte, à savoir si elles s'ajouteront aux activités habituellement réalisées par les opérateurs, ce qui soulèverait la question de l'augmentation de la charge de travail de l'humain. La seconde partie du modèle regroupe les formes dans lesquelles l'humain est « augmenté » par la machine.…”

Peer Review #3 of "Application of artificially intelligent systems for the identification of discrete fossiliferous levels (v0.1)"

“…180 181 DBSCAN performs through establishing areas of significant point densities, thus providing an 182 efficient means of finding non-parametric patterns among noisy data sets while easily detecting 183 outliers in low-density regions. This is especially useful when used for pattern recognition in 230 Considering the currently unquantifiable nature of numerous geological features that ML 231 algorithms may be unable to detect, a human-in-the-loop collaborative strategy was proposed for 232 this study (Kamar, 2016;Holzinger, 2016;Dellermann et al, 2019). The objectives of including 233 this hybrid intelligence strategy meant that human interaction could complement the strengths of 234 the pattern detection algorithms (Simard et al, 2017;Dellermann et al, 2019), whereby creating 235 a bridge between both unsupervised and supervised techniques for model creation.…”