Machine Teaching: A New Paradigm for Building Machine Learning Systems

Abstract

The current processes for building machine learning systems require practitioners with deep knowledge of machine learning. This significantly limits the number of machine learning systems that can be created and has led to a mismatch between the demand for machine learning systems and the ability for organizations to build them. We believe that in order to meet this growing demand for machine learning systems we must significantly increase the number of individuals that can teach machines. We postulate that we can achieve this goal by making the process of teaching machines easy, fast and above all, universally accessible.While machine learning focuses on creating new algorithms and improving the accuracy of "learners", the machine teaching discipline focuses on the efficacy of the "teachers". Machine teaching as a discipline is a paradigm shift that follows and extends principles of software engineering and programming languages. We put a strong emphasis on the teacher and the teacher's interaction with data, as well as crucial components such as techniques and design principles of interaction and visualization.In this paper, we present our position regarding the discipline of machine teaching and articulate fundamental machine teaching principles. We also describe how, by decoupling knowledge about machine learning algorithms from the process of teaching, we can accelerate innovation and empower millions of new uses for machine learning models.

Machine Teaching by Domain Experts: Towards More Humane,Inclusive, and Intelligent Machine Learning Systems

“…We argue here for exploring machine teaching techniques, existent and future, towards enabling experts' direct access to ML systems. We agree with (Simard et al, 2017) that focusing on the teacher requires simplifying many tasks executed today by the ML experts, so it is important to identify the set of requirements for machine teaching technology which are specific to facilitate access to experts: • Transparency: domain experts tend to be more effective if they understand the inner workings of the system (Thomaz and Breazeal, 2008).…”

“…In both cases, human beings are treated as machine feeders, working intensively to transform data into a form which can be easily digested by a ML algorithm. There is ample evidence that people do not perform well doing those tasks (Aroyo and Welty, 2015;Flexer and Grill, 2016): they get easily bored, make frequent mistakes, and use concepts and rules whose semantics varies wildly with time (Simard et al, 2017). Also, a consistent and motivated labeller is rare to find.…”

“…Although tools for transferring knowledge from experts to traditional AI systems were explored in the past (Quintana-Amate et al, 2015), the research of Stumpf et al [2009] and Kaochar et al [2011] were among the first to look into how experts could teach ML systems using concepts and methods beyond examples. In particular, (Fogarty et al, 2008;Amershi et al, 2015;Simard et al, 2017) have explored machine teaching by interactive feature selection and composition. Recent works have looked into other aspects of machine teaching, including the teachers' mental models (Sarkar, 2015), textual description of procedures (Azaria et al, 2016), and natural language explanations (Hancock et al, 2018).…”

The Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems

“…Shared Data Representation: The shared data representation is what is the data that is shown to both the human and the machine before executing their tasks. The data can be represented in different levels of granularity and abstraction to create a shared understanding between humans and machines [22,43]. Features describe phenomenas in different kinds of dimensions like height and weight of a human being.…”

“…Moreover, end users can provide the system with input for product recommendations and e-commerce or input from human non-experts accessed through crowd work platforms [64,58,65]. More recent endeavors, however, focus on the integration of domain experts in hybrid intelligence architectures that leverage the profound understanding of the semantics of a problem domain to teach a machine, while not requiring any ML expertise [22,66,67].…”