Interactive Optimal Teaching with Unknown Learners

Abstract

This paper introduces a new approach for machine teaching that partly addresses the (unavoidable) mismatch between what the teacher assumes about the learning process of the student and the actual process. We analyze several situations in which such mismatch takes place, including when the student?s learning algorithm is known but the corresponding parameters are not, and when the learning algorithm itself is not known. Our analysis is focused on the case of a Bayesian Gaussian learner, and we show that, even in this simple case, the lack of knowledge regarding the student?s learning process significantly deteriorates the performance of machine teaching: while perfect knowledge of the student ensures that the target is learned after a finite number of samples, lack of knowledge thereof implies that the student will only learn asymptotically (i.e., after an infinite number of samples). We introduce interactivity as a means to mitigate the impact of imperfect knowledge and show that, by using interactivity, we are able to recover finite learning time, in the best case, or significantly faster convergence, in the worst case. Finally, we discuss the extension of our analysis to a classification problem using linear discriminant analysis, and discuss the implications of our results in single- and multi-student settings.

Teaching Multiple Inverse Reinforcement Learners

“…In this case, the distribution could be used to decide when to do individual teaching or group teaching. The group teaching might still be possible, but some form of interaction might be needed (Walsh and Goschin, 2012 ; Haug et al, 2018 ; Melo et al, 2018 ).…”

“…For applications involving humans, the complexity of the algorithm is not a problem, but the problem is assumption of knowing the learner's decision-making process (i.e., the rewardless MDP describing the human). In future work, we will consider how to include interaction in the teaching process, to overcome the lack of knowledge regarding the human learner, as was done for other teaching problems (Melo et al, 2018 ). Other applications of machine teaching include the study of possible attacks to machine learners (Mei and Zhu, 2015 ).…”

“…Some works (Walsh and Goschin, 2012 ; Haug et al, 2018 ; Melo et al, 2018 ) investigate the impact that the mismatch between the learner and the teacher's model of the learner may have in the teaching dimension—a situation particularly relevant in group settings. The aforementioned works focus on supervised learning settings, although some more recent works have explored inverse reinforcement learning (IRL) settings (Kamalaruban et al, 2019 ).…”

Fine-Grained Machine Teaching with Attention Modeling

“…Some of these works (Patil et al 2014;Zhou, Nelakurthi, and He 2018) emphasize constructing the personalized optimal teaching set with consideration of the ability of different learners, such as Zhou et al proposes JEDI teaching framework (Zhou, Nelakurthi, and He 2018), where each learner has an exponentially decayed memory. Other works like (Simard et al 2017;Melo, Guerra, and Lopes 2018) concern the mismatch between teachers' assumption on the learners' performance and the actual performance of learners. The rest of works aim at improving the learning performance with different approaches (Mac Aodha et al 2018;Chen et al 2018).…”

Interactive Teaching Algorithms for Inverse Reinforcement Learning

“…Most of the work in machine teaching is in a batch setting where the teacher provides a batch of teaching examples at once without any adaptation. The question of how a teacher should adaptively select teaching examples for a learner has been addressed recently but only in the supervised learning setting [Melo et al, 2018;Liu et al, 2018;Yeo et al, 2019].…”