Human-Machine Collaborative Optimization via Apprenticeship Scheduling

Abstract

Coordinating agents to complete a set of tasks with intercoupled temporal and resource constraints is computationally challenging, yet human domain experts can solve these difficult scheduling problems using paradigms learned through years of apprenticeship. A process for manually codifying this domain knowledge within a computational framework is necessary to scale beyond the "single-expert, single-trainee" apprenticeship model. However, human domain experts often have difficulty describing their decision-making processes, causing the codification of this knowledge to become laborious. We propose a new approach for capturing domainexpert heuristics through a pairwise ranking formulation. Our approach is model-free and does not require enumerating or iterating through a large state space. We empirically demonstrate that this approach accurately learns multifaceted heuristics on a synthetic data set incorporating job-shop scheduling and vehicle routing problems, as well as on two real-world data sets consisting of demonstrations of experts solving a weapon-to-target assignment problem and a hospital resource allocation problem. We also demonstrate that policies learned from human scheduling demonstration via apprenticeship learning can substantially improve the efficiency 1 arXiv:1805.04220v1 [cs.AI] of a branch-and-bound search for an optimal schedule. We employ this human-machine collaborative optimization technique on a variant of the weapon-to-target assignment problem. We demonstrate that this technique generates solutions substantially superior to those produced by human domain experts at a rate up to 9.5 times faster than an optimization approach and can be applied to optimally solve problems twice as complex as those solved by a human demonstrator.

Joint Goal and Strategy Inference across Heterogeneous Demonstrators via Reward Network Distillation

“…In robotics, preference learning typically involves learning a human's preference and adapting a robot's behavior to better collaborate with human-beings [16,28]. There are several methods that could infer preference information without asking the human to explicitly provide preference information [14,15]. For example, Schafer et al [34] utilizes collaborative filtering for a recommender system and Xu et al [42] formulates a meta-IRL problem that could learn a prior over preferences.…”

Sequential attack salvo size is monotonic nondecreasing in both time and inventory level

“…The framework we consider is common to military weapon target engagements (Aviv & Kress, 1997;Gombolay et al, 2018;Kisi, 1976;Mastran & Thomas, 1973), shooting problems (Glazebrook & Washburn, 2004;Sato, 1997a), and more broadly to stochastic sequential resource allocation (Sato, 1996(Sato, , 1997b. Monotonicity properties for the related bomber problem, where a bomber has to survive sequential engagements with enemy aircraft by firing a volley (one or more) of weapons are discussed in (Weber, 2013).…”

Fast Approximations for Job Shop Scheduling: A Lagrangian Dual Deep Learning Method

“…For example, when a malfunction occurs or when operating conditions require a new schedule, replanning needs to be executed promptly as machine idle time can be extremely costly. (e.g., on the order of $10,000 per minute for some applications [12]). To address this issue, system operators typically seek approximate solutions to the original scheduling problems.…”