Predicting Human Discretion to Adjust Algorithmic Prescription: A Large-Scale Field Experiment in Warehouse Operations

Abstract

Conventional optimization algorithms that prescribe order packing instructions (which items to pack in which sequence in which box) focus on box volume utilization yet tend to overlook human behavioral deviations. We observe that packing workers at the warehouses of the Alibaba Group deviate from algorithmic prescriptions for 5.8% of packages, and these deviations increase packing time and reduce operational efficiency. We posit two mechanisms and demonstrate that they result in two types of deviations: (1) information deviations stem from workers having more information and in turn better solutions than the algorithm; and (2) complexity deviations result from workers’ aversion, inability, or discretion to precisely implement algorithmic prescriptions. We propose a new “human-centric bin packing algorithm” that anticipates and incorporates human deviations to reduce deviations and improve performance. It predicts when workers are more likely to switch to larger boxes using machine learning techniques and then proactively adjusts the algorithmic prescriptions of those “targeted packages.” We conducted a large-scale randomized field experiment with the Alibaba Group. Orders were randomly assigned to either the new algorithm (treatment group) or Alibaba’s original algorithm (control group). Our field experiment results show that our new algorithm lowers the rate of switching to larger boxes from 29.5% to 23.8% for targeted packages and reduces the average packing time of targeted packages by 4.5%. This idea of incorporating human deviations to improve optimization algorithms could also be generalized to other processes in logistics and operations. This paper was accepted by Charles Corbett, operations management.

Using Explainable Artificial Intelligence to Improve Process Quality: Evidence from Semiconductor Manufacturing

“…We make no specific assumption about which metamodel is used. This enables the straightforward use of established models in the operations management literature (e.g., tree ensembles; Cui et al 2018, Sun et al 2021. As part of the robustness checks, we showed that using an alternative metamodel (with similar modeling performance) results in identical improvement actions.…”

“…The metamodel is estimated based on all production parameters and the normalized yield using gradient boosting with decision trees (Ke et al 2017). Gradient boosting belongs to the category of tree ensemble algorithms, which are known for performing well on complex data sets and have already been applied in other operational applications (e.g., Cui et al 2018, Sun et al 2021. We utilize common procedures (cf.…”

Beyond correlation: Towards matching strategy for causal inference in Information Science

“…RCT is an experimental trial in which subjects are randomised and distinct treatments are performed on two groups to compare potential causal effects [17,18]. Take Sun et al [19] as an example: They proposed a new algorithm of prescribing order packing instructions (e.g. time order of packing different packages, which packages in which sizes of bins/boxes and so on) by considering workers' behavioural deviations.…”

“…Besides randomizations and control, RCTs require another principle called 'blinding' [7]. Workers under the RCT of Sun et al'.s study [19], regardless of SG or CG, were unaware of the RCT and had no clue on whether they had received the 'treatment' or not. Through RCT, Sun et al found that the average packing time of SG significantly increased, in comparison with CG (oftentimes used as a benchmark or baseline in RCTs) and thus concluded that their proposed algorithm improves productivity of packing.…”

Editorial: How to deal with the human factor in supply chain management?

“…Based on these factors, a model is built that suggests order decisions for bypassing the human planner (as they will not be altered anyhow). Sun et al (2019) illustrate how analytics can incorporate human behavior in warehouse operations using a field experiment at the Alibaba Group. As workers deviate from the suggested box sizes for e-commerce shipments, the authors use machine learning to predict if a worker uses a different size.…”