“…Override rules, sometimes also referred to as shields, have been applied ad-hoc in multiple DNN-enabled systems, such as DeepRM (Mao et al, 2016a) and Pensieve (Mao et al, 2017). Such rules, and related forms of runtime monitors, are also found in control systems for robots (Phan et al, 2017), drones (Desai et al, 2018), and in various other formalisms which are not directed particularly at deep learning (Hamlen et al, 2006;Falcone et al, 2011;Schierman et al, 2015;Ji and Lafortune, 2017;Wu et al, 2018). The formal methods community has recently taken an interest in override rules for systems with DNNs: for example, by proposing techniques to synthesize rules that affect the controller as little as possible (Avni et al, 2019;Wu et al, 2019).…”