“…Markov decision processes (MDPs) and POMDPs are the methodological tools of choice for sequential decision‐making problems in healthcare, especially for chronic diseases such as diabetes (e.g., Hoerger et al., 2004, Santoso and Mareels 2001, Shih et al., 2007), HIV/AIDS (e.g., Garg et al., 2012, Lee et al., 2014, Shechter et al., 2008), and cancer (e.g., Ahsen and Burnside 2018, Ayer et al., 2012, Chhatwal et al., 2010, Maillart et al., 2008, Nohdurft et al., 2017, Petousis et al., 2019, Zhang et al., 2012a). MDPs and POMDPs have also been employed to obtain optimal policies in other settings, such as to design therapeutic regimens for hypertension (Zargoush et al., 2018), develop personalized treatments (Ibrahim et al., 2016), and address emergency room congestion (Patrick 2011).…”