Personalised Medicine in Critical Care Using Bayesian Reinforcement Learning

Utomo, Chandra Prasetyo; Kurniawati, Hanna; Li, Xue; Pokharel, Suresh

doi:10.1007/978-3-030-35231-8_47

Cited by 8 publications

(5 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent days, machine learning techniques have been widely used in biomedical studies for prediction and knowledge discovery [ 23 , 24 ]. There are several applications of machine learning in biomedicine, such as genomic analysis [ [25] , [26] , [27] ], disease-gene analysis [ [28] , [29] , [30] , [31] ], mortality prediction [ 32 ], personalized medicine [ 33 , 34 ], drug discovery [ [35] , [36] , [37] , [38] ], prediction of adverse drug events [ [39] , [40] , [41] , [42] ], patient similarity [ 43 , 44 ], and explainable artificial intelligence methods in medicine [ [45] , [46] , [47] , [48] ]. One area for machine learning approaches in medicine is the association rule mining (ARM).…”

Section: Related Workmentioning

confidence: 99%

Discovering symptom patterns of COVID-19 patients using association rule mining

Tandan

Acharya

Pokharel

et al. 2021

Computers in Biology and Medicine

Self Cite

View full text Add to dashboard Cite

Background The COVID-19 pandemic is a significant public health crisis that is hitting hard on people's health, well-being, and freedom of movement, and affecting the global economy. Scientists worldwide are competing to develop therapeutics and vaccines; currently, three drugs and two vaccine candidates have been given emergency authorization use. However, there are still questions of efficacy with regard to specific subgroups of patients and the vaccine's scalability to the general public. Under such circumstances, understanding COVID-19 symptoms is vital in initial triage; it is crucial to distinguish the severity of cases for effective management and treatment. This study aimed to discover symptom patterns and overall symptom rules, including rules disaggregated by age, sex, chronic condition, and mortality status, among COVID-19 patients. Methods This study was a retrospective analysis of COVID-19 patient data made available online by the Wolfram Data Repository through May 27, 2020. We applied a widely used rule-based machine learning technique called association rule mining to identify frequent symptoms and define patterns in the rules discovered. Result In total, 1,560 patients with COVID-19 were included in the study, with a median age of 52 years. The most frequently occurring symptom was fever (67%), followed by cough (37%), malaise/body soreness (11%), pneumonia (11%), and sore throat (8%). Myocardial infarction, heart failure, and renal disease were present in less than 1% of patients. The top ten significant symptom rules (out of 71 generated) showed cough, septic shock, and respiratory distress syndrome as frequent consequents. If a patient had a breathing problem and sputum production, then, there was higher confidence of that patient having a cough; if cardiac disease, renal disease, or pneumonia was present, then there was a higher confidence of septic shock or respiratory distress syndrome. Symptom rules differed between younger and older patients and between male and female patients. Patients who had chronic conditions or died of COVID-19 had more severe symptom rules than those patients who did not have chronic conditions or survived of COVID-19. Concerning chronic condition rules among 147 patients, if a patient had diabetes, prerenal azotemia, and coronary bypass surgery, there was a certainty of hypertension. Conclusion The most frequently reported symptoms in patients with COVID-19 were fever, cough, pneumonia, and sore throat; while 1% had severe symptoms, such as septic shock, respiratory distress syndrome, and respiratory failure. Symptom rules differed by age and sex. Patients with chronic disease and patients who died of COVID-19 had severe symptom rules more specifically, cardiovascular-related symptoms accompanied by pneumonia, fever, and cough as consequents.

show abstract

Section: Related Workmentioning

confidence: 99%

Discovering symptom patterns of COVID-19 patients using association rule mining

Tandan

Acharya

Pokharel

et al. 2021

Computers in Biology and Medicine

Self Cite

View full text Add to dashboard Cite

show abstract

“…An initial 129 articles were screened on title and abstract, 27 of which were reviewed in full text resulting in 8 included journal articles (19)(20)(21)(22)(23)(24)(25)(26) and 7 conference papers (27)(28)(29)(30)(31)(32)(33). Backward and forward citation chases of the initially included articles returned 904 articles which were screened on title and abstract, of which 90 articles were reviewed in full text resulting in 10 included journal articles (34)(35)(36)(37)(38)(39)(40)(41)(42)(43) and 11 conference papers (44)(45)(46)(47)(48)(49)(50)(51)(52)(53)(54).…”

Section: Resultsmentioning

confidence: 99%

Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment

Otten,

Jagesar,

Dam

et al. 2023

Critical Care Medicine

View full text Add to dashboard Cite

Objective: Reinforcement learning (RL) is a machine learning technique uniquely effective at sequential decision-making, which makes it potentially relevant to ICU treatment challenges. We set out to systematically review, assess level-of-readiness and meta-analyze the effect of RL on outcomes for critically ill patients. Data Sources: A systematic search was performed in PubMed, Embase.com, Clarivate Analytics/Web of Science Core Collection, Elsevier/SCOPUS and the Institute of Electrical and Electronics Engineers Xplore Digital Library from inception to March 25, 2022, with subsequent citation tracking. Data Extraction: Journal articles that used an RL technique in an ICU population and reported on patient health-related outcomes were included for full analysis. Conference papers were included for level-of-readiness assessment only. Descriptive statistics, characteristics of the models, outcome compared with clinician’s policy and level-of-readiness were collected. RL-health risk of bias and applicability assessment was performed. Data Synthesis: A total of 1,033 articles were screened, of which 18 journal articles and 18 conference papers, were included. Thirty of those were prototyping or modeling articles and six were validation articles. All articles reported RL algorithms to outperform clinical decision-making by ICU professionals, but only in retrospective data. The modeling techniques for the state-space, action-space, reward function, RL model training, and evaluation varied widely. The risk of bias was high in all articles, mainly due to the evaluation procedure. Conclusion: In this first systematic review on the application of RL in intensive care medicine we found no studies that demonstrated improved patient outcomes from RL-based technologies. All studies reported that RL-agent policies outperformed clinician policies, but such assessments were all based on retrospective off-policy evaluation.

show abstract

“…BC can effectively recover the doctor's policies when the Electronic Health Record (EHR) is optimal and plentiful. However, these methods are not suitable if the ground truth of the treatment is unclear [7].…”

Section: Related Workmentioning

confidence: 99%

“…Communities and groups update best practices over time, since there are no ground truths with respect to best treatments [5,6]. This makes the existing system inexpedient [7]. On the other hand, Reinforcement Learning (RL) techniques learn and update themselves through trial and error.…”

Section: Introductionmentioning

confidence: 99%

Projection based inverse reinforcement learning for the analysis of dynamic treatment regimes

et al. 2022

View full text Add to dashboard Cite

Dynamic Treatment Regimes (DTRs) are adaptive treatment strategies that allow clinicians to personalize dynamically the treatment for each patient based on their step-by-step response to their treatment. There are a series of predefined alternative treatments for each disease and any patient may associate with one of these treatments according to his/her demographics. DTRs for a certain disease are studied and evaluated by means of statistical approaches where patients are randomized at each step of the treatment and their responses are observed. Recently, the Reinforcement Learning (RL) paradigm has also been applied to determine DTRs. However, such approaches may be limited by the need to design a true reward function, which may be difficult to formalize when the expert knowledge is not well assessed, as when the DTR is in the design phase. To address this limitation, an extension of the RL paradigm, namely Inverse Reinforcement Learning (IRL), has been adopted to learn the reward function from data, such as those derived from DTR trials. In this paper, we define a Projection Based Inverse Reinforcement Learning (PB-IRL) approach to learn the true underlying reward function for given demonstrations (DTR trials). Such a reward function can be used both to evaluate the set of DTRs determined for a certain disease, as well as to enable an RL-based intelligent agent to self-learn the best way and then act as a decision support system for the clinician.

show abstract

Personalised Medicine in Critical Care Using Bayesian Reinforcement Learning

Cited by 8 publications

References 13 publications

Discovering symptom patterns of COVID-19 patients using association rule mining

Discovering symptom patterns of COVID-19 patients using association rule mining

Does Reinforcement Learning Improve Outcomes for Critically Ill Patients? A Systematic Review and Level-of-Readiness Assessment

Projection based inverse reinforcement learning for the analysis of dynamic treatment regimes

Contact Info

Product

Resources

About