Guidelines for reinforcement learning in healthcare

Gottesman, Omer; Johansson, Fredrik; Komorowski, Matthieu; Faisal, A. Aldo; Sontag, David; Doshi‐Velez, Finale; Celi, Leo Anthony

doi:10.1038/s41591-018-0310-5

Cited by 266 publications

(175 citation statements)

References 4 publications

Supporting

Mentioning

175

Contrasting

Order By: Relevance

“…As a result, long-term effects are harder to estimate. 8 The main difference between supervised and unsupervised learnings is whether the training data set has labeled outputs corresponding to input data. The supervised learning infers a mathematical relationship between the inputs and the labeled outputs while the unsupervised learning infers a function that expresses hidden characteristics reside in input data.…”

Section: Supervised and Unsupervised Learningmentioning

confidence: 99%

Deep Learning in Medical Imaging

et al. 2019

View full text Add to dashboard Cite

The artificial neural network (ANN), one of the machine learning (ML) algorithms, inspired by the human brain system, was developed by connecting layers with artificial neurons. However, due to the low computing power and insufficient learnable data, ANN has suffered from overfitting and vanishing gradient problems for training deep networks. The advancement of computing power with graphics processing units and the availability of large data acquisition, deep neural network outperforms human or other ML capabilities in computer vision and speech recognition tasks. These potentials are recently applied to healthcare problems, including computer-aided detection/diagnosis, disease prediction, image segmentation, image generation, etc. In this review article, we will explain the history, development, and applications in medical imaging

show abstract

Section: Supervised and Unsupervised Learningmentioning

confidence: 99%

Deep Learning in Medical Imaging

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Similarly to the way we used drivers’ gaze in a driving simulator to train a self-driving car algorithm in that simulator, we can use rich neurobehavioural data from an expert driver in extreme conditions to train a real self-driving car algorithm to response successfully in extreme conditions. Likewise, on the side of control systems, we showed for example how using ethomic data obtained from natural tasks (movement data 41 , electrophysiological data 42 , decision making data 43 ) can be harnessed to boost AI system performance. The neurobehavioural approach demonstrated here suggests how we can succeed in future to close-the-loop between person and vehicle.…”

Section: Discussionmentioning

confidence: 99%

Neurobehavioural signatures in race car driving: a case study

Lima

Haar

Grassi³

et al. 2020

Sci Rep

View full text Add to dashboard Cite

Recent technological developments in mobile brain and body imaging are enabling new frontiers of real-world neuroscience. Simultaneous recordings of body movement and brain activity from highly skilled individuals as they demonstrate their exceptional skills in real-world settings, can shed new light on the neurobehavioural structure of human expertise. Driving is a real-world skill which many of us acquire to different levels of expertise. Here we ran a case-study on a subject with the highest level of driving expertise-a formula e champion. We studied the driver's neural and motor patterns while he drove a sports car on the "Top Gear" race track under extreme conditions (high speed, low visibility, low temperature, wet track). His brain activity, eye movements and hand/foot movements were recorded. Brain activity in the delta, alpha, and beta frequency bands showed causal relation to hand movements. We herein demonstrate the feasibility of using mobile brain and body imaging even in very extreme conditions (race car driving) to study the sensory inputs, motor outputs, and brain states which characterise complex human skills. One of the hallmarks of being human is our unique ability to develop skills and expertise. While all animals develop skills like walking, running, fruit picking or hunting, we as humans can develop a much broader and more diverse set of skills. With practice, most of us can learn to play a musical instrument, play a sport, or do arts and crafts. Nevertheless, only some of us can reach the highest level of expertise. Unlike the widespread view that this is entirely driven by practice 1 , there is accumulating evidence that practice is not enough 2 , making individual musicians, artists, athletes, and craftspeople who take this expertise to new heights of particular research interest. Novel technology for mobile brain and body imaging now enables us to study neurobehaviour in real-world settings 3-5. When carried out in natural environments, these measures can enable a meaningful understanding of human behaviour while performing real-life tasks. Studying the relation and inter-dependencies between brain activity and body movements of experts, while they perform their expert skills in real-world settings, can enable us to unpack this enigma. In recent years there has been an accumulating body of literature studying the neural signals associated with expertise, particularly in sports 6. EEG studies link expert performance to changes in EEG alpha and beta rhythms. However, most of these studies are using lab-based tasks, and therefore their findings have had little impact on sports professionals 6 (for example, the Go/No-Go task was used to study baseball expertise 7). While their findings showed that experts perform better and have more robust EEG inhibition responses, which can tell us something about their skill, it is far from true expertise. Other studies addressed expertise in a real task in a trial-by-trial design. For example, expert rifle shooters exhibited longer quiet eye period befor...

show abstract

“…A relatively new approach to optimising treatment of critically ill patients with sepsis in the ICU is with the use of computerised programs that utilise artificial intelligence (AI) in generating their recommendations. Unlike dosing software with Bayesian forecasting which uses PK and statistical modelling to individualise therapy to patients, AI software uses reinforcement learning to generate recommendations on appropriate interventions required to achieve predetermined targets for patients [104]. Artificial intelligence software examines data from large patient population databases to identify interventions associated with the target outcome and combines this information with an individual patient's characteristics to determine the most appropriate intervention that will maximise the probability of achieving the predefined outcome [105,106].…”

Section: Artificial Intelligence Softwarementioning

confidence: 99%

What Are the Current Approaches to Optimising Antimicrobial Dosing in the Intensive Care Unit?

et al. 2020

View full text Add to dashboard Cite

Antimicrobial dosing in the intensive care unit (ICU) can be problematic due to various challenges including unique physiological changes observed in critically ill patients and the presence of pathogens with reduced susceptibility. These challenges result in reduced likelihood of standard antimicrobial dosing regimens achieving target exposures associated with optimal patient outcomes. Therefore, the aim of this review is to explore the various methods for optimisation of antimicrobial dosing in ICU patients. Dosing nomograms developed from pharmacokinetic/statistical models and therapeutic drug monitoring are commonly used. However, recent advances in mathematical and statistical modelling have resulted in the development of novel dosing software that utilise Bayesian forecasting and/or artificial intelligence. These programs utilise therapeutic drug monitoring results to further personalise antimicrobial therapy based on each patient’s clinical characteristics. Studies quantifying the clinical and cost benefits associated with dosing software are required before widespread use as a point-of-care system can be justified.

show abstract

Guidelines for reinforcement learning in healthcare

Abstract: In this Comment, we provide guidelines for reinforcement learning for patient treatment decisions that we hope will accelerate the rate at which observational cohorts can inform healthcare practice in a safe, risk-conscious manner.

Cited by 266 publications

References 4 publications

Deep Learning in Medical Imaging

Deep Learning in Medical Imaging

Neurobehavioural signatures in race car driving: a case study

What Are the Current Approaches to Optimising Antimicrobial Dosing in the Intensive Care Unit?

Contact Info

Product

Resources

About