Markov Models and Cost Effectiveness Analysis: Applications in Medical Research

Abstract

Learning ObjectivesUnderstand how Markov models can be used to analyze medical decisions and perform cost-effectiveness analysis.This case study introduces concepts that should improve understanding of the following:1. Markov models and their use in medical research. 2. Basics of health economics. 3. Replicating the results of a large prospective randomized controlled trial using a Markov Chain and Monte Carlo simulations, and 4. Relating quality-adjusted life years (QALYs) and cost of interventions to each state of a Markov Chain, in order to conduct a simple cost-effectiveness analysis. IntroductionMarkov models were initially theroreticized at the beginning of the 20th century by Russian mathematician Andrey Markov [1]. They are stochastic processes that undergo transitions from one state to another. Over the years, they have found countless applications, especially for modeling processes and informing decision making, in the fields of physics, queuing theory, finance, social sciences, statistics and of course medicine. Markov models are useful to model environments and problems involving sequential, stochastic decisions over time. Representing such environments with decision trees would be confusing or intractable, if at all possible, and would require major simplifying assumptions [2]. Markov models can be examined by an array of tools including linear algebra (brute force), cohort simulations, Monte Carlo simulations and, for Markov Decision Processes, dynamic programming and reinforcement learning [3,4].

Sequencing Systemic Therapy Pathways for Advanced Hepatocellular Carcinoma: A Cost Effectiveness Analysis

“…We performed a cost effectiveness analysis (CEA) for systemic therapy in advanced HCC using different 1st-and 2nd-line scenarios, taking into account the usual toxicities associated with each class of medication. We used Markov modeling, ideal for environments involving sequential, stochastic decisions over time including cancer treatment, and we present a framework for modeling medical decision-making [17].…”

A Transitional Probability Model for Parkinson’s Disease Motor States With Applications to Missing Data

“…Markov model has been used extensively in various settings to forecast future events or to estimate transitional probabilities from one state to another. Applications include, to name a few, agriculture, 14 weather forecasting, 15 social sciences, 16 health economics, 17 and medical decision making. 18 In this paper, we present a model for estimating the transitional probabilities of PD motor states.…”

Cost effectiveness of different screening strategies for gestational diabetes mellitus screening: study protocol of a randomized community non-inferiority trial

“…MDPs are most useful in classes of problems involving complex, stochastic and dynamic decisions like medical treatment decisions, for which they can find optimal solutions [68]. Physicians will always need to make subjective judgments about treatment strategies, but mathematical decision models can provide insight into the nature of optimal choices and guide treatment decisions [69]. Markov models can be used to describe various health states in a population of interest, and to detect the effects of various policies or therapeutic choices.…”