Medical treatment decisions are often sequential and uncertain. Markov decision processes
(MDPs) are an appropriate technique for modeling and solving such stochastic and dynamic …

We provide a tutorial on the construction and evaluation of Markov decision processes
(MDPs), which are powerful analytical tools used for sequential decision making under …

Markov models are useful when a decision problem involves risk that is continuous over
time, when the timing of events is important, and when important events may happen more …

Markov decision processes (MDPs) have found success in many application areas that
involve sequential decision making under uncertainty, including the evaluation and design …

Clinical decisions often have long-term implications. Analysts encounter difficulties when
employing conventional decision-analytic methods to model these scenarios. This occurs …

In recent years, Markov decision processes (MDPs) and partially observable Markov
decision processes (POMDPs) have found important applications to medical decision …

It is over 30 years ago since DJ White started his series of surveys on practical applications
of Markov decision processes (MDP), 1, 2, 3 over 20 years after the phenomenal book by …

OBJECTIVE: In the modern healthcare system, rapidly expanding costs/complexity, the
growing myriad of treatment options, and exploding information streams that often do not …

Diagnosis of a disease and its treatment are not separate, one-shot activities. Instead, they
are very often dependent and interleaved over time. This is mostly due to uncertainty about …

Markov models are often employed to represent stochastic processes, that is, random
processes that evolve over time. In a healthcare context, Markov models are particularly …