Our goal in writing this book has been to provide a comprehensive, mathematically rigorous,
but still accessible treatment of the interaction between information and control in multi …

Markov decision processes (MDPs), also called stochastic dynamic programming, were first
studied in the 1960s. MDPs can be used to model and solve dynamic decision-making …

We consider reinforcement learning (RL) in Markov Decision Processes in which an agent
repeatedly interacts with an environment that is modeled by a controlled Markov process. At …

This technical note studies the problem of exponential estimates for Markovian jump
systems with mode-dependent interval time-varying delays. A novel Lyapunov-Krasovskii …

For discrete-time nonlinear systems, this research is concerned with optimal control
problems (OCPs) with constrained cost, and a novel value iteration with constrained cost …

The study of continuous-time Markov decision processes dates back at least to the 1950s,
shortly after that of its discrete-time analogue. Since then, the theory has rapidly developed …

In classical Markov decision process (MDP) theory, we search for a policy that, say,
minimizes the expected infinite horizon discounted cost. Expectation is, of course, a risk …

We consider linear programming (LP) problems in infinite dimensional spaces that are in
general computationally intractable. Under suitable assumptions, we develop an …

Solutions to decentralized stochastic optimization problems lead to recursions in which the
state space enlarges with the time-horizon, thus leading to non-tractability of classical …

Control and optimization of dynamical systems in the presence of stochastic uncertainty is a
mature field with a large range of applications. A comprehensive treatment of such problems …