Value iteration is a powerful yet inefficient algorithm for Markov decision processes (MDPs)
because it puts the majority of its effort into backing up the entire state space, which turns out …

This paper describes GPU based algorithms to compute state transition models for
unmanned surface vehicles (USVs) using 6 degree of freedom (DOF) dynamics simulations …

Markov decision processes (MDPs) provide a rich framework for planning under uncertainty.
However, exactly solving a large MDP is usually intractable due to the “curse of …

Abstract Markov Decision Processes have become the standard model for probabilistic
planning. However, when applied to many practical problems, the estimates of transition …

This paper investigates optimal power flow management problem in an electric vehicle
charging station. The charging station is powered by solar PV and is tied to the grid and a …

We present a look-ahead based trajectory planning algorithm for computation of dynamically
feasible trajectories for Unmanned Sea Surface Vehicles (USSV) operating in high seas …

This work presents the concept of Continuous Search (CS), which objective is to allow any
user to eventually get their constraint solver achieving a top performance on their problems …

We introduce the lookahead-bounded Q-learning (LBQL) algorithm, a new, provably
convergent variant of Q-learning that seeks to improve the performance of standard Q …

Current methods for solving Stochastic Shortest Path Problems (SSPs) find states' costs-to-
go by applying Bellman backups, where state-of-the-art methods employ heuristics to select …

Probabilistic model checking is a technique employed for verifying the correctness of
computer systems that exhibit probabilistic behaviour. A related technique is controller …