In the framework of current off-policy actor-critic methods, the state–action pairs in an
experience replay buffer (called historical behaviors) cannot be used to improve the policy …

On-policy reinforcement learning algorithms have been shown to be remarkably efficient at
learning policies for continuous control robotics tasks. They are highly parallelizable and …

On-policy deep reinforcement learning (DRL) has the inherent advantage of using multi-step
interaction data for policy learning. However, on-policy DRL still faces challenges in …

Soft actor-critic (SAC) is an off-policy actor-critic (AC) reinforcement learning (RL) algorithm,
essentially based on entropy regularization. SAC trains a policy by maximizing the trade-off …

This paper presents Noisy Importance Sampling Actor-Critic (NISAC), a set of empirically
validated modifications to the advantage actor-critic algorithm (A2C), allowing off-policy …

Compared to on-policy counterparts, off-policy model-free deep reinforcement learning can
improve data efficiency by repeatedly using the previously gathered data. However, off …

Abstract We introduce Phasic Policy Gradient (PPG), a reinforcement learning framework
which modifies traditional on-policy actor-critic methods by separating policy and value …

Actor-critic (AC) algorithms are known for their efficacy and high performance in solving
reinforcement learning problems, but they also suffer from low sampling efficiency. An AC …

This paper presents the first actor-critic algorithm for off-policy reinforcement learning. Our
algorithm is online and incremental, and its per-time-step complexity scales linearly with the …

Off-policy reinforcement learning algorithms maintain a replay buffer to utilize samples
obtained from earlier policies. The sampling strategy that prioritizes certain data in a buffer to …