Abstract
The main goal of this work is to play a role of playground to further improve our own conceptual model, or “unbiased Deep Deterministic Policy Gradient (uDDPG)” developed recently: due to search for an algorithm that can provide faster convergence to better policies the agent can meet a problem of falling to the local optimum regions and slightly improve from there for the rest of experiments. In other words, proposed concept has a problem of limited policy generalization, especially for more complex systems. To achieve a better exploration and consequently search for higher scores, we tried to equip our concept with simplified ideas from the notable works: we unanchored uDDPG or in parallel added Q network's update without the link to real Q values but with Entropy Regularization of soft Q learning based algorithms, e.g. Soft Actor and Critic; additionally, reduced ideas behind Forward-Looking Actor (FORK), Twin Delayed DDPG and Distributional Perspective on Q value were utilized for more “optimistic” but “safer” search.