Multi-Level Deep Deterministic Policy Gradient reinforcement learning for safety-constrained trajectory optimization of collaborative industrial robots

Akhanseri Ikramov, Sayat Ibrayev

2026

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.3389/frai.2026.1914042
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Collaborative robots operating in modern manufacturing environments require trajectory planning strategies that simultaneously support efficiency, adaptability, and operational safety. This study proposes a Multi-Level Deep Deterministic Policy Gradient (ML-DDPG) framework for safety-constrained trajectory optimization. The framework combines a strategic waypoint level, a continuous DDPG motion-control policy, and a reactive feasibility projection that corrects unsafe actions before execution. The trajectory-planning problem is formulated as a Markov Decision Process with reward terms for goal progress, obstacle clearance, smoothness, energy, execution time, and safety. Descriptive experiments in a custom PyTorch-based three-dimensional environment compare ML-DDPG with conventional planners and the DDPG, TD3, and SAC baselines. In the retained single training run, ML-DDPG produced observed values of 98.7% success, 1.08 mm trajectory error, a 0.98 smoothness index, and no recorded collision. These values are single-run observations and are not means over independent seeds; variability and statistical significance cannot be estimated from the available record. The exported traces show faster reward growth within the 200-episode budget, and the single-run ablation record associates the largest observed gains with the combined multi-level, reward, and projection components. One UR5e demonstration yielded 0.018 m mean tracking error, 0.042 m maximum tracking error, and no collision in that trial. A separate ABB welding sequence is included only as qualitative feasibility evidence because synchronized trajectory, force, and weld-quality logs were unavailable. The findings motivate further evaluation, while multi-seed training, repeated hardware trials, complete reporting of evaluation procedures, and cross-platform validation remain necessary before broader generalization.