Abstract
Bearings‐only tracking (BOT) in sonar’s passive mode presents significant nonlinearity challenges, addressed here through a measure of nonlinearity (MoN) based on the filter’s covariance matrix. This study computes MoN to evaluate filter performance and enhance target motion estimation precision. To advance beyond traditional methods, we introduce a novel LSTM‐based MoN framework that captures temporal nonlinear patterns, an adaptive MoN using reinforcement learning for dynamic adjustments, and attention‐based multisensor fusion for improved accuracy. MoN values are estimated across various filtering techniques, including extended Kalman filter (EKF), unscented Kalman filter (UKF), particle filter (PF), and the proposed enhancements. Monte Carlo simulations (1000 runs) demonstrate the following quantified improvements: the LSTM–MoN achieves a 10%–15% RMSE reduction during maneuvering phases over static covariance‐based MoN methods and a 24.8% overall RMSE improvement over the Gaussian sum shifted Rayleigh filter (GSRF) in Scenario 1; the adaptive Q‐learning MoN provides a 19.8% improvement over the constrained multiple model particle filter (CMM‐PF) in high‐noise maneuvering conditions; and the attention‐based fusion accelerates convergence by 20% and delivers an 18.2% accuracy gain over fixed‐weight fusion. Collectively, these innovations achieve an overall RMSE reduction of 14.5%–25% against contemporary benchmarks, with detailed trade‐offs in computational complexity. These advances enhance robustness in multisensor underwater environments, validated under varying noise levels (1°–10°) and target maneuvers.