Abstract
Offline reinforcement learning offers a way to develop O-RAN controllers from historical network data without testing new actions on a live network. Its usefulness, however, depends on whether the logs preserve meaningful action sequences and whether policy values can be estimated reliably. We examine these questions using the public TRACTOR dataset and a preference-conditioned multi-objective Conservative Q-Learning (MO-CQL) controller. After checking timestamps and resource allocations, we retained 5438 transitions from 63,594 recorded action events and split them by experiment for training, validation, and testing. The recorded network state predicts throughput reasonably well, but the logged action adds little predictive information, and resource efficiency is difficult to predict. Comparisons with simple policies do not show a stable performance advantage for MO-CQL. The data also provide limited evidence for long-term evaluation: only 217 of 1422 adjacent test pairs form verified consecutive action sequences, and the longest such sequence contains four transitions. By contrast, a discount factor of 0.99 places weight on roughly 100 transitions. Estimates from fitted Q evaluation change substantially with training duration and the treatment of sequence boundaries. A separate experiment with known dynamics shows that splitting stored data into short files does not itself cause this problem. What matters is whether those boundaries change the return being estimated; neural approximation introduces further error. These findings offer a practical way to check the reliability of offline policy comparisons in O-RAN. Under a fixed FQE protocol, one MO-CQL checkpoint yields value estimates close to those of five separately trained scalar CQL policies while using fewer parameters and less storage. The efficiency objective is treated as a weak-signal, telemetry-sensitive proxy, and the main contribution is the evaluation-reliability framework.