Value-Aware Error Weights under a Robust Transition Set
Abstract
Adaptive model-error weighting emphasizes the regions important to a policy, while a robust Bellman update protects a defined transition set. Evaluation should preserve support coverage and ambiguity scale, inspecting each contribution independently. Faster optimization does not establish the scale's plausibility or prevent exploitation of errors outside the represented protection. This focused synthesis develops an empirical comparison that preserves the terminal action, observation process, and total verification cost. Its proposed tests are prospective; no new simulation run, model-training result, or intervention effect is reported.
References
- Z. Qiao, J. Lyu, B. Lyu, Y. Shu, S. Gao, and S. Qiu, “Model-Based Offline RL via Robust Value-Aware Model Learning with Implicitly Differentiable Adaptive Weighting,” Proceedings of the 2026 International Conference on Learning Representations, 2026.
- Z. Yu, L. Dai, S. Xu, S. Gao, and C. Ho, “Fast Bellman Updates for Wasserstein Distributionally Robust MDPs,” Advances in Neural Information Processing Systems, pp. 30554–30578, 2023.
- Sutton, Richard S., and Andrew G. Barto. Reinforcement Learning: An Introduction. 2nd ed., MIT Press, 2018. http://incompleteideas.net/book/the-book-2nd.html.
- Levine, Sergey, Aviral Kumar, George Tucker, and Justin Fu. “Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems.” arXiv, 2020, arXiv:2005.01643. https://arxiv.org/abs/2005.01643.
- Kumar, Aviral, Aurick Zhou, George Tucker, and Sergey Levine. “Conservative Q-Learning for Offline Reinforcement Learning.” Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 1179–1191. https://arxiv.org/abs/2006.04779.
- Lewis, Patrick, et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems, vol. 33, 2020, pp. 9459–9474. https://arxiv.org/abs/2005.11401.
- Geifman, Yonatan, and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems, vol. 30, 2017. https://arxiv.org/abs/1705.08500.
- Ovadia, Yaniv, et al. “Can You Trust Your Model’s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift.” Advances in Neural Information Processing Systems, vol. 32, 2019. https://arxiv.org/abs/1906.02530.
- Bertsimas, Dimitris, and Melvyn Sim. “The Price of Robustness.” Operations Research, vol. 52, no. 1, 2004, pp. 35–53. https://doi.org/10.1287/opre.1030.0065.
- Rasmussen, Carl Edward, and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT Press, 2006. https://gaussianprocess.org/gpml/.