Foundation Models as Decision Priors in Robotics: World-Model vs Policy Prior

DOI: https://doi.org/10.62517/jike.202604126

Author(s)

Changyi Wu

Affiliation(s)

University of Lancashire, Electronic Engineering, Preston, PR2 2QQ, The UK

Abstract

We survey how foundation models (FMs) act as decision priors in robot autonomy, contrasting world-model priors with policy priors across verifiability, constraint handling, latency, planning horizon, and out-of-distribution robustness. We formalize non-interchangeability boundaries, propose integration patterns (hierarchical, dynamic switching, blended control, human-in-the-loop, iterative co-evolution), and outline a benchmarking protocol together with deployment governance checklists. The resulting framework provides actionable guidance for engineering safe, efficient foundation-model-driven robot decision systems.

Keywords

Foundation Models; Robotics; Decision Priors; World Models [12]; Policy Priors; Safety; Benchmarking; Integration Strategies

References

[1] R. Bommasani et al., 2021. On the Opportunities and Risks of Foundation Models. Stanford CRFM Report (arXiv:2108.07258). [2] NeurIPS 2022 Workshop on Foundation Models for Decision Making. 2022. [Online]. Available: neurips.cc/virtual/2022/workshop/49988 [3] R. Firoozi et al., 2024. Foundation Models in Robotics: Applications, Challenges, and the Future. arXiv preprint arXiv:2312.07843. [4] A. Lykov et al., 2025. PhysicalAgent: Towards General Cognitive Robotics with Foundation World Models [12]. Submitted to IEEE Conference (arXiv:2509.13903). [5] A. Sohn et al., 2024. Introducing RFM-1 [15]: Giving robots human-like reasoning capabilities. Covariant Blog, 11 Mar 2024. [Online]. Available: covariant.ai/insights/introducing-rfm-1-giving- robots-human-like-reasoning-capabilities/ [6] M. Ahn et al., 2022. Do As I Can, Not As I Say: Grounding Language in Robotic Affordances (SayCan [1]). In Proc. of CoRL 2022, pp. 87–102. [7] J. Liang et al., 2023. Code as Policies [6]: Language Model Programs for Embodied Control. In Proc. IEEE ICRA 2023, pp. 9493–9500. [8] I. Singh et al., 2023. ProgPrompt [7]: Program Generation for Situated Robot Task Planning using LLMs. Autonomous Robots, 47(1): 999–1012. [9] M. Shridhar et al., 2023. ChatGPT for Robotics: Design Principles and Model Abstractions. arXiv preprint arXiv:2303.17071. [10] W. Ye et al., 2024. Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own. In Proc. CoRL 2024 (accepted, arXiv:2310.02635). [11] J. Huang et al., 2024. Foundation models and intelligent decision-making: Progress, challenges, and perspectives. The Innovation, 5(6): 100726. [12] P. Bowman-Davis, 2023. World Models [12] and the Sparks of Little Robotics. Andreessen Horowitz (a16z) Blog. [Online]. Available: a16z.com/world-models-and-the-sparks-of-little-robotics [13] R. Bandaru, 2023. Foundation Models for Robotics: Vision-Language-Action (VLA). Personal Blog. [Online]. Available: rohitbandaru.github.io/blog/Foundation-Models-for-Robotics-VLA/ [14] A. Latyshev, G. Gorbov, A. I. Panov, 2025. Safe Planning and Policy Optimization via World Model Learning. arXiv preprint arXiv:2506.04828. [15] N. Hansen, H. Su, X. Wang, 2024. TD-MPC2: Scalable, Robust World Models for Continuous Control. In Proc. ICLR 2024. [16] D. Shah et al., 2023. LM-Nav [8]: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action. In Proc. CoRL 2022, PMLR 205:334–345. [17] D. Driess et al., 2023. PaLM-E [5]: An Embodied Multimodal Language Model. arXiv preprint arXiv: 2303.03378. [18] Y. Chen et al., 2024. AutoTAMP [10]: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers. In Proc. IEEE ICRA 2024 (to appear, arXiv:2306.06531).