General Agents Need World Models

Jonathan Richens, David Abel, Alexis Bellot and Tom EverittThis paper focuses on the necessity of world models for creating general and capable AI agents, specifically those that can generalize to multi-step goal-directed tasks. The authors formally demonstrate that any agent capable of this type of generalization must have learned a predictive model of its environment, and that the accuracy of this learned model is directly tied to the agent's performance and the complexity of the goals it can achieve. They provide a method for extracting this learned world model from the agent's policy and show that myopic agents, which only optimize for immediate outcomes, do not require a world model. The work has implications for the development of safe, general, and interpretable AI, suggesting that explicitly model-based approaches may be more fruitful than model-free ones for achieving advanced AI capabilities.