Atari-GPT: Benchmarking Multimodal Large Language Models as Low-Level Policies in Atari Games

Nicholas R. Waytowich, Devin White, MD Sunbeam, and Vinicius G. Goecks

KnowFM Workshop at AAAI 2025 · Poster

Atari-GPT tests multimodal language models as zero-shot, low-level policies in Atari. A model observes the current game frame and returns the controller action to execute next, without game-specific training.

The tested models could identify game elements and produce plausible strategies, but overall performance remained well below human and dedicated reinforcement-learning agents. A separate evaluation points to one likely weakness: spatial reasoning was weaker than object recognition and strategy selection.

I built the evaluation harness and ran the in-context learning experiments, which used human-labeled frame–reasoning–action examples as demonstrations.

Atari-GPT system diagram showing game frames sent to a multimodal model, actions returned to Atari, and a separate human-scored question-answer evaluation — **Figure 1.** Atari-GPT has two evaluations: closed-loop game control and a separate question-answer test of visual understanding, spatial reasoning, strategy, and game identification.Figure from Waytowich et al. (2024), CC BY 4.0.Open full-size figure

Benchmark

GPT-4V Turbo, GPT-4o, Gemini 1.5 Flash, and Claude 3 Haiku were evaluated in Alien, Breakout, Frogger, Ms. Pac-Man, Pong, Seaquest, and Space Invaders. At each decision, the model received the current frame and two previous frame-response pairs, then returned JSON containing an action and an explanation. Each rollout lasted 1,000 timesteps; results average four rollouts per model and environment.

The understanding test asked four questions about a static frame: visible elements, their locations, an acceptable next strategy, and the game identity. Responses were scored against a human-written rubric. The primary comparison covers the same four models used for gameplay.

What the models could and could not do

GPT-4o had the highest average human-normalized reward at 23.2%. GPT-4V Turbo reached 18.36%, Claude 3 Haiku 12.36%, and Gemini 1.5 Flash 8.5%. Performance varied by game: Space Invaders was strongest, while every tested model scored below the random-agent result in Pong.

Bar chart of average human-normalized Atari reward: GPT-4o 23.2 percent, GPT-4V Turbo 18.36 percent, Claude 3 Haiku 12.36 percent, and Gemini 1.5 Flash 8.5 percent — **Figure 2.** Average human-normalized reward across seven games without in-context demonstrations. Scores use four 1,000-step rollouts per model and environment.Figure from Waytowich et al. (2024), CC BY 4.0.

GPT-4o also led the four-model understanding comparison. Spatial reasoning was the weakest category for every model, even when visual recognition, strategy, and identification were substantially better. This gap is consistent with action errors accumulating over long gameplay rollouts.

Scope

This is a snapshot of hosted systems available in 2024. API inference took roughly two to three seconds for GPT-4o, Gemini 1.5 Flash, and Claude 3 Haiku, and five to seven seconds for GPT-4V Turbo; inconsistent output and rate limits added delay. The understanding test used a human-authored rubric and few repeats. Its source is inconsistent about trial count: the method states four trials, while the results text and figure report two runs.