More benchmarks · VideoGameBench Lite suite (3 games)

VideoGameBench: can AI play games from pixels?

0 records and 2 tracked runs or leaderboards for VideoGameBench Lite suite (3 games) only. Nothing on this page is compared with other games. How we verify.

Can AI play video games from raw pixels?

Barely, so far. On VideoGameBench (real-time), the current #1 is Gemini 2.5 Pro (preview 03-25): 0.48% (Overall score), as of May 23, 2025 (verified, from the official leaderboard). The Lite version pauses the game while the model thinks, and we keep its scores separate.

Tracked runs & leaderboards

VideoGameBench (real-time) ARCHIVE

Can vision-language models finish classic games from raw frames alone? The game keeps running while the model thinks. The overall score averages progress across 10 test games, including Pokémon Crystal.

⚠ The overall score is the benchmark's own average across its game suite. It is not an AI vs Games ranking.
How was this measured? (applies to every row)
Harness
VG-Agent (VideoGameBench reference agent)
Inputs
raw game frames + high-level objective and controls
Tools
none
Assistance
none
Budget
real-time (game does not pause)
Sources
www.vgbench.com · arxiv.org
Submit a correction
#ModelOverall scoreStatusAs of
1 Gemini 2.5 Pro (preview 03-25)
listed as VG-Agent + Gemini 2.5 Pro · best game: Kirby's Dream Land (DX) 4.8%
 0.48%Verified2025-05-23
2 GPT-4o (2024-08-06)
listed as VG-Agent + GPT-4o · best game: Pokemon Crystal 0.9%
 0.09%Verified2025-05-23
3 Claude 3.7 Sonnet (2025-02-19)
listed as VG-Agent + Sonnet 3.7 · 0% on every game
 0%Verified2025-05-23
3 Llama 4 Maverick (17B-128E Instruct FP8)
listed as VG-Agent + Llama 4 Maverick · 0% on every game
 0%Verified2025-05-23
3 Gemini 2.0 Flash
listed as VG-Agent + Gemini 2.0 Flash · 0% on every game
 0%Verified2025-05-23

Last checked: Oct 18, 2026 · auto-update: ok

VideoGameBench Lite (game pauses while the model thinks) ARCHIVE

The same benchmark with the game paused while the model thinks. Scores are not comparable with the real-time table.

⚠ The overall score is the benchmark's own average across its game suite. It is not an AI vs Games ranking.
How was this measured? (applies to every row)
Harness
VG-Agent (VideoGameBench reference agent)
Inputs
raw game frames + high-level objective and controls
Tools
none
Assistance
none
Budget
paused between actions (Lite)
Sources
www.vgbench.com · arxiv.org
Submit a correction
#ModelOverall scoreStatusAs of
1 GPT-4o (2024-08-06)
listed as VG-Agent + GPT-4o · best game: Kirby's Dream Land 4.8%
 1.6%Verified2025-05-23
1 Claude 3.7 Sonnet (2025-02-19)
listed as VG-Agent + Sonnet 3.7 · best game: Kirby's Dream Land 4.8%
 1.6%Verified2025-05-23
1 Gemini 2.5 Pro (preview 03-25)
listed as VG-Agent + Gemini 2.5 Pro · best game: Kirby's Dream Land 4.8%
 1.6%Verified2025-05-23
4 Llama 4 Maverick (17B-128E Instruct FP8)
listed as VG-Agent + Llama 4 Maverick · 0% on every game
 0%Verified2025-05-23
4 Gemini 2.0 Flash
listed as VG-Agent + Gemini 2.0 Flash · 0% on every game
 0%Verified2025-05-23

Last checked: Oct 18, 2026 · auto-update: ok

This Week in AI vs Games

A short weekly email with new records, memorable failures and runs to watch, every claim linked to its source.

Newsletter launching soon. Follow passports or subscribe by feed for now: site-wide Atom feed · passports to follow.

Latest issue: Issue #1: The Ender Dragon is down