Small offline models reportedly went from about 7% to 56% on ARC-AGI-3 in a month. Here is what the benchmark tests, why the jump caused an argument on Reddit, and how to judge any AI benchmark for your own work.
Published October 6, 2026. Scores are as reported in the Reddit thread; check the Kaggle leaderboard for current numbers.
Top scores in the ARC Prize 2026 Kaggle competition reportedly rose from about 7% to 56% within 30 days, using small models that run offline. ARC-AGI-3 was built to show tasks that are easy for people and hard for AI, so a fast jump by small local models set off a debate: real progress, or competitors learning the benchmark?
The Reddit thread: r/MachineLearning: “Top ARC-AGI-3 scores on Kaggle just went from 7% to…”.
ARC-AGI is a series of benchmarks from the ARC Prize Foundation, started by François Chollet. Earlier versions used static grid puzzles. ARC-AGI-3 is interactive: the agent is dropped into small game-like environments with no instructions and has to explore, infer the goal and solve it efficiently (ARC-AGI-3 paper).
It measures skill acquisition, how quickly a system learns something new, rather than how much it already knows.
| "This is real progress" | "This is benchmark fitting" |
|---|---|
| Small, offline models did it, so it is not just scale. | Public games let teams tune search strategies to the game style. |
| Agents that explore and test hypotheses are a general skill. | Hidden test games may not behave like the public ones. |
| Earlier ARC versions also fell faster than expected. | Every benchmark saturates once it becomes a target. |
Both sides can be right. Fast benchmark gains usually mix genuine technique improvements with fitting to the test.
A leaderboard score cannot tell you whether an AI will answer your customers correctly or qualify your leads. The useful test is small and specific: take 20-50 real examples of the task, run the AI on them, and have the person who does the job today grade the output.
This is how Dooza pilots work: we run the AI on your real work, measure it against how your team does the job, and you decide with evidence instead of leaderboards.
Dooza is an AI-native company that builds AI products and services for small businesses, from the Dooza Workforce app to the Dooza Agents platform. A Dooza engineer scopes your pilot on a free 30-minute call, and every product starts with a refundable pilot: 100% refund within 14 days. Book a free pilot call or see pricing.
An interactive AI benchmark from the ARC Prize Foundation in which agents must learn the rules of unfamiliar game-like environments by exploring them. It measures how efficiently a system acquires new skills.
A viral r/MachineLearning thread in early October 2026 reported top Kaggle scores rising from about 7% to 56% within 30 days. Check the Kaggle leaderboard for current figures.
No. Kaggle submissions run in a sandbox without internet access, so they cannot call hosted model APIs. Teams use models that run offline.
Not reliably. Test AI on 20-50 real examples of your own task and have the person who does that job grade the results.
Automate your business with AI employees that work 24/7. Start with a refundable pilot: 100% refund within 14 days.
A Florida woman treated Claude like a private diary. A threat she wrote was flagged, reviewed by people at Anthropic and sent to police. Here is what AI chats are and are not, and the policy every business using AI should write this week.
PewDiePie says OpenAI banned him twice while he trained Ajax, a 9B local model, on outputs from OpenAI models. Here is what distillation means, why AI labs ban it, and the platform risk lesson for businesses.
Start with a refundable pilot — 100% refund within 14 days. A Dooza engineer scopes it with you on a free 30-minute call. Pricing depends on the product; see pricing.
Refundable pilot · 100% refund within 14 days · No contracts