After looking at my LLM benchmark table, many people ask how exactly I test models and what the evaluation criteria are. To avoid repeating it in every video, I published this article explaining my methodology.
Each model receives 7 projects to be tested, with a total of 27 prompts.
The projects cover Laravel/PHP, React + TypeScript, Flutter/Dart and Go, and every one of them starts from existing code, not a blank folder. That's what most of our real work looks like.
What kind of tasks these are
- The model is the implementer. One coding agent (Claude Code, Codex CLI, OpenCode, Cursor…) gets one prompt and a repository, and does the work itself.
- It's not orchestration. There are no subagent swarms, no multi-agent pipelines, and no human feedback in between. I'm not testing how well a model manages other agents.
- The tasks are not long-running. The median task takes 4–11 minutes, depending on the project. These are focused, ticket-sized jobs, not overnight "build me a SaaS" sessions.
- No retries or coaching. Every prompt starts in a fresh copy of the project, and the model never sees the hidden tests or the scoring rubric.
The 7 projects
Starting point → prompt → expected result
How big is one task?
Median values per prompt, across 2,076 runs of all tested models since August 2026:
In other words, these are tasks a senior developer would plan for an hour or two, and a good agent finishes in under 10 minutes.
How it becomes leaderboard points
The same prompt is launched several times, each in a fresh copy of the project, so that one lucky (or unlucky) run doesn't decide the score:
If you want more information about methodology or you have ideas or suggestions, email me povilas@laraveldaily.com.
