7-minute video for Premium members. Experiment: what if I benchmark models not on ONE prompt, but on the set of FOUR prompts? I tried this approach with 5 top LLMs, which scored the best?
To each model, I gave a sequence of 4 prompts.
Prompt 1:
# Phase 1 — Private catalog and member access.
Build the first usable version of the equipment library.
<...> 20+ more rows in this prompt
Prompt 2:
# Phase 2 — Reservations, checkout, and return
Members can now request one active equipment item for an inclusive start and end date. A valid request begins as `pending` and appears on that member's reservations page.
<...> 20 more rows in the prompt
And then phase 3 and 4 in separate prompts.
In other words, the whole mini-project built in a sequence of prompts, in the same agent session, with the same context.
And then, there was a set of evaluation tests (that LLMs did NOT see upfront) asserting if all specification was properly implemented.
Finally, there was an evaluation on code quality from another external LLM judge.
Here is how I calculated the final score across test completion, weighted results, and code quality.
Premium members get:
- The complete score table for all five models
- My 7-minute video breakdown
- Cost, speed, and code-quality comparisons
- My recommendation for long-running coding tasks
