Six Frontier AI Models, One Blind Coding Benchmark

I gave six frontier AI models the same six coding problems, stripped every answer of its author, and then made the models grade each other. No model was allowed to score itself. In this test Opus 5 led every one of the six domains, the field behind it is tight, and because nobody scored themselves, none of it is self-flattery. It is six tasks and one run, so read it as a signal about these tasks rather than a league table.

1. The standings

I ran two scoring formats. A per-domain pass, where six models scored every answer from 0 to 10 in each of six domains, and an earlier holistic pass, where five models gave a single 0 to 100 score per answer. The holistic pass predates Opus 4.7 joining the field and therefore covers five of the six models. Where they overlap, the two formats produce the same ordering, which is a useful check that the result is not an artefact of one scoring shape.

2. How the benchmark works

My design goal was to remove the two biggest sources of bias in any "which model is best" comparison: a single opinionated judge, and self-scoring. Four rules did the work.

• Blind. Every answer was stripped of its model name and relabelled "Answer 1…N" before any judge saw it.
• Peer jury. Each of the six models graded the other five. Nobody grades themselves, so no model can inflate its own score.
• Per-domain. Instead of one blurry overall number, judges scored each answer separately in every one of the six domains.
• Fresh sessions. Every answer came from a clean session with no memory, so no model had prior context or a warm-up advantage.

Each cell in the heatmap below is therefore the mean of five independent, blind peer scores. Five competitors who cannot see the labels converging on the same answer is a reasonable check against a single judge's taste. It is not a correctness proof: no answer here was compiled, unit-tested or reviewed by a human expert.

3. The six tasks

I chose tasks small enough to answer concisely but discriminating enough to separate strong engineers from merely fluent ones. Each one has a trap in it.

4. The heatmap

This is the headline visual. Green is good, red is not, and a star marks the best answer in each column.

5. Domain by domain

Rust: Opus 5 (9.0), then GPT Sol (8.6) and Grok (8.2)

Every model reached the same correct architecture for the LRU cache: a HashMap<K, usize> pointing into an index-based doubly-linked list inside a Vec, guarded by a single Mutex. Using array indices instead of pointers sidesteps the borrow checker entirely, with no Rc<RefCell<…>> and no unsafe. The separation came from polish. Opus 5 and Kimi used a leaner NIL sentinel index; the others wrapped nodes in Option.

On the bug fix, the whole field spotted the classic error: after v.remove(i) the next element shifts into slot i, but i += 1 skips it. Only Opus 5 and Grok also supplied the O(n) two-pointer rewrite instead of leaving an O(n²) remove inside a loop.

The correct minimal fix is simply this: only advance i when nothing was removed.

Go: Opus 5 (9.0), Kimi (7.6), Opus 4.7 (7.4)

This is the domain that spread the field most, and the discriminator is subtle. When a worker sends its result, is that send guarded against context cancellation? Opus 5 and Kimi wrapped it in a select { case results <- r: case <-ctx.Done(): return }, so a cancelled pool cannot deadlock on a consumer that has stopped reading. GPT Sol (6.6) and Grok (5.8) sent unguarded, which is a goroutine-leak risk on shutdown.

Grok's answer was the most elegant on the surface, a fully generic RunPool[T,R], but its Job.Run did not take a context, so mid-job cancellation is impossible. The peers penalised it hard, and they were right to.

Postgres: Opus 5 (9.6), Kimi (9.2), Grok (8.8)

The strongest domain across the board. Everyone used ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY created_at DESC, id DESC) filtered to rn = 2, with the id as a deterministic tie-breaker and a matching composite index. Everyone correctly explained why RANK() or DENSE_RANK() would break on duplicate timestamps: two rows tie at rank 1, nobody gets rank 2, and the customer disappears from the result. Opus 5 edged ahead with a LATERAL … OFFSET 1 LIMIT 1 alternative, while Kimi and GPT Sol added an INCLUDE (amount) covering index.

Svelte: Opus 5 (8.6), GPT Sol (8.2), then a cliff

The frontend task exposed the biggest quality gap in the whole benchmark. Opus 5 was the only answer with all three of: an AbortController to cancel in-flight requests, an aria-live status region, and an SSR-safe generated id, all driven reactively from a single $effect. GPT Sol was close, adding a "committed" state so the component does not re-search after a selection.

Below those two it drops off sharply. Grok (6.6) shipped a real stale-response race: no abort and no sequence guard, so a slow older request can overwrite a newer one. Kimi avoided the race with a sequence counter but was the least runes-idiomatic. Fable 5 and Opus 4.7 (both 5.6) were judged the shallowest.

Design and UX: Opus 5 (9.0), GPT Sol (7.6), Grok (7.0)

Design here means deliberate interaction choices, not decoration. Opus 5 stood out clearly: arrow keys wrap, aria-activedescendant tracks the highlighted option without moving DOM focus so keyboard users can keep typing, and, the thoughtful part, pressing Enter with nothing highlighted submits the free text rather than forcing a dropdown selection. Fast users who already know what they want never have to wait on the network.

Next.js: Opus 5 (8.2), Kimi and Fable (7.4)

The tightest domain. Everyone described the same server-component shell with a client island, fetching through a Route Handler or Server Action to keep database clients and API keys on the server. Opus 5 and Kimi added the extra detail that earns the point: caching frequent queries with unstable_cache or revalidate, and wrapping the island in <Suspense> so the rest of the page streams.

6. The holistic peer jury

Before the per-domain pass, an earlier run had five models grade each other with a single 0 to 100 score. Opus 4.7 was added to the field later and does not appear here. Among the five it covers, it reached the same ordering as the per-domain pass. The matrix below shows every judge's verdict, and the diagonal is blank because no model judges itself, which is why each answer carries four ratings rather than five.

7. Model profiles

8. Practical takeaways

The guidance falls straight out of the heatmap.

• Default coder, any language: Opus 5.
• Independent second opinion: GPT Sol, but not for concurrency-critical Go.
• Database and SQL work: Kimi k3 is genuinely strong here.
• Be careful with Grok on concurrency-critical Go and on unguarded frontend fetches.
• Generation gap: the jump from Opus 4.7 to Opus 5 is large, and it is largest on frontend and design specifically.

9. Conclusion

The signal here is unusually clean because it comes from competitors grading each other blind. On these six tasks, Opus 5 led every domain, and every peer independently put it first. That is a strong result for this test, not a universal ranking.

Behind it, the picture is nuanced rather than a straight line. Pick GPT Sol for polish and architecture, Kimi k3 for databases, and be cautious with Grok on Go and on frontend races. The two models that trailed, Fable 5 and Opus 4.7, lose specifically on raw code quality on these tasks rather than on presentation.

One footnote worth stating plainly: Fable 5, which ran the harness, finished last in the holistic pass. The peers did not spare the orchestrator.

10. Method and reproducibility

11. Frequently asked questions