Claude Code vs Codex: 100 Connect Four Games Head-to-Head

Previously: Same Question, Different Proposals

Last time, we at MIF put the same question to two AI CLIs — Claude Code (model: Fable 5) and Codex (model: GPT-5.6-Sol, reasoning effort: ultra) — and had them propose the contest itself. What came out of that was Round 1: the Connect Four arena. Connect Four is played on a seven-column, six-row grid where players alternate dropping discs, and the first to line up four of their own discs vertically, horizontally, or diagonally wins. Because the outcome is unambiguous, programs written by AIs can be judged by a shared referee instead of by human impressions. This is the project whose climax is a match rather than a scorecard — and this article is the live run.

How do you measure Claude Code and Codex? Code quality alone isn’t enough

At the design stage we got a little greedy. Having each AI write one bot and pitting them against each other tells you only which one writes the stronger code. As people who use AI at work, we wanted to know one more thing: can an AI run a team of AIs?

So Round 1 used two lanes. Claude is the commander of Lane A; Codex is the commander of Lane B. Rather than writing code themselves, each commander spins up one Claude worker and one Codex worker in fresh sessions and runs the team. Both lanes follow the same four stages.

1. Kickoff — the commander launches a Claude worker and a Codex worker on top of a clean copy prepared from a shared, pinned commit
2. Solo builds — each worker independently writes a specification and implements one solo bot (① and ② in Lane A, ④ and ⑤ in Lane B)
3. Merge — the commander compares and verifies the two solos and builds a Mix bot combining the best of each (③ and ⑥)
4. Final — Lane A’s Mix ③ faces Lane B’s Mix ⑥ under the referee program

This shape lets one experiment observe both questions at once: which model writes the stronger bot (the solo matchups between workers) and which model runs the better development team (the final between the Mixes).

A Mix here is not just a name for two answers placed side by side. It is a product of team management — the commander decided what to use as the base and what to bring in. So the final reflects not only the workers’ code but the commander’s choices. That said, the design does not isolate the commander as the cause of the result. That distinction matters when you read the numbers below.

The rules of the 100-game series — the referee is untouchable from the contestant side

Fairness rests entirely on a shared referee program, referee.py. The referee was placed in the repository as shared material before the contest began, and the candidate bots cannot modify it. Nor did one AI simply write it and walk away: the other reviewed it for fairness before it was finalized.

The contract imposed on the bots is plain. Receive the board and your own disc color, and return the column number to drop into. That is all. Search algorithm, evaluation function, time management — every design decision was left to the workers.

The match conditions, by contrast, were fixed strictly.

• 50 undisclosed legal opening positions × swapping who moves first = 100 games. The opening positions were prepared outside the repository by the judging side (a human) after the implementations were frozen, to stop deterministic bots from replaying the same game 100 times
• A two-second limit per move. Each bot runs in a separate process; exceeding the time limit, playing an illegal move, throwing an exception, or hanging without responding is an immediate forfeit
• Run sequentially on one machine (no parallelism), with every game record saved as JSON

Keeping every game record is not only about producing win-loss numbers. It is so a loss can be traced move by move after the fact, instead of being explained by impression. The linchpin of this arrangement was refusing to fit the evaluation to the implementations. Fix the pass criteria and the referee first, keep unseen inputs (the undisclosed positions) in reserve, and measure everyone with the same yardstick — the same thinking that goes into acceptance testing for business systems, applied rigorously to a small game.

Schematic of a Connect Four board (filled = first player, ringed = second player, four in a diagonal as the winning line). Not an actual game record.

We were equally careful with the fairness conditions. Both lanes started from a clean copy of the same pinned commit, and the workers were launched with a shared set of instructions. Each phase began in a fresh session, and neither lane saw the other’s deliverables until the Mixes were frozen. Dangerous permission bypasses were forbidden to both workers.

Results of the 100-game series — the solos were close, the Mix final was not

Across every game, there were zero forfeits. That means the tallies below can be read as decided on the board, not as numbers distorted by interface violations or timeouts.

Matchup	Claude side	Codex side	Draws
Lane A solo matchup (① vs ②)	48	45	7
Lane B solo matchup (④ vs ⑤)	45	42	13
Final: Claude-led Mix ③ vs Codex-led Mix ⑥	57	35	8

In the solo matchups, the bot built by the Claude worker came out slightly ahead in both Lane A and Lane B. To be precise, though, all we can say is that both lanes pointed the same way. The two lanes had different commanders and were not identical in every detail, so we will not call this an independent reproduction. And the numbers themselves show a margin that is close to even.

The final was a different picture. The individual bots had been that close, yet the Mixes finished 57 to 35. Even so, we will not attribute that gap purely to the commanders’ judgment. A Mix layers several decisions — which implementation was chosen as the base, which elements were brought in, how the verification was run — and differences in conditions remain between the lanes.

The most important finding — the Mix could not beat the best solo

The most valuable observation for us was not the win-loss record. It was this: in both lanes, the Mix the commander built by merging two solos failed to beat that lane’s strongest solo bot.

Qualitatively, Lane A’s Mix roughly preserved the strength of the solo it was built on. Lane B’s Mix, on the other hand, came out weaker than either of its parts. We read the gap in the final as this difference in merge quality surfacing.

In other words, there is a clear step between being able to compare two implementations and being able to build a stronger third one from them. That suggests a hypothesis: blending two roughly equivalent engines does not raise the ceiling on playing strength. Merging may pay off not when two capabilities overlap but when they are orthogonal. On a task like Connect Four, where “strength” collapses onto a single dimension, the gain from mixing may simply be small to begin with.

What to do with “mixing makes it better” in practice

To be honest, my own instinct has been that “have an AI review it” and “merge the ideas of two AIs” generally produce something better. That has held up in plenty of situations, and review and integration are standard improvement moves in software development anyway. Which is exactly why this result surprised us. On this task at least, simply involving multiple AIs did not guarantee a quality gain. Based on this experience, a commander running a team of AIs in practice needs at minimum these roles:

• Decide which deliverable is the baseline
• Compare before and after with the same evaluation method
• Keep every change revertible if it does not improve things
• Separate “the merge is done” as a task from “it got better” as an assessment

Put differently: a blended team needs an acceptance test as strict as the one applied to the individual versions. Keep the current best as the baseline, and re-measure the merged version on the same metric. If it does not beat the baseline, do not force the replacement. This time we could judge the Mixes by measurement rather than by how persuasive their names and descriptions sounded, because we had built the shared referee — the yardstick — first.

Comparing commander styles — Claude the pragmatist, Codex the formalist

Beyond wins and losses, the commanders showed clear personalities in how they did the job. Both took their two workers through to completion, and neither had to restart one. Against that common baseline, the contrast is the interesting part.

• Lane A (Claude commander) leaned pragmatic. It verified deliverables through worker logs and acceptance tests, and when a worker could not complete a commit, the commander stepped in once and did it. That is the only intervention on record
• Lane B (Codex commander) leaned formal. It built in hash-based sealing of every deliverable and an independent audit step, and ran its internal verification matches as a full round robin across the solos and Mixes. There are zero interventions on record

Neither is the right answer. It is an observation that the contrast familiar from human development teams — the leader who signs off quickly on something that works versus the leader who leaves an audit-proof paper trail — showed up in AI commanders as well. Thoroughness of management cannot be measured on one scale, and the depth of formal evidence was not the same thing as the strength of the final deliverable.

Wall-clock time — on time cost, Claude Code came out ahead

Finally, the other ledger: time. In Lane A, the two workers took 16.5 minutes for the Claude worker and 48 minutes for the Codex worker to complete the same task — a roughly threefold gap. The quality contest was close, but on development experience including wait time, Claude Code came out ahead in this run.

That gap cannot be read directly as “model speed,” however. The Codex worker ran with reasoning effort set to the ceiling, ultra, a setting that spends time thinking in pursuit of quality. Lowering the effort could shorten the time, and this experiment did not measure what that would do to quality. This is also one task with one observation each, and we did not publish per-worker times for the other lane. Record time and score separately — that is a policy we will keep throughout this series.

Next time — into e-commerce work, where results don’t line up in a single column

Round 1 was a match that could be collapsed into one number at the end. Round 2 goes the other way, into real business work. We hand both sides CSV data on an online store’s products, orders, and customers, and have them build a tool that proposes, with supporting evidence, which products, customers, and actions to prioritize this week. It is a task that tests where you draw the boundary — deterministic code for the aggregation, AI for the explanation — and the evaluation axes multiply beyond detection accuracy to include how clearly the reasoning reads and how safe the output is. It is also a serious contest, because we intend to use the result in our own e-commerce business.

When the evaluation axes no longer line up in a single column, how does the relationship between solo and Mix change? Episode 3 continues our hypothesis that merging pays off when capabilities are orthogonal.

This series (6 parts)

1. Same Question, Different Proposals
2. 100 Connect Four Games Head-to-Head (this article)
3. Claude Code vs Codex on Real EC Data
4. Does Merging Beat the Best Solo Run?
5. Can Claude Opus Run the Lane? Opus vs Fable as Controller
6. Three Controllers Compared, and ModelOrcs