GitGalaxy · A study of AI coding agents

Every Model Has a TellThree AI models built programs for the same ten jobs. The shape of their code kept giving them away.

We gave Claude Code (running two different models) and OpenAI's Codex identical jobs: a chess engine and a logic-puzzle solver, each in five languages, 60 programs in all. Then we ignored every name and every word of every comment, measured only the code's structure, and asked: can you still tell who wrote it?

Programs group by language first; inside a language, by model

Each dot is one of the programs, placed so that programs with similar structure sit close together. Colour is the model that wrote it; the labels mark the five languages. Tap a dot to open that program.

Handwriting gives people away. A detective can match a ransom note to a suspect without reading a word of it, from the slant of the letters and the pressure of the pen. Code has handwriting too: how a program is cut into pieces, how long each piece runs, how many decisions it makes, which parts call which. Ignore what the names and comments say, and that shape is still there.

To find out whether AI coding agents have handwriting, we ran a controlled experiment. Three models built the same two programs, a chess engine that plays legal chess and a solver for logic puzzles called SAT, in C, Java, Python, Rust and COBOL. Each run started in a fresh, sealed container with the same written instructions and no human in the loop. An automatic judge with a hidden test set decided when a program was done.

Then we pointed GitGalaxy at the results. It reads source code the way an X-ray reads a skeleton: no running the program, no AI judge, just the structure.

How it works, in one paragraph. A job is one task in one language, such as "chess engine in Rust"; there are ten. For every program, GitGalaxy builds a shape profile: how many functions it has and how big they are, how many decisions each makes, which functions call which, and where comments sit (not what they say). Two profiles reduce to one shape difference: 0 means identical shape, and about 1 is typical for two programs written in different languages. Every comparison that matters below is made between programs doing the same job.

83%of programs had a closest look-alike, among programs doing the same job, written by the same model (43 of 52; a finding to repeat). Guessing would get 27%.
2× as alikeTwo programs by the same model for the same job differ about half as much as two by different models (shape difference 0.32 vs 0.62).
10 of 10jobs in which Haiku documented more of its functions than both Sonnet and Codex. It's the clearest single habit.
The short version
  • Our predictions weren't confirmed. We wrote down in advance that Claude would differ from Codex in five specific ways. After correcting for running several tests, none of the five passed; the closest missed by a hair.
  • But each model kept its handwriting. A model's second, independent round of programs looked like its first, and a program's closest look-alike came from the same model 83% of the time.
  • The habits belong to models, not companies. Sonnet and Haiku both run inside Claude Code, yet they differ from each other as much as either differs from Codex.
  • Language matters most. Changing a program's language moves its shape further than changing the model. That's why the model fingerprint only shows when you compare like with like.

Handwriting without a name

Here is the simplest possible test. Take any program, and line it up against every other program that did the same job. Which one has the most similar shape? If structure carried no trace of its author, the closest look-alike would come from the same model about as often as a blind pick.

A blind pick would land on the same model 27% of the time. The closest look-alike was the same model 43 times out of 52.

Two honest footnotes. This test was an afterthought, not one of the predictions we registered, so it needs repeating on fresh programs. And for Sonnet and Haiku, the closest look-alike was usually the model's own second round (34 of the 43 matches): this is mostly the "a model repeats itself" result seen from another angle. Codex ran once per job except in two, so only 4 Codex programs could be scored here (all 4 matched).

Who's the closest look-alike?

The closest look-alike is the same model three times as often as chance (exploratory)

measuredblind guess
For each program, its closest look-alike among the 3 to 9 other programs doing the same job. "Blind guess" is the share of same-model programs in each line-up, averaged. A shuffle test (relabelling the models at random 2,000 times) never reached 43.

The prediction that wasn't confirmed

This study began with someone else's experiment. Mathieu Acher, a computer scientist at the University of Rennes, published pairs of programs that Claude Code and Codex built for the same tasks, with a human steering each one. When we scanned his pairs, the two agents looked different: Claude seemed to comment more, Codex seemed to bail out of functions early with return, and so on.

Looking at data and then announcing what you saw is how you fool yourself, so we did it the slow way. We wrote down five predictions and the exact test for each, published them with a timestamp, and only then ran 60 fresh programs that no prediction had seen. Each test counts the plain way: in how many of the ten jobs did the prediction point the right way? The last column is the chance of a result that strong by luck alone, corrected for running several tests at once.

The scorecard

The vendor story didn't hold; the model story did

Each row is one prediction written down before the new programs existed. "Held in" counts the jobs where it pointed the right way.

confirmed survives the multiple-test correctionclose right direction, just shortnot shownreversed the opposite came out
Show all 15 registered tests
Source: the registered tests (sign tests over jobs, Holm-corrected within each group). The added Haiku tests were written down before the runs, too. "Not testable" means no program in either group did the thing at all. *Fragile: it loses significance if the failed runs are left out. The full registration and every deviation from it are linked under "Check my work".

The closest call was Codex's early exits (leaving a function early with return): they showed up in 9 of 10 jobs, and missed the bar by a hair. Nothing was refuted either. But the rows that did pass tell a different story from the one we went looking for. Haiku, the smaller Claude model, documents its functions far more than Codex and far more than its sibling Sonnet. Sonnet, meanwhile, reaches for while loops more than Haiku in every language that has them.

The habits didn't follow the company name on the box. They followed the model inside it.

What moves a program's shape

A fingerprint is only interesting if you know what else leaves marks. So we compared every pair of programs, about 1,700 of them, and sorted the pairs by what they had in common.

Every pair of programs

Changing the language changes the code three times as much as changing the model

Each dot is one pair of programs; hover for the pair. The black tick is each row's median.

Distance between the two programs' structure: 0 = same shape
Structural profile distances for all pairs among the 59 programs GitGalaxy could read. Pairs reuse programs, so the rows describe this set of 59 rather than independent trials. Pairs within the same job and language are coloured by model when both programs share it.
Show as a table

Read the chart top to bottom. Two runs of the same model on the same job sit close (median 0.32). Swap the model and the distance roughly doubles. Swap the task instead and it's about the same. Swap the language and the distance triples. A chess engine in COBOL and one in Python, written by the same model, have less in common than two Python chess engines by different models.

Language is the loudest accent. The model is the handwriting underneath it.

A model repeats itself

A fingerprint has to be stable to be useful. So Sonnet and Haiku each did the whole job twice, in two independent rounds, without any memory of the first. If the first round's style were a fluke, the second round would land anywhere.

Round one vs round two

A model's second attempt looks more like its first than like Codex

round 2 vs round 1 (same model)round 1 vs Codex
Distance between the two rounds of one model, against the distance from that model to Codex on the same job. One row per job; Codex's C SAT solver is missing (no functions GitGalaxy could read).
Show as a table

Haiku's comment habit

One habit is visible without any scanner at all. Haiku writes comments, and in particular it puts a comment above its functions. It does this more than Sonnet in every one of the ten jobs, and more than Codex in every one of them too. Codex, in most jobs, writes almost none.

Comments, job by job

Haiku documents more functions than both rivals in all 10 jobs

Show as a table

Be careful with what this means. It's not "Claude comments more": Sonnet and Haiku run in the same Claude Code tool and differ sharply. And it's about how much they comment and where, not what the comments say. A tool that sorts comments by meaning was right only about half the time on this code, so we didn't use it.

The clock

The agents also worked at very different speeds. Sonnet typically finished in under three minutes. Codex took about six. Haiku, the smaller and supposedly faster model, took around twenty. One Haiku run and one Codex run ran out the 60-minute clock.

How long each run took

Haiku worked about eight times longer than Sonnet

Active working time per run. The scale stretches short times so that equal gaps mean equal ratios (1 to 10 minutes is as wide as 6 to 60). The tick marks each round's median; diamonds are runs that hit the 60-minute cap. Both capped programs passed when checked afterwards.
Show as a table

What else GitGalaxy can now see

The registered tests are finished. But the same scans answer more questions, and these are where the next study starts. Everything in this section is exploratory: measured after the fact, each with its own comparison, and to be confirmed on fresh runs before anyone builds on it.

Play with it

See the wiring: one job, three models

Pick a task and language. Each map shows the program's functions as dots (bigger = called more often) and its calls as lines.

Play with it

The distance map: watch the blocks form

Every program against every other. Dark = similar shape. Re-sort the rows and see which grouping makes tidy blocks.

similardifferent

Swipe sideways to see the whole map.

Sorted by language first, the map breaks into five big blocks: language dominates. Sort by model first and the blocks dissolve, but inside each language the same-model squares stay darker.
Explore all 60

Every program, one card each

The text edition has every number without the interactives. Pick a program to see its habits and its line-up of look-alikes. Links like #sat-rust-haiku-r1 open a card directly.

What it can't tell you

How we know it's not luck

Built on Mathieu Acher's paired-agent studies (agentic-cobol-study, MIT licence), whose human-steered pairs suggested every prediction we tested. Any error here is ours.

Check my work