How listings are checked
Every game on this site has a Source tab: the game’s program, with comments saying what each part does. This page explains how the site measures two things about those comments: how much of the program they cover, and how many of them are right. The examples come from C64 games on the site.
The About tab, with Commando’s numbers
| Coverage | 100 % of the bytes the game uses have a description |
|---|---|
| Comments checked | Audited & low error rate. Every comment was audited once, then the last sample of 70 comments had 1 wrong, so between 0 and 8 % of the listing’s comments are likely wrong. |
Under these rows, the About tab lists every step that led to them. Commando’s list is further down.
The Source tab shows the game’s listing. A listing is the whole program, byte by byte, in the order it sits in memory. Code appears as instructions, and tables and variables appear as bytes. A routine or a table starts with a name and a comment, and the comment says what the whole routine or table does. Every byte up to the next name belongs to it.
Coverage is the share of the program’s bytes that belong to a routine, table or variable with a description. At 100 %, every routine and every table has a name and a description. A Bronze game is below 100 %, and a Silver game is at 100 %.
Coverage counts comments. It can’t tell a right comment from a wrong one, so a comment with a mistake in it counts the same as a correct one. The second row, “Comments checked”, measures how many are right.
A wrong comment usually gets the big picture right: it knows what the routine is for. The mistake is in one detail, such as a count, a unit, or which routine calls it, and that detail sounds as believable as the rest. Each mistake below was found in a C64 listing on this site by an agent that hadn’t written it, and each has since been fixed.
the enemies run every frame.
they run once per pass of the game logic, and the game logic skips the frame where the map scrolls by a whole row. Fifteen other comments made the same mistake.
the engine’s pitch rises for the first 60 of the plane’s 160 steps.
it rises for 61.
the high bytes at $0925 are all $09.
the table starts 00 09 09 09 …, so the first one, for type 0, is $00.
a description of each weight table that fits the other one.
swap the two descriptions and both are right.
the rates are counted in frames.
the main loop moves the paths and the animation on every other frame ($68), so the rates are counted in pairs of frames. Eight of the 20 decoder comments in the sample had this mistake.
a square’s tests run when the party arrives.
two bits of the square’s flags decide. With bit 7 clear, the tests run when the party steps onto the square. With bit 6 clear, they run only when the player chooses Use. The audit rewrote 2,017 of the decoder’s 7,973 comments.
In Bubble Bobble’s three samples, every comment named the right routine, table or variable. All of the mistakes were in details. That is why mistakes like these last: a reader who knows what the routine is for sees nothing strange, and neither does the agent that wrote it.
Some comments are written by a small program rather than one at a time. A decoder, for example, writes a description of every level from one sentence pattern. When that sentence is wrong, hundreds of comments are wrong at once. So a sample treats those comments as a group of their own and always draws some of them.
A listing has hundreds or thousands of comments, and checking every one takes a long time. A sample checks a few of them and uses the result to estimate the rest:
Sixty comments can’t give an exact answer. If 2 of 60 are wrong, the whole listing might have 1 % of its comments wrong, or it might have 11 %. The calculator shows that range: how much of the whole listing is likely wrong, given what the sample found. The About tab works it out the same way.
How much of the listing is wrong. The dot is what the sample found, and the bar is the likely range (95 % sure). The dashed line is good enough: one wrong in 30.
A sample is good enough when at most 2 of its 60 comments are wrong. A bigger sample is good enough at one wrong in 30. A listing with few mistakes usually gets a good enough sample, and a first draft almost never does.
| Comments actually wrong | Chance a sample of 60 is good enough |
|---|---|
| 1 % | 98 % |
| 3 % | 73 % |
| 5 % | 42 % |
| 10 % | 5 % |
| 13 %, a typical first draft | 1 % |
This table assumes the checker finds every mistake. The section on checkers shows what happens when it misses some.
First drafts are far from good enough. On 11 October 2026, 17 games had a first sample on record, and 16 of them had more than 2 wrong in 60. The usual result was about one comment in eight. Commando’s first sample had 12 of 70 wrong, Spy vs Spy’s had 12 of 60, and Lode Runner’s had 17 of 60.
Fixing the comments a sample found wrong leaves the rest of the listing as it was. The sample looked at only 60 comments, and the others have mistakes at about the same rate. Bubble Bobble’s first sample had 11 of 60 wrong. Those were fixed, along with others like them, and the next sample still had 10 of 60 wrong.
An audit checks every comment. New checkers, which wrote none of the comments, each take one part of the listing. They use the same brief, plus the kinds of mistake the sample found, and this time they fix what they find. Then a new sample measures the result, with a new seed and a checker that took no part in the writing or the audit. Commando went through each step:
After an audit, the next sample was good enough in 4 of the 7 games that took one, and a fifth missed by one comment.
The same wrong idea often turns up in many comments: Commando’s “every frame” was in 16 of them. So when a checker finds a wrong comment, the run searches the whole listing for others that say the same thing, and fixes them all together.
An audit reads every comment again, so it takes a large part of a run. A game’s first run doesn’t do one: it takes one sample, fixes what the sample found, and publishes. Most of what a page is worth comes from explaining the whole program, and the sample tells readers how far to trust the comments. The Source tab then says the listing hasn’t had a full audit, and the run recommends one for a later run to do when someone asks.
A checker finds only the mistakes its model is able to see. On 11 October 2026, Claude models were tested on 60 comments that held 20 known mistakes. They found:
| Checker | Mistakes found, of 20 |
|---|---|
| Opus | 16 to 18, and 18 with the kit’s brief |
| Fable | about 12 |
| Sonnet | 10 to 12 |
| Haiku | 4 to 7 |
Opus raised no false alarms, and Haiku raised a few. A GPT model found three mistakes that every Claude run had missed. A comment counts as wrong when either checker’s finding holds up, so a second checker from another company is worth running when the contributor has one.
A weak checker makes a listing look better than it is, because the mistakes it misses count as right. Set how many of a listing’s comments are actually wrong, and see how many of them each checker finds in a sample of 60, and whether it then calls the listing good enough.
Each bar is how many wrong comments the checker finds in a sample of 60, on average, if it finds the middle of its range in the table above. The dashed line is good enough: 2 or fewer. The verdict is green when it is the right answer and red when it is the wrong one.
That is why the About tab names the model behind every step. Check with the strongest model you have. Even then, the real number of mistakes is probably higher than its count: in one full audit, the checkers found 6 wrong comments among 53 that a strong checker had passed in the sample just before.
Games go through different steps. One run samples twice before its audit. Another audits only the routines. A third builds a Play tab, which fixes comments along the way. So the About tab lists every step that measured the comments or made them more accurate, in the order they happened, with the models that did each one. You can see how much work went into the listing and judge it for yourself. Commando’s ledger:
How the listing was checked · Commando
| Step | By | What it found | Of the listing |
|---|---|---|---|
| Research | not recorded, claude-opus-5-5[1m] | Looked the game up online (its manual, reviews or wiki pages) before reading its code. | |
| Annotation | not recorded | Wrote the listing’s comments.first run, 52.8 % of the image; every comment later rewritten | |
| Annotation | claude-opus-5-5[1m] | Wrote the listing’s comments. 100 % of the program is described, in 861 comments.second run rewrote every comment from the bytes, to 100 % | |
| Sample 1 | claude-opus-5-5[1m] | 12 of 70 comments wrong, so between 10 and 28 % of the listing is likely wrong. All 12 were then corrected.60 hand-written and 10 map comments | |
| Audit 1 | claude-opus-5-5[1m] | Checked all 897 comments and corrected 137.four agents in disjoint ranges | |
| Sample 2 | claude-opus-5-5[1m] | 1 of 70 comments wrong, so between 0 and 8 % of the listing is likely wrong. It was then corrected.60 hand-written and 10 map comments; fifteen more with the same claim corrected with it |
“Not recorded” means nobody wrote the model down at the time. The ledger says so instead of guessing.
Bars likely wrong the sample’s range corrected right, or likely right not yet measured
Rows Green: looked up online, or a sample good enough. Red: not looked up, or a sample with too many wrong. Orange: a sample too small to say. Blue: an audit, dashed when it checked only part. Purple: a port checked against the game, the Play tab’s or a page’s. Grey: annotation, or not recorded.
Each kind of row:
Only samples and full audits change the “Comments checked” row. A Play tab’s port reads only the code it needs, and the older maintainer’s check tested the game’s notes and names, so both of them leave the row as it is.
Any AI model can run the kit, and a proven model can take a game to Silver on its own. A model is proven when the site’s maintainers have listed it as good enough, or when it took a game to Silver early on, before this check existed. When any other model wrote or checked a game’s comments, the game stays at Bronze until a maintainer has checked it. Its page is still published in full, and its banner says it is waiting for the check.
The check is needed because a weaker model’s mistakes look as confident as its correct work, and because it is a weaker checker too. When it samples its own listing, it misses more of its own mistakes: in the test above, Haiku found 4 to 7 of the 20, and Opus found 16 to 18. So a run’s own sample can make its listing look better than it is.
The maintainer, or their agent on a proven model, does the same measuring every run does:
These steps go in the ledger as ordinary sample and audit rows, with the maintainer’s name in the By column. The page then shows the maintainer’s result, good enough or not. A model becomes proven only when the maintainers add it to their list, once its work has measured well.
Until 12 October 2026, the check worked differently: a maintainer tested about 40 of the run’s claims, and one mistake failed the whole run. Every listing has some mistakes, though, even a good one. At 5 % wrong, a test of 40 claims comes back with zero mistakes about one time in seven, so the old check failed most good runs as well as the bad ones.
The “Comments checked” row sums up the ledger, using the last sample and any full audits around it. At Silver and above, the Source tab shows the same verdict in a line under its title, with a shorter sentence on why and a link to the ledger. These examples use Commando’s numbers and some made-up ones.
A listing whose first sample is good enough still says “Not audited”. The site says “Audited & low error rate” only once two things have happened: an agent that didn’t write the comments has read every one of them, and a later sample has measured the result.
Silver needs at least one sample, and the sample may show any result. The ledger shows how much checking went into the listing, and the reader judges it. Games made before 12 October 2026 can be Silver with no sample at all, and their pages show the “Not measured” line. When the ledger was first filled in, on 12 October 2026, 4 of the 38 Silver and Gold games had “Audited & low error rate”.
No single number says so. Each check answers one question and leaves others open.
| The question | What answers it | What it leaves open |
|---|---|---|
| Is every part of the program explained? | Coverage at 100 % | Whether the explanations are right |
| Can the measuring be trusted? | A proven model did it, or a maintainer did it again on one | The result, which the row shows |
| Are the listing’s comments right? | “Audited & low error rate”, from a strong checker | Which ones are wrong: even a strong checker misses some |
| How much work went into them? | The ledger on the About tab | Any step nobody recorded |
| Are the article’s claims right? | The Source tab’s notes: every claim traced to the code or seen running in the emulator | The listing’s comments, which are measured on their own |
| Does it describe the game a player knows? | Every feature confirmed, traced, or marked open | Details smaller than a feature |
| Has a person read it? | Gold: a person has been through every section | The comments, unless they read those too |
| Does it play like the original? | Platinum: the whole game ported to JavaScript and run beside the original until the two agree | Whether the comments are right |
A game is as well understood as the site can show when its coverage is 100 %, its row says “Audited & low error rate” from a strong checker, its ledger shows the work behind that, and its notes trace every claim. Even then, up to about one comment in 30 may have a wrong detail. Every page has an Edit link at the bottom, so anyone who finds one can fix it.