We fact-checked 450 claims in AI-generated code wikis. 1 in 5 diagram arrows was wrong.

Statements checked

450

Diagram arrows wrong

15/75

Sentences wrong

4/229

Angada wrong

1/200

DeepWiki wrong

22/200

AI tools now write wikis for code: pages that explain how a project works, often with diagrams. We wanted to know how often those pages are wrong.

We picked 450 statements at random from nine public AI wikis. Three tools wrote them: Angada (ours), DeepWiki and cubic. The wikis cover four open-source projects: React, click, ripgrep and Cobra. We checked every statement against the project's code. Three AI judges graded each one without knowing which tool wrote it. Every "wrong" verdict points to the lines of code that prove it.

The text was mostly right. The diagrams were not.

A sentence in a wiki tells you what a function does. An arrow in a diagram tells you how two parts connect: this one calls that one, or this step runs first. The arrows were wrong far more often.

• Diagram arrows and steps15 / 7520.0%
• Names of functions, fields and flags3 / 595.1%
• Project overview1 / 283.6%
• Where things live in the code2 / 593.4%
• Sentences about what code does4 / 2291.7%

All three tools together, 450 statements. Each statement was checked against the version of the code its wiki was built from.

About 1 in 5 diagram arrows did not match the code. Sentences about what the code does were wrong in only 4 of 229 cases. This was true for all three tools: in each one, the diagrams were wrong more often than the rest of the page.

One of the wrong arrows was ours

Angada's React DevTools page drew an arrow from handleBridgeConnectionMessage to renderOpenedDevToolsPanels. That call does not happen. Here is the first function:

function handleBridgeConnectionMessage(message: mixed): void {
  switch (getExtensionBridgeConnectionType(message)) {
    case EXTENSION_BRIDGE_CONNECTION_READY:
      isBridgeConnected = true;
      flushPendingBridgeMessages();
      break;
    case EXTENSION_BRIDGE_CONNECTION_DISCONNECTED:
      isBridgeConnected = false;
      break;
  }
}

It sets a flag and sends the messages that were waiting. It does not draw any panels. A different function, mountReactDevTools, does that. All three judges agreed. It was the only wrong statement out of Angada's 200.

Why did it get through? Before a page goes live, Angada checks that every sentence points to real lines of code. For diagrams, it checked the boxes but not the arrows. Both functions are real and are in the same file, so the boxes passed, and the invented arrow passed with them.

What wrong looks like

All three judges marked these four statements as wrong. Contradicted means the code says the opposite. Unsupported means nothing in the code backs it up. Each card links to the wiki page and to the code. We have shortened the wording of each statement.

What the code really says:

• Cobra. It is the other way round. Cobra finds the command and reads the flags first. The hooks run after that.
• React. The build replaces only the first two. __VARIANT__ is used only when the tests run.
• click. The project deleted the requirements/ folder before the version the wiki says it was built from. Its dependencies are now in pyproject.toml.
• ripgrep. --color and --line-number are in Output, but --json is in a different group, OutputModes.

These are 4 of the 25 wrong statements. See all 25, with the code for each.

How the three tools compare

We build Angada and we ran this study. Keep that in mind when you read this chart.

• Angada1 / 2000.5%
• DeepWiki22 / 20011.0%
• cubic (React only)2 / 504.0%

200 statements each for Angada and DeepWiki, from four projects. 50 for cubic, which only has a wiki for React.

cubic's number is less certain than the others, because it is based on 50 statements from one project.

Angada uses Claude models to check its own pages, and two of our judges were Claude models. So we added a judge from a different company, OpenAI's GPT, which graded all 450 statements on its own. It was a little stricter, but the result was the same: Angada 2 of 200 wrong, DeepWiki 30 of 200, cubic 4 of 50. GPT and Claude Opus agreed on 97% of the statements.

The gap is not only about diagrams. If you remove the diagrams, Angada has 0 wrong out of 190, DeepWiki 9 out of 140, and cubic 1 out of 45.

Wikis go out of date

Code changes all the time. If a wiki is not updated, it slowly becomes wrong.

cubic's React wiki is from August 2025. React has had 944 more commits since then. Against the code from that date, 2 of its 50 statements were wrong. Against today's code, 5 were wrong. For example, it lists error levels that React's compiler has since replaced (then, now).

DeepWiki's click wiki has a different problem. It says it was built from commit cdab890 in February 2026. But it describes files that the project had already deleted in an earlier commit. Seven of its eight wrong click statements are about those deleted files.

What to do about it

If your team reads AI-written docs:

1. Check the arrows first. Diagrams were 75 of our 450 statements, but 15 of the 25 wrong ones. If you have 30 minutes to review a page, spend them on the diagram.
2. Ask for proof for every arrow. An arrow should link to the line of code where the call happens. If it does not, treat it as a guess.
3. Treat the wiki like code. Give it an owner. Show which version of the code each page was built from, so that people can review the changes.
4. Check a random sample, not the top of the page. The first statements on a page are usually the easy ones. Our own first try made this mistake (see below).

What we fixed

We fixed the DevTools diagram on the same day. It now shows the call that the function really makes, flushPendingBridgeMessages. We also added three missing diagrams and removed links to pages that did not exist yet.

More importantly, Angada now checks the arrows too. If an arrow says that one function calls another, and the code does not make that call, the page does not go live.

Limits

• We are not neutral. We build Angada, and Angada came out on top. That is why we used a judge from another company and published all our data. We would like someone else to run this study again.
• cubic was only checked on React, with 50 statements. Its number does not tell you about cubic in general.
• 50 statements per wiki is enough to tell Angada and DeepWiki apart. It is not enough to compare the four projects with each other.
• We checked accuracy, not completeness. Angada had other problems that this study does not count: four linked pages that were never published, and three missing diagrams.
• Close calls went to the wiki. When the judges did not agree, the majority won, and a tie counted as correct.
• Judges did not see tool names, but the version of the code they checked against can hint at the tool.

How we did it

We wrote the rules before we started. We then picked 50 statements per wiki at random, with a seed that we published. A statement is one fact that you can test against the code, for example "X calls Y". We checked each one against the version of the code that its wiki was built from.

Three AI judges graded every statement, and none of them saw which tool wrote it. Claude Opus graded all 450. Claude Sonnet checked every "wrong" verdict again, plus a random 10% of the others. OpenAI's GPT graded all 450 on its own.

Our first try took statements from the top of each page only. Every tool got about 3% wrong, because the top of a page holds the easy facts. We did not use that result.

Explore all 450 statements: every statement, each judge's verdict and the code that decides it, filtered by tool, project and verdict. The page also links the full dataset, with the seed and the scripts, so that you can run the study again.

If you find a mistake in our work, tell us at hello@angada.ai.