A routine check of whether the methodology page still matched actual practice turned up something more serious than a wording gap: Citigroup, one of the ten institutions in this pilot since launch and the illustrative example used in this site's own methodology text, had three of its four "Confirmed" model-family relationships sourced from an article that doesn't support them.
What was wrong
Citigroup's Google (Gemini), Microsoft (Azure OpenAI), and OpenAI (GPT) rows all cited the same American Banker article, headlined "Wells Fargo, Citi and Goldman lead in AI venture investment." Reading the actual article: it covers Citi's venture-capital bets in AI startups (Lakera, Norm AI, Jump AI, Prophet Security) — none of them Google, Microsoft, or OpenAI — and says nothing about Citi deploying Gemini, Azure OpenAI, or ChatGPT operationally. Three "Confirmed" relationships, sourced to an article about a different thing entirely. Only Citigroup's fourth relationship, Anthropic (Claude), held up: a Fortune article on Claude for Financial Services explicitly names Citi, alongside JPMorgan Chase, Goldman Sachs, AIG, and Visa, as running Claude in production.
The three unsupported rows are now deleted. Citigroup remains in the sample with its one real, sourced relationship.
A knock-on effect: two "first" claims were wrong too
Because Citigroup's four families were treated as real, two later posts on this blog made comparisons against them that don't hold up — and one was wrong independently of the Citigroup error entirely. The August 30 post called Allianz SE "the first institution in the sample with three simultaneously confirmed model families." That was never true: Raymond James Financial, Citizens Financial Group, and MUFG had already reached three confirmed families over a week earlier, in the international-expansion update. That post has been corrected to describe Allianz as joining that group, not leading it. Separately, September 1's claim that Banco Santander was the sample's first four-family institution turns out to be accurate now that Citigroup's inflated count is gone — Santander is, genuinely, the only one.
The methodology gap this exposed
Independently of the Citigroup error, this check found that the methodology page didn't actually document three exclusion rules this project has been applying for weeks — rules that were only ever written down in a blog post about what we don't count: an investment stake in a model lab isn't the same as using its models; a cloud-infrastructure case study that never names a specific AI product doesn't count; and exploratory language ("exploring," "considering") doesn't clear the bar for Confirmed or even Probable. A visitor reading only the methodology page — the page every blog post points to as "what counts as evidence" — would never have learned these rules. They're now on that page directly, in the confidence-levels section, alongside a corrected example (Raymond James Financial, not Citigroup) and an updated honesty note that no longer hardcodes an institution count that goes stale every time the sample grows.
Why this matters more than a typo
This project's entire pitch to regulators, reinsurers, and journalists is that its numbers are more trustworthy than the alternative because every row traces to a checkable source. A flagship example institution failing that exact test — for weeks, in the site's own reference documentation — is the kind of error that should be found by us, not by someone deciding whether to trust this data for underwriting. It was found by re-reading our own methodology against our own practice, which is now something we intend to do on a standing basis rather than only when asked.