Back to the blog AI

Good AI Fails on Bad Master Data

Thomas Grafenau25. August 202610 min read

On one project we used the best model available at the time. It then reliably found the wrong customer — because there were three of him. Once as "Müller GmbH", once as "Mueller G.m.b.H.", once as "Müller Gesellschaft mbH", all three with slightly different addresses, all three created over the years by people who did their job properly. The model didn't hallucinate. It did exactly what the data said. And the data said three customers where there should have been one.

That is the uncomfortable truth behind most disappointing AI projects in the SME world. It is almost never the model. Today's models are better than the data they are set loose on deserves. A language model turns a clean record into a clean answer and a contradictory record into a contradictory one — it just doesn't mention it. It delivers the contradiction in the same calm, confident tone as the truth. And that is exactly what makes bad master data more dangerous under AI than it was before.

What a Model Really Does with Bad Master Data

A person searching a grown customer list for "Müller" sees the three hits, pauses and thinks: hang on, that's the same one. They know the customer, they recognise the address, they know the G.m.b.H. spelling is a leftover from the old accounting system. That recognition happens on the side, unwritten, and it is the reason the list has worked for years despite its duplicates. It works because people mentally merge what the data keeps apart.

A model has none of that recognition. It sees three records and treats them as three, because there are three. It sums revenue onto the wrong one, sends the invoice to the outdated address, reports the customer as new even though they've been around for twelve years. None of this is a mistake in the classic sense. The model calculates correctly — on a foundation that was already wrong before it touched it. It didn't make the error. It merely scaled it and gave it a confident voice.

Good AI fails on bad master data, not on the model.

A Project Where This Became Visible

On one of our projects, an AI assistant was meant to help the sales team: ask about a customer, get their history, open items, recent orders, contacts. A useful, down-to-earth case, nothing spectacular. In the demo it ran cleanly — on five hand-picked customers someone had checked beforehand. Then we let it loose on the real database: around 14,000 customer and 4,200 supplier records, grown over eleven years, out of two company mergers, one ERP switch and countless manual entries.

The result was sobering and instructive at once. Among the suppliers we found around 600 duplicates — the same supplier in two, three, once even five spellings, sometimes with the legal form, sometimes without, sometimes with an umlaut, sometimes with "ae". Among the customers, a mandatory field the assistant needed for its answers was simply empty in about a fifth of the records — not wrong, empty, because the field didn't even exist before 2018. And a status field meant to distinguish "active" from "inactive" had accumulated seven different variants over the years, from "a" through "active" to "ACTIVE (see note)".

The full article is in the PDF

Practical paper · branded

AI for your company

Got a concrete case? We'll assess it for you — no sales pitch.

Briefly describe what's going on at your end. Thomas will look at whether and how the last mile to a production-ready solution is feasible for you — an honest technical assessment, not a pitch.

No newsletter, no mailing list. Just a reply to your enquiry.