The slick AI assistant answered every question reliably. Including the one about colleagues' salaries. This wasn't an attack, not a hack, not a flaw in the model. Someone in sales typed, out of curiosity, 'What does the team lead in purchasing actually earn?' — and the assistant replied politely, in full, and with a source reference, because the salary list sat as a PDF in the same storage as the product data sheets it was meant to access. The model did everything right. It answered a question the person asking should never have been able to ask. And that, precisely, is not a prompt problem. It's a permissions problem.
When SMEs talk about AI assistants these days, almost every conversation circles the same two questions: which model do we choose, and how do we word the prompts. Both are the visible, exciting side. The managing director has seen an assistant that summarises contracts and drafts quotes, and he wants one too. The IT lead sits beside him thinking about something else, because he knows that an assistant which accesses internal data, in the same moment, opens a new and very porous door into exactly that data. The question 'which model' is the small one. The question 'who's allowed to see what through this assistant' is the one nobody has on the slide.
Why the assistant is more dangerous than the search that came before
A classic document system was never truly watertight, but it was inconvenient. Whoever wanted the salary list had to find the folder, hold the right, open the file. The inconvenience was a quiet protection: you didn't stumble by chance onto things that were none of your business, because you had to search actively. An AI assistant flips exactly this inconvenience on its head. It's built to take the effort away. In seconds it searches everything it's allowed to access, summarises it, and phrases a smooth answer — without the person asking ever seeing a folder, opening a file, or noticing where the information actually came from.
That shifts the entire security model. Before, the barrier protected you: you had to know where something sat. Now it's enough to ask the right question in plain language — and anyone can do that, that's the whole point. The assistant turns 'theoretically accessible, practically never found' into 'served on request'. What used to slumber in the depths of a folder structure is suddenly one polite question away. Draw the assistant's access no tighter than the old storage, and you haven't kept confidentiality — you've merely removed the barrier that stood in for it.
An AI assistant in a company doesn't start with prompts. It starts with permissions.
A project where this became visible early
In one of our projects, an assistant was meant to relieve the workforce: answering questions about products, ongoing projects and internal processes, fed by whatever documents had accumulated over the years. On paper, a clear case — a searchable knowledge base with a nice interface. Around 40,000 documents sat in the shared storage, grown over some twelve years. Nobody could say off the cuff what all of it contained. That was exactly the point at which we paused, before we had so much as touched the model.
We didn't ask 'which model'. We asked 'what's actually in there, and who's allowed to see it today'. In a sample of a few hundred documents, this one storage held: salary lists from an old HR round, a solicitor's letter about a legal dispute, the costing with the real purchase prices, job applications that someone had parked there years ago and never deleted. Around eight per cent of the sample was material never meant for everyone. It sat within everyone's reach only because the folder permissions had been generously inherited over the years and nobody had ever tidied up. As long as a human was searching, it never came to light. An assistant that reads everything would have found it reliably — and handed it over on request.
Had we simply pointed the assistant at this storage, we wouldn't have built a knowledge base. We'd have built a leak. One that feels good because it's helpful, and precisely for that reason nobody would have recognised as a leak, until the first awkward answer turned up in the wrong chat. The damage wouldn't have been loud. It would have been polite, well phrased and with a source reference — and exactly for that reason hard to claw back.
We didn't cancel the project. We reversed the order. First an honest inventory of what sits in the storage and which class of confidentiality it holds. Then an access model that defines which role may see which document class. Only after that the assistant — built so that it checks the asker's permission before it draws on even a single document for an answer. The AI piece was, in the end, the smallest part. The largest part was clarifying who's allowed to see what — a question the company had been putting off for twelve years and that the assistant merely made unavoidable.