The answer that belonged to someone else

The analyst was three weeks into a provider-fraud case when she asked the knowledge assistant a routine question: had this clinic come up anywhere else? The answer came back fast, well written, and sourced. It mentioned a billing pattern, a former office manager by name, and an interview summary from a second investigation. She had never heard of that investigation, and there was a reason for that. It sat behind a need-to-know restriction, opened by a different team, and her name was not on the list. Nobody had broken in. Nobody had misconfigured a firewall. She asked a fair question and the system answered it with everything it had.

When we traced it, the cause was boring, which is the worst kind. The case management system enforced the restriction perfectly. You could not open that file without clearance. But the assistant did not read from the case management system. It read from an index built by copying the text of every case into one place so it could be searched by meaning. The words made the trip. The restriction did not. And there was a second leak hiding behind the first: a colleague with clearance had asked a near-identical question an hour earlier, and a response cache keyed on the meaning of the question had handed her his answer.

Almost every enterprise AI system I review sits on top of access controls that took years to get right. A worrying number of them then build an AI layer that copies the content out from under those controls and answers from the copy. This post is about where that happens, why the model is the wrong place to fix it, and what it takes to make an AI system respect boundaries the rest of the stack already enforces.

The AI layer flattens what the stack kept apart

Traditional systems keep data apart by keeping it in place. A record lives in a system, the system knows who may read it, and every read goes through that check. AI systems work differently. To search by meaning, you chunk content, embed it, and write it to an index. To answer quickly, you cache. To keep a conversation coherent, you store memory. To improve a model, you tune it on examples. To audit it, you keep traces. Each of these is a copy, and each copy is a fresh chance to leave the permissions behind.

So the risk in a shared AI platform is less often a dramatic breach than an ordinary convenience. The index was easier to build as one index. The cache hit rate was better when it ignored who asked. The fine-tune was cheaper on pooled data. None of those choices looks like a security decision when someone makes it, and every one of them is one.

It helps to be precise about what "tenant" means here, because it covers more than one thing. Sometimes it is a separate customer on a shared platform. Sometimes it is a business unit, a matter, a case, a patient population, or a single user whose entitlements differ from the person at the next desk. The mechanics are the same. Two parties share infrastructure, and one of them must never see what belongs to the other.

Six places the boundary slips

The index is the obvious one. A shared vector index with a metadata filter can be fine, but the order of operations matters more than people expect. If you rank the nearest matches first and filter for permissions after, you can return nothing useful to a user who is entitled to plenty, and you can leak through side effects: what ranked, how long it took, whether a result existed at all. Filtering has to happen inside retrieval, against the identity of the person being served, before anything is ranked or returned.

The cache is the one that surprised the team in that investigation. A semantic cache matches on what a question means, and two people can ask the same question with very different rights to the answer. If the cache key does not include who asked and what they were entitled to see, it becomes a very efficient way to hand one person another person's answer.

Memory is next. Agents that remember earlier turns, earlier sessions, or earlier tasks carry context forward. Memory scoped to "the agent" rather than to the person and purpose it served is a channel between people who should never share one.

Then the model weights. Tuning a model on pooled data from several tenants means each tenant's examples can resurface in another's output, and models do memorize, especially rare and specific strings like names, account numbers, and contract terms. Once something is in the weights, there is no permission check left to apply.

Then the exhaust. Logs, traces, eval sets, and the decision snapshots we argued for in the decision you couldn't run twice are all copies of sensitive material. A trace store that every engineer can query is a second, unguarded copy of everything the system ever read.

And finally the prompt itself. Shared system prompts, few-shot examples pulled from one client's real cases, tool descriptions that embed customer specifics: all of it ships in the context of every request, for every tenant.

You cannot ask a model to keep a secret it has already read

The tempting fix is an instruction. Tell the model which user it is serving and ask it to answer only from what that user may see. I understand the appeal, and it fails for the same reason prompt defenses fail generally. Anything in the context window has, for practical purposes, been disclosed to whoever is reading the output. The model can summarize it, paraphrase it, hint at it, or be talked into repeating it. We made the parallel argument about injected instructions in the instruction nobody typed: a defense written in the same channel as the attack loses. Here, the material you are trying to protect is sitting in the same channel as the request to reveal it.

So the enforcement point has to come before the context is assembled. What the model never receives, it cannot leak. That moves the work out of the prompt and into retrieval, caching, memory, and tool access, where a real permission check can say no.

It also means the check has to run against the right identity. An agent acting for a user should see what that user may see, no more, even if the agent's own service account can reach everything. Authority borrowed from a broad service credential is how a narrowly scoped person ends up with a broadly scoped answer. That is the same delegation problem we wrote about in the keys nobody signed for, applied to reading rather than acting.

Permissions have to travel with the content

The design principle that falls out of all this is simple to say and tedious to do: every copy of a piece of content carries the entitlement of the original, and every read of that copy is checked against the person being served, at the moment of the read.

In practice, when content is chunked and embedded, the chunk inherits the access rules of its source, and when the source's rules change, the chunk's change with them. A file moved under a legal hold, a case sealed, a patient's consent withdrawn: those updates have to reach the index on the same timeline as the source system, or the index becomes the place where revoked access goes on working. We made a similar point about stale content in the document you retired. Stale permissions are worse than stale facts, because a stale fact misleads one answer and a stale permission exposes everything behind it.

Caches get scoped by entitlement, not just by question. Memory gets scoped to the person and purpose, with an expiry. Traces and eval sets inherit the access rules of the data in them. And tuning on regulated data stays inside the boundary that data belongs to.

Match the strength of the wall to what is behind it

Not every boundary needs the same wall. Logical separation, one shared index with enforced filters, is reasonable for low-sensitivity content where a mistake is embarrassing but recoverable. Physical separation, separate indexes, separate keys, sometimes separate deployments, is what you want where the law or a contract draws the line: protected health information, export-controlled technical data, client privilege, sealed investigative material. The cost goes up as you move toward physical separation, and that cost buys you a boundary that does not depend on every filter in every code path being written correctly.

The mistake I see most often runs in one direction. Teams pick one isolation model for the whole platform, usually the cheapest, and apply it to data whose obligations differ by an order of magnitude. A shared filtered index for marketing collateral and export-controlled drawings is one bug away from an export violation. Treat the strength of the wall as a policy decision tied to the kind of data, and make it per data class rather than per platform.

A shared AI layer does not inherit your access controls by default. It inherits your content. Unless the permissions are carried along and checked at every read, the most capable system in the building ends up as the one place where everyone can ask about everything.

Test the wall the way an attacker would

Isolation that is never tested is a belief, and beliefs about access control tend to be wrong in ways nobody notices until a person sees something they should not. The test we like is a canary. Plant distinctive, harmless markers in one tenant's data, a made-up project name or an odd phrase that appears nowhere else, then probe from another tenant: direct questions, paraphrased questions, questions designed to warm the cache first, multi-turn conversations that try to pull through memory. If a canary ever shows up on the wrong side, you have found your leak before a customer did.

Those probes belong in the release gate, not in an annual review. Every change to retrieval, caching, memory, or the model is a change to the wall, and should clear cross-tenant cases before it ships, the same discipline we described in the deploy that looked like an edit. Each leak you ever find becomes a permanent test case.

Why isolation lives in the substrate

Every solution in our portfolio handles data that belongs to someone specific. Throughline works across cases where need-to-know is the whole point. NeuraMed and Praman-Anvesh carry patient information under HIPAA. MatryxAI sees supplier pricing that one buyer must never see from another. Clairant handles claimant files. Lumen answers questions across a corpus where privilege and ethical walls decide who may read what. If each of them implemented isolation its own way, we would be counting on six teams to get the same hard thing right, every release, in every component that copies data.

So it sits underneath them. In our architecture, Neura-Cortex carries the identity of the person being served through every retrieval, cache lookup, memory read, and tool call, applies the policy packs that define what that identity may see for HIPAA, ITAR, and financial personal data, and records in the audit log what each answer was entitled to draw on. A solution inherits the wall by running on the platform. That is the argument we made in the floor under the stack: the parts that have to be right everywhere should be built once.

The analyst did the right thing. She stopped, flagged it, and did not use what she had read. The team fixed the index so restrictions traveled with the text, scoped the cache to entitlement, and added her question to the release gate as a permanent canary. It could have gone differently with a less careful person, or a defendant's lawyer reading the disclosure log. When we review an AI system now, the question we put to every component that holds a copy of anything is a short one. Who was this computed for, and is that the person asking?

---

MTekLabs designs, deploys, and governs production-grade agentic AI platforms for government and commercial enterprises. Our solutions are built human-in-the-decision-loop where it matters and auditable by design. Explore them at mteklabs.com.