How to Stop Your AI Assistant From Oversharing: 5 Tests for 2027

Access control decides what AI can reach. What AI should say takes a different set of checks.

Written ByMalek Rabah

– Operations & Delivery Director at Optimum Partners

Publish date

November 10, 2026

Publish date

November 10, 2026

TL;DR

✓ AI can leak information without anyone breaking in: an ordinary question gets an answer that combines facts the user could see separately into something they should never know. We call this the permission gap.

✓ We check 4 points in every AI interaction, and keep settled rules in the application, where a model can’t reinterpret them.

✓ 5 tests built on 20 real requests find what a security review isn’t designed to look for.

For most of the last decade, protecting information meant controlling access: who can open the file or see the record.

Enterprise AI kept that model. The assistant inherits your permissions, retrieves only what you’re allowed to read and runs in an approved environment. Security signs off, and that’s the right foundation.

Then the assistant does what it was built to do and connects things. It reads 10 documents you’re allowed to open, finds the pattern across them and states it in one clear sentence.

Nothing was breached and no permission was violated. Yet the answer contains a fact your company never decided you should know.

We’ve built AI into our own operations, our own products and client systems in regulated industries. The most serious privacy problems we’ve dealt with came from helpful answers far more often than from attackers. With new rules on AI in hiring and employment arriving in 2027, those answers are about to get a lot more expensive.

The Permission Gap

Take an HR assistant built on company documents. A department head asks a normal question: “Summarize what’s going on with my team this quarter.”

The manager can legitimately open every source the assistant uses: leave requests, performance notes, an HR meeting summary and an approved budget line for a “role transition.” Each item on its own is routine.

The assistant, doing its job well, joins them. One engineer has taken repeated medical leave, has a pending performance review and sits under a budget line that suggests a planned exit.

The manager never searched for any of that. The answer now contains health information and an unannounced decision about a person’s job.

Access controls answer “can this person open this?” They don’t answer “should this person receive this conclusion?”

That’s why a permissions cleanup before a Copilot or ChatGPT Enterprise rollout, good advice as it is, won’t close the gap on its own. In the HR example, every permission was already correct.

40%

of organizations delayed their Copilot rollout by 3 months or more because of oversharing concerns.

Gartner survey of IT leaders, 2025

The permission gap shows up in 3 ways we see repeatedly. All 3 come from ordinary use, and none of them needs a hacker.

4 Checkpoints Where Information Moves

A security review is one gate at the front door. AI needs checks at every point where information moves or changes shape. We use 4.

The first checkpoint is where most programs stop. The second is where most real incidents begin.

Checkpoint 3 needs special attention. In our AI memory series, we argued that company knowledge is an asset worth owning.

It becomes a liability when it holds personal data you can’t find or delete. Deleting a source document doesn’t delete the summary or memory built from it.

Checkpoint 4 is where the stakes rise. Once assistants can send email or update records, a wrong answer becomes a wrong action, and a wrong action can change the business before anyone looks.

When one AI coordinates several smaller AI workers, each worker should carry only the permissions its own job needs.

How Mustang handles the 4 checkpoints

✓ The Grip, Mustang’s company knowledge layer, reads your systems live with each person’s own access and keeps no copies. Revoke access at the source and the AI loses it too.

✓ A specialist kernel screens data before it reaches any AI model, and another checks the consequences before anything changes in your systems.

✓ Each AI worker is built for one task with only what that task needs, then retired. No agent sits around holding access it no longer needs.

Put the Rules in the System, Where a Prompt Can’t Argue

The tempting fix is to tell the model: “Never reveal salary or health information.” It works in the demo, and that’s the trap.

Models misread context, follow instructions buried in a retrieved document and produce confident answers that break the rule they were given. A prompt is a request. A control is something a clever prompt can’t talk its way around.

We apply the same principle to cost. In Where Automation Beats AI, we moved permission checks and fixed routing out of the model because the business already knows those answers. Privacy rules belong there too.

“If the business has already decided something is prohibited, AI shouldn’t get a vote.”

In practice, we split the decisions like this:

The model still plays a role: explaining a restriction without revealing what it protects, asking for clarification or routing to a person. It just doesn’t get the final call.

In Mustang, a rule like “salary data stays with HR” is stored once as company knowledge, with an owner, a version and an expiry date. The kernels enforce it on every task.

Why 2027 raises the stakes

✓ January 1, 2027: California’s rules on AI in employment decisions take effect, the strictest in the US.

✓ December 2, 2027: the EU AI Act’s high-risk rules start applying to AI used in employment.

Neither automatically covers a team summary assistant. Both point the same way: regulators want to see how a decision was controlled, and a policy document can’t block an inappropriate answer.

5 Tests Before an AI Assistant Goes Live

Budget a few days with your pilot group. The people side, where users paste or ask for things without realizing the risk, is covered in our guide to preparing your team for AI.

1. Collect 20 real requests

✅ Do this: Pull pilot questions word for word, including the vague ones: “summarize,” “what’s going on with,” “compare.”

🚩 Red flag: A list of tidy test cases written by the build team. Vague questions are where the assistant connects the most dots.

2. Run each one as 3 different roles

✅ Do this: Ask every request as a frontline employee, a manager and someone from another department.

🚩 Red flag: Answers that change in substance between roles, like the manager getting the planned exit.

3. Check every answer at the 4 checkpoints

✅ Do this: Ask the 4 checkpoint questions of each answer.

🚩 Red flag: A single yes. Most will show up at checkpoint 2.

4. Mark where the protection came from

✅ Do this: For each problem caught, note whether a system control stopped it or the model simply happened to behave.

🚩 Red flag: Anything stopped only by the model. Count it as unprotected, move the rule into the system and test again.

5. Revoke, delete and ask again

✅ Do this: Revoke one user’s access to a source and delete one document. Ask the same questions.

🚩 Red flag: Old information still appearing. Retention isn’t under control.

In our experience, test 4 is the one that changes the conversation with leadership. It turns “the AI seems safe” into a clear list of what’s enforced and what isn’t.

These results shape our go-live decision more than model accuracy does. A system that answers well but can’t tell where its own boundaries are is still a pilot.

The Point Is Controlled Usefulness

We aren’t arguing for slower AI. Companies that hold back successful pilots usually do so because they can’t yet say what the system will do when circumstances change.

At TAG’s //SHIFT: AI Summit, we named ownership, error handling and human oversight as the questions that decide whether a workflow earns a permanent place. Privacy belongs on that list.

Good controls let AI move quickly where the boundaries are clear, ask for help where judgment is needed and stop where the business has already decided.

The AI you can trust is the one that knows what to keep to itself.

If you want a second set of eyes on where your AI assistant might be answering more than it should, let’s talk.

Frequently Asked Questions

Yes. It can combine permitted information into a conclusion the user was never meant to receive, such as a likely dismissal or a health issue.

Partly. A permissions cleanup controls what the AI can read. It doesn't control what the AI concludes from several documents the user is allowed to open.

Not always. Summaries, indexes, memory and logs built from the document can remain. Deletion needs to reach every place the information has traveled.

Sources

Gartner, Microsoft 365 and Copilot survey of IT leaders, 2025
California Privacy Protection Agency, CCPA regulations on automated decisionmaking technology; Littler, “California’s Long-Awaited Final Regulations on Automated Decisionmaking Create New Compliance Challenges for Employers”
White & Case, “EU AI Omnibus enters into force, amending the AI Act,” July 2026
Cloud Security Alliance, “EU AI Act’s High-Risk Deadline: Deferred, Not Cancelled,” 2026