How Do You Keep Customer Data Out of Your AI Automations?

How do you keep customer data out of your AI automations?

By deciding what each automation actually needs before you build it, and sending only that. Most workflows that leak personal data do it by accident, passing a whole CRM record or a full email thread when the model only needed a job title and a sentence.

The good news is that the major providers do not train on business API data by default, and both say so in writing. The bad news is that this is the question teams worry about, while the real risks sit somewhere else entirely: retention, logs, and the systems you connect the model to.

We build these automations for B2B marketing teams, which means we regularly get asked to route form fills, enrich leads and summarise calls. Here is the checklist we work through before any of that touches a model.

Do the providers train on your data?

Not by default on the business tiers, and both state it plainly. Anthropic's privacy documentation, last updated on March 16, 2026, says "We do not use API data for training (unless you have an agreement with us that states otherwise)." OpenAI's developer documentation says "data sent to the OpenAI API is not used to train or improve OpenAI models (unless you explicitly opt in to share data with us)."

Note what both sentences have in common: they are about the API. Consumer chat products have different terms, and that distinction is where most real exposure lives. An employee pasting a customer list into a personal chat account is a different situation from your automation calling an API.

So the first control is not technical. It is telling your team which surface is approved for what, and giving them an approved one so they do not improvise.

What happens to the data if it is not used for training?

It is usually retained for a period for abuse monitoring, which is a separate question from training and the one teams forget to ask. OpenAI's documentation states that abuse monitoring logs are generated for all API feature usage and retained for up to 30 days, unless longer retention is required by law.

Thirty days is not alarming. It is also not zero, and if you are handling health, financial or otherwise regulated data, the difference matters to your legal team even when it does not matter to your engineer.

OpenAI documents a Zero Data Retention control for eligible customers, and notes that eligibility varies by endpoint. Chat completions and responses are listed as eligible, while conversations and assistants endpoints are not. That detail is worth checking before you design around it.

What does the law actually require here?

For anyone handling EU personal data, Article 5 of the GDPR sets the frame, and two principles do most of the work. Data minimisation requires personal data to be "adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed." Purpose limitation requires it to be collected for specified, explicit and legitimate purposes.

Read minimisation as a design instruction rather than a legal one and it becomes useful. If the automation classifies enquiries by industry, it needs the company and the message. It does not need the name, the email address or the phone number, and sending them is both a legal risk and a design smell.

Article 5 also sets storage limitation and integrity and confidentiality, and Article 5(2) adds accountability: the controller must be responsible for, and be able to demonstrate compliance with, those principles. Demonstrate is the operative word, and it means writing things down.

Step one: write down what the automation needs

List the fields the model genuinely requires to do its job, before you write any code. Then compare that list to what your integration currently sends. On every audit we have done, the second list is longer, usually because someone passed an entire record because it was easier than selecting fields.

Be specific about why each field is there. "Company size, because the routing rule depends on it" is a justification. "It was in the payload" is not, and it is the thing you will struggle to defend later.

This list is also your documentation for the accountability requirement. It costs ten minutes and it is the artefact an auditor or a security questionnaire will actually ask for.

Step two: strip or replace what it does not need

Two techniques cover almost everything. Field selection means building the payload explicitly from named fields rather than passing an object through. Tokenisation means replacing an identifier with a placeholder before the call and swapping it back afterwards.

Tokenisation is underused and simpler than it sounds. If the model is drafting a reply, send "CONTACT_NAME" and substitute the real name into the output in your own code. The model never sees it, the draft is still personalised, and nothing about the quality changes.

Do the substitution in your own system, not in a prompt instruction. A prompt asking the model not to repeat personal data is a request, and requests are not controls.

Step three: decide where the output goes

The output is personal data too, and teams consistently forget this. A model summary of a sales call is a new record about a named person, and it inherits every obligation the original had. Where it lands, who can read it, and how long it lives are all decisions someone has to make.

Our default is that AI generated summaries go to the same place as the source record, with the same access controls, and are labelled as machine generated. The label matters because a summary read as a verbatim note is a different kind of document from a summary read as a draft.

Keeping a record of what was sent and what came back is part of this, and it doubles as your debugging trail. We covered the shape of that in keeping an audit trail for AI automations.

What about data reaching the model by accident?

This is the risk that grows as you give agents tools. A model with read access to a shared inbox, a CRM or a document store can pull in personal data you never intended to send, because it decided the data was relevant to the task.

Scope the credentials, not the prompt. Give the automation a service account with access to exactly the records it needs, and nothing else. An agent cannot leak what it cannot read, and that is the only guarantee in this whole area that actually holds.

The related failure, where untrusted content in a document redirects the automation's behaviour, is its own problem, and we wrote it up in prompt injection in real automations.

Is there a framework worth following?

The NIST AI Risk Management Framework is the most useful general one, published as version 1.0 on January 26, 2023. It is explicitly voluntary, intended to help incorporate trustworthiness considerations into the design, development, use and evaluation of AI systems.

Its four core functions are Govern, Map, Measure and Manage. For a small marketing team the value is not the full framework, it is the ordering: govern and map before you measure and manage. Decide who owns the decision and what the system touches, then worry about metrics.

If you operate in the EU there is a separate regulatory layer to think about, which we covered in what the EU AI Act means for marketing automations.

What does a reasonable setup look like for a small team?

One approved provider on a business tier. One documented list per automation of the fields it sends and why. Tokenised identifiers wherever the model does not need the real value. Scoped service accounts instead of broad API keys. And a named person who signs off before a new automation touches customer records.

That is achievable in an afternoon and it covers the great majority of real exposure. Everything beyond it, formal DPIAs, vendor security reviews, zero retention agreements, is a function of your industry and your customers' contracts rather than of the technology.

The mistake we see most often is the opposite order: a long policy document and no field level control. The document reassures the company and the payload still contains a phone number nobody needed.

Where should you start this week?

Open the single automation that touches the most customer records and read its payload. Not the code, the actual payload. Then delete every field you cannot justify in one sentence. That exercise takes about twenty minutes and usually removes most of the exposure in one pass.

Do it before you add more automations, not after. Field discipline is a habit that is easy to establish on the first workflow and painful to retrofit across fifteen.

We set these systems up alongside the websites and CRMs they plug into, so we see where the payloads come from and where they end up. If you want someone to review what your automations are actually sending, we are happy to take a look. You can find us at phoenix.studio.