Why Can Nobody Else Run the AI Automation You Built?

Why can nobody else run the AI automation you built?

Because the reasoning never left your head. The workflow diagram shows what happens, but not why the threshold is 0.7, why one step runs nightly, or what the thing did wrong in week two that forced the extra check. Without that, the next person cannot safely change anything.

We build these for clients and for our own studio, and we have watched this pattern repeat enough times to stop blaming individuals for it. The tools are genuinely fast now. Building an automation takes an afternoon. Documenting it takes longer than building it, which nobody plans for.

This is our honest account of what we write down, what we deliberately do not, and the one page that makes a handover survive.

Is this really a documentation problem?

Partly. It is also an adoption speed problem. Google's DORA research, published on 23 September 2025 and based on a survey of nearly 5,000 technology professionals, found AI adoption among software development professionals had reached 90 percent, up 14 points on the year before, with a median of two hours a day spent working with these tools.

When something gets adopted that fast, the practices around it do not keep up. Nobody wrote a runbook standard for a category of tool that barely existed two years ago. So the automations exist and the documentation does not, and that gap is structural rather than lazy.

The same research found something we think about often. It describes AI as a mirror and a multiplier, boosting efficiency in cohesive organisations and highlighting weaknesses in fragmented ones. Documentation habits are exactly the kind of weakness that gets multiplied.

What makes an AI automation harder to document than normal code?

Three things. The behaviour is probabilistic, so the same input can produce different output and your notes have to describe tendencies rather than rules. The prompt is part of the logic, but it lives in a text box rather than a file. And the failure modes are quiet, so there is no stack trace to point the next person at.

Normal software fails loudly and in one place. An AI automation drifts. It starts classifying a slightly different set of things into the wrong bucket after a model update, and the only person who would notice is the one who remembers what it used to do.

That is why the reasoning matters more here than in ordinary code. Ordinary code can be read. A prompt plus a threshold cannot be read for intent. Our piece on evaluating AI automations in production covers the measurement side of this.

What do you actually write down?

Six things, and they fit on one page. What this automation is for, in one sentence a non technical person understands. What goes in and what comes out. Every number in the system and why that number. What happens when it fails. What it is not allowed to do. And who to call.

The why behind each number is the part people skip and the part that pays. A confidence threshold of 0.7 is meaningless on its own. A note saying we set this to 0.7 because at 0.6 it was routing marketing enquiries as support tickets gives the next person permission to change it, and a way to test whether they should.

Everything else is optional. We have written elaborate documentation for automations that got deleted two months later. One good page written on day one beats a wiki nobody finishes.

Where should the prompt live?

In version control, next to the code, not in the tool's web interface. This is the single highest value change most teams can make. A prompt is logic. Logic that only exists inside a Zapier step or an n8n node has no history, no review, and no way to tell you what changed last Tuesday.

We keep the prompt and its output schema in the same file, because changing one usually means changing the other. If the automation runs in n8n, Make or Zapier, the node references the versioned text rather than holding it. That is slightly more setup and enormously less pain later.

It also gives you a place to leave comments. A prompt with three lines explaining why a particular instruction exists is worth more than a separate document describing the prompt, because it cannot drift out of sync with the thing it describes.

How do you document what the automation is not allowed to do?

Explicitly, as a short list of boundaries written in plain language. Which systems it can write to. Which it can only read. What it must never do without a person approving. What data must never leave your infrastructure. These are the rules that protect you, and they are almost never written anywhere.

We write them as refusals rather than permissions, because refusals age better. Saying this automation may never send an external email is durable. Saying it may send emails to the sales team is a rule that quietly breaks when the sales team changes tools.

Pair this with an audit trail so the boundaries are checkable rather than aspirational. If you cannot answer what did this thing do last Thursday, you do not have a boundary, you have a hope. Our article on audit trails for AI automations goes through the mechanics.

Does the trust data suggest anything about how much to document?

It suggests being honest about confidence levels, because your colleagues are already sceptical. The same DORA research found only 24 percent expressed substantial trust in AI output, with 4 percent saying a great deal and 20 percent a lot, while 30 percent had limited confidence, at 23 percent a little and 7 percent not at all.

That is a team where most people do not fully trust the thing you built. Documentation that oversells it will be discounted. Documentation that says plainly this gets the category right most of the time and here is how we check earns more credibility than a claim of accuracy you cannot support.

The same research found over 80 percent reported enhanced productivity and 59 percent noted positive effects on code quality, so the scepticism coexists with real benefit. Both things are true and your notes should reflect both.

What about the automation that only you understand?

Write the failure page first and the rest later. If you have limited time, document how it breaks rather than how it works. Working systems explain themselves through their output. Broken ones explain nothing, which is precisely when someone else has to touch them.

List the three most likely failures, what each looks like from the outside, and the first thing to check for each. That is maybe two hundred words and it is the difference between a colleague fixing something in ten minutes and escalating it to you on a Saturday.

We do this before handing anything to a client team. It is also a good design review in disguise, because writing down how something breaks tends to reveal a failure you had not handled. Our piece on building versus buying AI automation touches on when this maintenance burden should push you toward a vendor instead.

How do you keep documentation current without it becoming a chore?

Tie it to the change, not to a calendar. A monthly documentation review is a meeting that gets cancelled. A rule that a prompt change is not done until the one page reflects it is a habit that holds, because it is attached to work that was happening anyway.

Keep the page short enough that updating it is a two minute job. The longer the document, the more likely someone updates the automation and not the notes, and a document that is confidently wrong is worse than no document at all.

We also date it. A line saying last checked against real behaviour on a given date tells a reader how much to trust it, which is more useful than pretending it is always current.

Is any of this different from normal engineering practice?

Honestly, not much. This is runbooks and version control, applied to a category of system that skipped both because it felt like configuration rather than code. The discipline is old. What is new is how many people are now building production systems without an engineering background to borrow the habits from.

That is the part we find genuinely interesting. Marketing teams are shipping automations that write to production databases. They are doing useful work. They just inherited none of the practices that made that safe, and nobody handed them the short version.

So the short version is this. Prompts belong in version control. Numbers need reasons. Boundaries should be written as refusals. Failures should be documented before features. That is most of it.

What would we do first on Monday?

Pick the automation your business would most miss, and write its one page. Not all of them. One. Give it the sentence, the inputs and outputs, the numbers with reasons, the failure list, the boundaries, and a name to call. Then check whether the prompt is in version control, and move it if it is not.

That is an hour. It removes most of the risk in the system that matters most, and it gives you a template for the rest when you get to them.

If you are sitting on a set of automations that only one person can safely touch, or you want help getting them into a shape a team can own, we are happy to walk through it. Find us at phoenix.studio.