Is Context Engineering Replacing Prompt Engineering?

Is Context Engineering Replacing Prompt Engineering?

It is absorbing it rather than replacing it. Prompt engineering is about writing good instructions. Context engineering is about deciding what the model sees at all, across a whole run. Once your automation loops, calls tools, and reads documents, the wording of the prompt stops being the thing that decides whether it works.

We have watched this shift change how our own AI work fails. Two years ago a broken automation usually meant a badly worded instruction. Now it usually means the model was handed too much, or the wrong slice, of something true.

That is a different discipline, and it is worth naming properly rather than treating as prompt engineering with extra steps.

What Does Context Engineering Actually Mean?

Anthropic's engineering team put a usable definition on it in September 2025, describing context engineering as the set of strategies for curating and maintaining the optimal set of tokens during model inference. In the same piece they define prompt engineering more narrowly, as writing and organising instructions for optimal outcomes.

The distinction is scope. A prompt is one message. Context is everything in the window: the system instructions, the tool definitions, the documents you retrieved, the results of the last nine tool calls, and the conversation so far.

Their framing of why this matters is the part we quote most often to clients. An agent running in a loop generates more and more data, and that data competes for a fixed window. Someone has to decide what stays. If nobody decides, the loop decides, and it decides badly.

Why Did the Problem Change?

Because the workloads changed. A single-turn task, such as rewriting a paragraph or classifying a support ticket, fits comfortably in a window with room to spare. Instruction quality dominates because context scarcity is not a constraint.

An agent that reads a CMS, checks three APIs, writes a draft and validates it against a checklist is a different animal. Every step adds tokens, and most of those tokens are stale by the time the next step runs. The work is now curation.

The industry response to this was initially to buy bigger windows, which felt like it should solve the problem. It did not, and the reason is measurable.

Is There Evidence That Long Context Actually Hurts?

Yes, and it is more specific than the folk wisdom suggests. Chroma published a study in July 2025 by Kelly Hong, Anton Troynikov and Jeff Huber testing 18 models, including the GPT-4.1, Claude 4, Gemini 2.5 and Qwen3 families, on tasks where input length was the only variable being changed.

Their headline finding was that models do not use their context uniformly, and that performance grows increasingly unreliable as input length grows. Even a single distractor reduced performance against a clean baseline. On conversational tasks, focused prompts significantly outperformed full prompts across every model tested.

One result is genuinely counterintuitive and worth sitting with: models performed better on shuffled haystacks than on logically structured ones. Whatever intuition you have about tidy, well-organised context helping the model, the measurement does not support it in that setup.

Anthropic describes the same underlying phenomenon as context rot, noting that as the number of tokens in the window increases, the model's ability to accurately recall information from that context decreases. They tie it to the transformer's pairwise token relationships stretching thin, and frame it as an attention budget that every added token draws down.

What Does Good Context Curation Look Like in Practice?

Four moves, all of which come down to putting less in front of the model on purpose. Anthropic names them clearly, and they match what we end up doing in production.

Compaction means summarising the conversation as it approaches the limit and restarting from the compressed version, while deliberately preserving the decisions and details that would be expensive to lose. Structured note-taking means letting the agent keep persistent notes in an external file it can reload later, so continuity does not depend on everything staying in the window.

Just-in-time retrieval means holding lightweight references rather than preloading the documents, and fetching content at runtime when a step actually needs it. In web work this maps neatly onto not dumping your entire CMS into a prompt, which is the single most common mistake we see.

The fourth is architectural, and it deserves its own answer.

Should You Split Work Across Sub-Agents?

Often, yes, and the reason is context hygiene rather than cleverness. A specialised agent handling one focused task starts with a clean window, does its work, and returns a distilled summary. Anthropic puts that summary in the range of one to two thousand tokens returning to the coordinating agent.

That number is the point. A research step that would have filled the main window with forty pages of retrieved text instead contributes a couple of thousand tokens of conclusions. The coordinator stays sharp because it never saw the mess.

The trade is real though. More agents means more moving parts, more failure modes, and more places for a handoff to lose something that mattered. We have argued the general case in whether you need many AI agents or one good one, and context pressure is the strongest argument for splitting, not organisational tidiness.

Does This Mean Prompt Wording No Longer Matters?

No, and the framing that it does is how teams end up with an elegant architecture that produces vague output. Instructions still decide the shape of the answer, the tone, the format, and what the model does when it is unsure.

What changed is where the leverage sits. On a single-turn task, rewriting the instruction is the highest-value thing you can do. On a long agentic run, the instruction is probably fine and the window is full of things that should not be in it. Spending a day rewording a system prompt while a stale tool result poisons every turn is a common and expensive mistake.

Our general guidance on writing the instructions themselves still holds, and we covered it in prompt engineering for web work. Treat that as the foundation and context engineering as the layer above it.

How Does This Change What a Web Team Should Build?

It changes what you build around the model more than what you ask it. The valuable engineering moves to retrieval that returns small, relevant slices, to tools that return summaries rather than raw payloads, and to memory that persists outside the window.

Concretely, that means a tool which returns the three fields the agent needs rather than the whole CMS item. It means chunking documents so a retrieval step can pull a section instead of a manual. It means an explicit decision about what gets carried forward at every handoff, written down, rather than whatever happened to still be in the transcript.

It also raises the value of clean, well-structured source content, which is the same work that makes pages easy for answer engines to read. The overlap between building for agents and building for AI search is larger than it looks.

How Do You Know If Your Context Is the Problem?

Look for the pattern rather than the symptom. Context problems have a signature: the automation works on short inputs and degrades on long ones, it forgets instructions given early in a run, it repeats work it already did, or its quality falls off as a session goes on rather than failing outright.

Instruction problems look different. They fail consistently, from the first turn, in the same way every time. If your automation is wrong on turn one, that is a prompt. If it is right on turn one and confused by turn twelve, that is context.

The cheapest diagnostic is to log what the model actually received at each step, not what you think you sent. We have lost count of the times the real window contained something nobody meant to be there. Memory design is a related piece of this, and we went into it in what it means to give an AI agent memory.

Where Does This Land?

Our read is that context engineering is now the main skill for anyone shipping agentic work, and that the name will probably stop being used within a couple of years because it will just be called building the system. That is what happened to prompt engineering.

What will not change is the underlying constraint. Attention is finite, longer input is measurably less reliable, and something has to decide what the model sees. Whether that decision is made deliberately by your team or accidentally by your loop is the whole difference between an automation you can trust and one you have to babysit.

If you are building something agentic and it works in testing but drifts in production, that is a conversation we have often and are happy to have with you. Come and find us at phoenix.studio.