Prompt Engineering Didn’t Die. It Got a Bigger Job: Prompt, Context, Orchestration

Every few months someone announces that prompt engineering is dead. Then they post a screenshot of an agent doing something impressive, which was produced by — and I cannot stress this enough — a very carefully written prompt. The correction I keep making is this: context engineering didn’t replace prompt engineering. It absorbed it. These aren’t three eras on a timeline. They’re three layers that run at the same time, one nested inside the next, and most agent failures in 2026 happen at the layer nobody is looking at.

What I’ll Cover in This Blog

✔️ Why the three layers are nested, not sequential
✔️ What is actually inside your context window on every single turn
✔️ Context rot — the failure mode that gets worse as you add more information
✔️ The three fixes: compaction, structured notes, and sub-agents
✔️ How to diagnose which layer your agent is failing at, in one pass

Now, let’s dive in. 🔥

The Scenario

Brightpath Legal Ops built an agent to answer contract questions for their commercial team — “what’s our termination notice period with this vendor?” It demoed beautifully. In production it started confidently citing the wrong clause about one time in six.

The team’s first instinct was the model. They upgraded it. Same error rate. Then they rewrote the prompt — three times, each version longer than the last. Slightly worse, actually.

The real cause was two layers up: the retrieval step was returning the superseded version of the master agreement alongside the current one, the conversation history had grown long enough that early instructions were being crowded out, and nothing in the assembled window told the model which document won. The model was doing exactly what it was asked, with what it was given. It wasn’t a model failure. It was a context failure — and no prompt rewrite fixes that, because the prompt was never the broken part.

Layer 1 — Prompt Engineering: Smaller Job, Higher Stakes

Prompt engineering is tuning the instructions and examples: what the model should do, in what voice, in what format, with what refusals. What changed isn’t its importance — it’s its share of the window. In a 2023 chatbot, your prompt was nearly everything the model saw. In a 2026 agent, your prompt might be 5% of the tokens, sitting alongside retrieved documents, tool definitions, memory and a long conversation history.

That makes precision more valuable, not less. Three things I insist on in any production prompt:

🔹 An explicit output contract. Not “be helpful” — the exact shape of a valid answer, including what to do when the answer isn’t available. Ambiguity here shows up later as parsing errors and made-up fields.
🔹 A refusal path. “If the retrieved documents don’t contain the answer, say so and name what’s missing.” Most hallucinations I investigate are really unspecified refusal behaviour.
🔹 Examples that encode edge cases, not happy paths. One good adversarial example is worth ten polite ones.

Layer 2 — Context Engineering: The Whole Window, Every Turn

Context engineering is the practice of deliberately designing everything the model sees on each inference call. Anthropic frames it as the natural evolution of prompt engineering — and the operative word in the definition is everything: system prompt, user input, retrieved documents, conversation history, tool definitions, and long-term memory, assembled fresh for every turn.

Two consequences that catch teams out:

🔹 It’s re-done on every turn. Your context isn’t a setup step; it’s a per-call assembly job. Whatever logic decides “what goes in the window right now” is your product.
🔹 Tokens are a budget, not a bucket. A big context window is an invitation to over-fill it, and over-filling is exactly what degrades quality. The goal is the smallest set of high-signal tokens that makes the right answer likely — not the largest set of tokens that might contain it.

Context Rot: Why “Just Add More Context” Backfires

Here’s the counter-intuitive part, and it’s the reason long-context models didn’t end this discipline. As the number of tokens in the window grows, a model’s ability to accurately use any particular piece of that context declines. Anthropic calls this context rot, and it’s why a bigger window is a bigger opportunity to dilute your signal, not a licence to dump everything in.

Practically, rot shows up as three symptoms I now recognise instantly:

🔹 Instruction drift. The agent followed your format for six turns and then stopped. The instructions are still technically in the window — they’re just buried.
🔹 Recency bias. It answers from the last thing it saw rather than the most authoritative thing it saw.
🔹 Conflict blending. Given two documents that disagree, it produces one fluent answer containing bits of both. This is the Brightpath failure exactly, and it’s the most dangerous one because it looks like an answer, not an error.

The three fixes, in the order I reach for them

1️⃣ Compaction. When a conversation nears the window limit, summarize it and restart with the summary — preserving decisions and constraints, discarding redundant tool output. Anthropic describes exactly this pattern in Claude Code: pass the history back to the model, keep the critical details, continue with compressed context.

2️⃣ Structured note-taking. Instead of carrying everything in the conversation, have the agent write durable notes outside the window — a running decision log, an entity list, a task file — and re-read only what the current step needs. Memory becomes an artifact you can inspect, not a hope.

3️⃣ Sub-agents. When one window genuinely cannot hold the job, split it: each sub-agent gets a clean window and a narrow task, and only their results come back to the coordinator. This is where you cross out of context engineering and into the third layer.

Layer 3 — Agent Orchestration: Many Windows, One Outcome

Orchestration is what you do when the work doesn’t fit one context window, or when it benefits from independent perspectives. It’s the layer of parallel sub-agents, sequencing, retries, verification passes and merging.

The engineering discipline here is not “spawn more agents.” It’s:

🔹 Decide the split deliberately — by subsystem, by document set, by dimension of review. Overlapping agents produce redundant tokens and contradictory conclusions.
🔹 Keep sub-agent outputs small and typed. A sub-agent that returns three pages of prose has just moved the rot problem up a level. Return structured results.
🔹 Verify before you merge. An independent checking pass beats one long confident run, especially for anything that will be acted on. I made this case at length in How to Give Your AI a Spine.
🔹 Budget the whole tree. Cost and latency compound across a fan-out. Know your ceiling before you run it, not after.

How to Diagnose Which Layer Failed

This is the practical payoff of thinking in layers. When an agent gets something wrong, run this triage before touching anything:

The habit that makes this work: log the fully assembled context for failing turns and read it with your own eyes. Almost every team I’ve done this with has the same reaction within ten minutes — “oh, that’s what it was looking at.” You cannot debug a window you’ve never seen.

Why This Is the Moat

Anyone can write a prompt. Prompts are also trivially copyable — if your entire advantage is a clever system message, your advantage is one screenshot away from being everyone’s.

What isn’t copyable is the assembly: which of your documents get retrieved, how your entities are resolved, what your memory keeps, which of your tools are exposed to whom, and how the whole thing is split across agents when it gets big. That’s engineering built on top of your data and your permissions — and it’s the reason two companies using the identical model get wildly different results.

Which is also why this layer sits directly on top of the data foundation. Context engineering with no unified, current, permissioned data underneath is just an elaborate way to retrieve the wrong thing quickly — the argument I made in Data First, AI Second.

Common Pitfalls

✔️ Answering a context failure with a prompt rewrite. Longer prompts are the aspirin of AI engineering: occasionally helpful, rarely curative.
✔️ Filling the window because it’s big. More context measurably degrades recall past a point. Curate.
✔️ Exposing every tool to every agent. Tool definitions consume tokens and attention on every single call. Expose what the task needs.
✔️ Unbounded conversation history. Without compaction or notes, every long session eventually drifts.
✔️ Sub-agents that return prose. Structured, typed results — or you’ve just relocated the problem.
✔️ No eval set. You can’t tell whether a context change helped without a fixed set of questions with known answers.

Conclusion

Prompt engineering didn’t die. It got promoted into a bigger system with harder problems.

✔️ Three nested layers — prompt inside context inside orchestration; none of them retired.
✔️ Context is re-assembled every turn — the assembly logic is your product.
✔️ Context rot is real — more tokens can mean worse recall, so curate for signal.
✔️ Compaction, structured notes, sub-agents — the three fixes, in that order.
✔️ Triage before you tinker — was it in the window, were the instructions followed, did the job fit?
✔️ The assembly is the moat — prompts are copyable, your context pipeline isn’t.

Most agent failures I’m called in to fix aren’t model failures. They’re context failures wearing a model failure’s clothes.

Connect with me on LinkedIn →

Got an agent that demos well and drifts in production? Send me the failing case — nine times out of ten we find it in the assembled window, not the model.

Which layer bites you most often — retrieval, prompt, or orchestration? Tell me in the comments.

About the Author — Abubakar Asif