Context engineering is the new prompt engineering

Prompt engineering as a craft is fading, not because it stopped mattering but because the models got good enough at following clear instructions that clever phrasing buys you very little. The work that actually moves quality now is upstream: what information is in the context window when the model runs, where it came from, and how it is arranged.

This is context engineering, and on real systems it is where most of the effort goes.

The context window is a budget

Even with large context windows, more is not better. Models attend unevenly across long inputs, cost scales with tokens, and irrelevant material measurably degrades answers. Every call is a decision about what earns its place: system instructions, retrieved documents, prior turns, tool results, examples. Treat it like a budget with line items.

Retrieval is a pipeline, not a vector search

• Query rewriting so the search reflects intent, not the raw message
• Hybrid retrieval, keyword plus semantic, because pure vector search misses exact terms
• Reranking to put the few genuinely relevant chunks at the top
• Deduplication and compression so you are not spending tokens on near-copies
• Citations carried through so the answer can be checked

Order and structure matter

The same facts produce different answers depending on placement. Instructions and the most important context near the start and end, clear delimiters between sources, consistent formatting. This is unglamorous and it is worth measuring.

Agents make it harder

In an agent loop the context is rebuilt on every step and fills with tool outputs, some of them large and mostly useless after they are read. Teams are adding context management as an explicit component: summarising completed steps, dropping stale tool results, keeping a compact running state. Without it, long-running agents drift and slow down as the window fills.

How to get better at it

1. 1Log the exact context sent on every call, not just the user message
2. 2When an answer is wrong, look at the context first, the model is usually working with bad inputs
3. 3Measure retrieval quality separately from answer quality
4. 4Set a token budget per call and enforce it