What Is Context Engineering? The Complete Reference

Prompt engineering is dead. The era of trying to trick a model into behaving correctly by appending “take a deep breath and think step-by-step” to the end of a query is over. In production environments, enterprise teams do not write prompts; they engineer context.

Context Engineering is the systematic process of curating, structuring, and injecting data into an LLM’s context window to constrain its behavior and guarantee reliable, deterministic outputs.

While prompt engineering focuses on the linguistic phrasing of instructions, context engineering focuses on data architecture. It treats the LLM’s context window as a highly volatile, ephemeral database that must be precisely populated at runtime.

The Architecture of the Context Window

To understand context engineering, you must treat the LLM context window as compute memory (RAM). When an agent receives a query, the context window starts empty. The context engineer’s job is to build a pipeline that loads the exact state required to solve the problem before the inference step begins.

A production-grade context payload typically consists of four layers, strictly segregated using XML or JSON formatting:

1. System Instructions (The Bootloader)

This is the immutable core of the context. It defines the model’s identity, its operating constraints, and its output schema. It does not contain dynamic data. Example: “You are a database router. You will receive a user query. You must output a JSON object containing the target database name and nothing else.”

2. Episodic Memory (The State)

LLMs are stateless. To maintain a conversation or multi-step execution graph, the context pipeline must fetch previous interactions and inject them. Context engineers implement sliding windows or token-aware summarization algorithms to ensure episodic memory does not overflow the context limit.

3. Retrieved Knowledge (The RAG Layer)

This is the data the LLM needs to answer the specific query. Context engineering dictates how this data is structured. Instead of dumping raw PDFs into the prompt, the text is extracted, chunked, embedded, retrieved via vector search, and injected with explicit source citations.

4. Untrusted Input (The User Payload)

This is the actual user query. Because this data is untrusted, context engineers isolate it within strict delimiters (e.g., <user_input>) to mitigate prompt injection attacks. The system instructions explicitly command the model to treat anything inside these delimiters as passive data, not executable instructions.

Context Engineering vs. Prompt Engineering

The distinction between the two disciplines becomes clear when you look at how they solve problems.

If an LLM hallucinates an answer to a customer’s question:

• A Prompt Engineer will rewrite the prompt: “You are an expert support agent. Please be very careful and do not lie.”
• A Context Engineer will build a retrieval pipeline (RAG) that fetches the official documentation, injects it into the context window, and sets a strict system constraint: “Answer using strictly the documents provided in <kb_articles>. If the answer is not present, output exactly: UNABLE_TO_ANSWER.”

Prompt engineering relies on the model’s parametric memory (its training weights). Context engineering explicitly overrides parametric memory with deterministic, retrieved data.

Frequently Asked Questions

Is RAG a type of Context Engineering?

Yes. Retrieval-Augmented Generation (RAG) is the most common and powerful Context Engineering pattern. RAG handles the automated retrieval of the data, while Context Engineering dictates how that data is structured and presented to the model.

Why use XML tags for context structuring?

Modern models (particularly the Claude and Llama families) are explicitly instruction-tuned to recognize and isolate XML tags. Using <context> or <system_prompt> tags allows the attention mechanism of the transformer to clearly separate instructions from data, drastically reducing hallucination and prompt injection.

How do I handle context window limits?

As context windows expand (approaching 1M+ tokens), the challenge shifts from fitting the data in to maintaining attention. Context engineers handle this by strictly filtering retrieved documents, using cross-encoders to rerank RAG results, and dynamically summarizing episodic memory before injection.