A foundation model only knows two things when it answers: what it learned in training, and what is in the request in front of it. Context engineering is the practice of deciding what goes into that request, so the model has the right information, in the right amount, at the right time. This lesson explains what fills the context window, why the window is a limited resource, and how good applications manage it.
What the context window holds
The context window is the total number of tokens a model can work with in one request; each model has its own size, and on many models the response shares the same window as the input. Everything the model uses to answer must fit inside it:
- Instructions: the system prompt that sets the model's role, rules and output format.
- Retrieved data: relevant chunks pulled from documents or databases, as in Retrieval Augmented Generation (RAG).
- Conversation history: earlier turns, so the model can follow a dialogue.
- Tool results: data returned when an application or agent calls an API or runs a search.
- The user's request: the actual question or task.
Prompt engineering focuses on wording the instructions well. Context engineering is broader: it decides which of all this material to include at all.
Why the window is limited
Models are stateless. They don't remember earlier requests on their own, so anything the model should know must be sent again each time. But the window has a fixed maximum size, every token sent costs money and adds processing time, and stuffing in loosely related material can bury the important details and make answers worse. More context isn't automatically better context.
Managing context well
| Technique | What it does |
|---|---|
| Retrieve selectively | Send only the chunks relevant to this question, not whole documents |
| Summarize history | Replace long past conversations with a short summary |
| Use memory stores | Save key facts and preferences outside the model and bring back only what's needed |
| Order deliberately | Put stable instructions first and the specific request last |
| Trim tool output | Pass the model the parts of an API result it needs, not the raw dump |
On AWS, Amazon Bedrock AgentCore Memory gives agents short-term memory, which keeps track of turns within one session, and long-term memory, which extracts facts and preferences that carry across sessions. Bedrock Knowledge Bases handles selective retrieval.
Context engineering and agents
Agents make context engineering more important. An agent may run many steps, calling tools and collecting results as it goes, and each step adds to its context. Without care, the window fills with stale or irrelevant material, costs rise and the agent loses track of its goal. Well-designed agents keep their working context lean, store what they may need later in memory, and pull it back only when relevant.
On the exam
If a model "forgets" what a user said in an earlier session, the fix is a memory store, not a bigger model. If answers degrade after a long conversation or after adding lots of documents, the problem is too much or poorly chosen context, and the fix is selective retrieval or summarizing. If cost rises with conversation length, remember that history is resent on every turn.