Agent Memory: In-Context, External, Procedural
How agents remember what they have done and what they know across multiple tasks.
Agents need memory to stay useful across steps. In-context memory is what sits in the current prompt. External memory is persisted data in a store, fetched only when relevant. Procedural memory is task policy and learned workflow structure, like system instructions and tool-use rules.
Here is the failure mode in concrete terms: a standard chat completion API is stateless, so every single turn re-sends the entire conversation history. If you never summarize or externalize anything, the prompt you pay for at turn 20 contains all 20 turns, and you paid for the growing prefix on every one of those 20 calls, not just the last one. Try it below.
Every chat-completion API call is stateless: the full conversation is re-sent on every turn. Drag the slider to see what that does to your prompt size and bill, with and without external memory.
The fix is not 'use a smaller model,' it is architectural: keep only the last N turns verbatim, and replace anything older with a periodically-updated summary of fixed size. That turns quadratic cost growth into linear cost growth, and is exactly what external memory buys you.
In-context memory is your short-term brain. External memory is your notebook. Procedural memory is your playbook. Strong agents use all three intentionally.