Agent Memory, Explained From Zero

The assistants that feel genuinely helpful seem to remember you — your name, your preferences, what you worked on yesterday. Yet the model underneath forgets everything the instant a chat ends. This final topic resolves that apparent contradiction, and in doing so it ties the whole course together. It starts from nothing and builds up how memory is really made.

1 · The model forgets the moment you leave

A fact that surprises people: the model itself has no memory. It doesn't remember you between conversations, and within one conversation it only “remembers” because the app keeps re-sending the history each turn. The instant a chat ends and the window is cleared, everything is gone. Start a new conversation tomorrow and, as far as the model is concerned, you're a complete stranger — it won't recall your name, preferences, yesterday's work, or any decision you reached together. That's because, as the last topic showed, the model only knows what's on its desk right now, and a fresh conversation is a fresh, empty desk. So how do assistants seem to remember across sessions? The answer is that memory isn't a property of the model at all — it's something the application builds around it.

2 · Memory lives outside the model

The trick is simple once you see it. The application keeps a separate store — a database, a file, some persistent place — where it writes down things worth remembering: your name, your preferences, key facts from past conversations. This store lives outside the model and survives when chats end. Then, at the start of a new conversation, the app reads the relevant remembered facts and places them onto the desk — into the context window — before the model even sees your first message. So the model still has no memory of its own; it just finds the important facts already laid out in front of it. It's like a doctor who doesn't personally remember every patient but reads your chart before you walk in, so the visit feels continuous. Agent memory is exactly that chart: the app writes to it, reads from it, and loads it into context — giving a forgetful model the appearance of a long, continuous memory.

3 · Two kinds: short-term and long-term

It helps to split memory into two kinds, because they solve different problems. Short-term memory is just the current conversation — the running history the app re-sends each turn so the model stays coherent within this chat. We met its limits last topic: it fills the desk and eventually needs summarising, and it vanishes when the chat ends. Long-term memory is the persistent store that survives across conversations — the saved facts that let the assistant recognise you next week. The interesting part is the relationship: often, when a conversation ends or a fact seems important, the app promotes something from short-term into long-term — noticing “the user prefers metric units” and writing it to the lasting store. So short-term is the working desk of the moment, and long-term is the filing cabinet that outlives any single session. Real assistants use both: the desk for now, the cabinet for always.

4 · Deciding what is worth remembering

A store that remembers everything is as useless as one that remembers nothing — you'd bury the useful facts in trivia and, as we learned, crowd the desk with noise when they're loaded back. So a real skill is deciding what's worth keeping. Good candidates are durable facts: your name, your role, your stated preferences, important decisions, ongoing projects — things likely to matter again. Poor candidates are disposable: passing small talk, a one-off question already answered, the weather you asked about once. Some systems decide this with simple rules, some let the model judge what seems worth saving, and many let you explicitly say “remember this.” But the principle is constant: memory should be curated, not a firehose. This is the context-engineering lesson one level up — be deliberate about what earns a place, because everything you remember eventually competes for room on that finite desk.

5 · Retrieving the right memory at the right time

Once you've stored many memories, a new problem appears: you can't load them all back — there are too many, and most are irrelevant to the current question. You need to fetch just the memories that matter right now. If that sounds familiar, it should — it's exactly the retrieval problem from RAG. In fact, long-term memory is often built with the very same machinery: memories are embedded and stored in a vector database, and when you ask something, the system retrieves the memories most relevant to your current message and loads only those into context. So “remember our discussion about the budget” pulls up the budget memories and leaves the rest filed away. This is a lovely payoff of the course building on itself: agent memory is largely retrieval plus context engineering, pointed at the assistant's own accumulated notes instead of at documents. The pieces you've learned combine to solve a brand-new problem.

6 · Memory you can see and correct

Memory raises trust and privacy questions you should design for from the start. If an assistant remembers things about a person, that person deserves to know what it remembers and to correct or delete it. A good memory system is transparent: it can show the user their remembered facts and let them say “that's wrong, forget it” or “delete everything.” This matters practically too — memories can be mistaken or go stale, and a wrong fact quietly loaded into every conversation causes persistent, confusing errors, so an easy correction path is essential. And remembered facts are personal data: they must be stored securely, kept private to that user, and never leaked into someone else's session. This connects straight back to earlier lessons — enforce access so one user's memories never surface for another, and treat the store with the same care as any sensitive database. Memory makes assistants feel personal, and personal means the privacy and control obligations are real.

7 · What makes an agent feel like one

Step back and see what memory unlocks. Without it, even a capable, tool-using assistant is a stranger every session — you re-explain yourself each time and it can't build on anything. With memory, the experience transforms: it greets you by name, recalls your preferences, picks up an ongoing project where you left off, and remembers decisions made weeks ago. That continuity is a large part of what makes something feel like a real agent rather than a one-shot chatbot — an assistant that knows you, can act on your behalf through tools, and accumulates understanding over time. It's the final layer that ties the course together: the model provides intelligence, prompts and context shape each interaction, retrieval and tools connect it to information and action, and memory gives it continuity across all of them. Intelligent, connected, and persistent — that combination is the foundation every serious AI application is built on.

8 · A simple test you can run this week

9 · Glossary — every term, spelled out

Agent memory

The app-built ability for an assistant to remember across conversations, since the model itself can't.

Short-term memory

The current conversation's running history, re-sent each turn to keep the chat coherent.

Long-term memory

A persistent store of durable facts that survives across conversations.

Promotion

Moving an important fact from short-term into long-term memory so it lasts.

Memory retrieval

Fetching just the stored memories relevant to the current message, usually via vector search.

User control

Letting people see, correct, and delete what an assistant remembers about them.