Four-Tier Agent Memory Construction
also known as conversational-semantic-episodic-procedural memory build, CSEP memory stack
Build agent memory as four parts that work together. Conversational memory holds the recent turns. Semantic memory holds facts and their embeddings. Episodic memory holds traces of past interactions. Procedural memory holds learned skills and routines. Each part has its own rules for what to write, how to look things up, and how to shrink it when it gets too big. This rejects the shortcut of using one vector store for everything. It makes the team decide, for each part, what is stored, how it is retrieved, and how it is compressed when it fills up.
Methodology process overview
Intent. Replace 'agent memory is one vector store' with four clear parts, conversational, semantic, episodic, and procedural, each with its own rules.
When to apply. Use this when you design memory for any agent that must remember things beyond a single conversation: assistants, coding agents, and long-running ops agents. It helps most when a team has hit recall problems with one plain vector store. Don't apply it for single-turn agents and stateless tools. Skip it too when the model's context window already covers the memory needs and you do not need lookup at all.
Inputs
- Memory budget — The limits for each part: how many tokens, how much storage, and how much delay you can afford.
- Recall scenarios — The questions or moments where the agent must remember something, and which part should answer each one.
Outputs
- Four-tier memory architecture — Conversational, semantic, episodic, and procedural stores, each with a clear interface.
- Per-tier policies — Write, lookup, and compression rules for each part, written down and easy to tune.
Steps (6)
Build the conversational tier
Keep a window on the live conversation. Pick how you trim it (sliding window, summary plus window, or layered summaries) and decide what gets promoted to the longer-lived parts.
Build the semantic tier
Store extracted facts and their embeddings so you can look them up by similarity. Decide what counts as a fact and what stays as an episode.
Build the episodic tier
Save real past interactions and events with timestamps and IDs. Episodes are the agent's diary. You use them to recall what happened last time.
usesEpisodic Memory
Build the procedural tier
Store skills, routines, and plans that worked, so the agent can replay them. This is how the agent gets faster and better at a recurring task.
Define retrieval orchestration across tiers
Decide which part answers which kind of recall request. Decide how to merge or rank results when more than one part replies.
Define compression and forgetting policies
Give each part a way to shrink: summarise, cluster, decay by recency, or drop on purpose. Without these rules the parts grow without limit.
Framework-specific instructions
Pick a framework and generate a framework-targeted rewrite of this methodology's steps.
Choose framework
AI-generated for Agent Development Kit (ADK) (Google) — verify against official docs.
Principles
- Memory is not one store. It is four parts that work together with different jobs.
- Every part has its own write, read, and compression rules.
- Look-ups are routed across the parts, not handed to a single vector index.
- Forgetting is a feature. Design it on purpose.
Known failure modes (3)
- ✕Memory Poisoning
Without per-tier write policies, attacker-controlled content can be promoted from conversational to semantic or procedural memory.
- ✕Context Fragmentation
Tiers retrieve disconnected fragments that recombine into an incoherent context — symptom of missing retrieval orchestration.
- ★Memo-As-Source Confusion
Notes the agent wrote to itself in conversational memory are later retrieved as if they were source evidence — a confusion that crisp tier boundaries prevent.
Related patterns (7)
- ★★Episodic Memory
Record past events as time-stamped first-person experiences the agent can recall later, separately from extracted facts (semantic) and learned how-to (procedural).
- ★Semantic Memory
Maintain a dedicated store of what the agent holds to be true about the user and the world, separate from event records (episodic) and learned how-to (procedural).
- ★Procedural Memory
Maintain a third agent memory type alongside episodic (past events) and semantic (facts): procedural memory captures *learned how-to* — reusable skills, workflows, and self-rewritten system instructions that map situations directly to actions.
- ★★Vector Memory
Store memories as embeddings in a vector index and retrieve the most semantically similar items at query time.
- ★★Short-Term Thread Memory
Carry the relevant slice of conversation context across turns within a session.
- ★Skill Library
Let the agent grow its own toolkit by writing reusable skills that subsequent runs can call.
- ★★Memory-Type Storage Specialization
Use different storage technologies optimized per memory type — fast in-memory stores (Redis-class) for episodic, vector databases (Pinecone/Weaviate) for semantic, relational or workflow engines for procedural — instead of one general store for everything.
Related compositions (1)
Related methodologies (1)
Sources (2)
AI Agents in Action
Ch 8 'Understanding agent memory and knowledge', §8.6 'Implementing memory in agentic systems', §8.7 'Understanding memory and knowledge compression' “8.6 Implementing memory in agentic systems ... 8.7 Understanding memory and knowledge compression”
"AI Agents in Action" book: A Deep Dive into the Future of Intelligent Systems
“implementing memory in agentic systems, and memory/knowledge compression”
Provenance
- Added to catalog:
- Last updated:
- Verification status: verified