Test-Time Memorization (Titans)
Memory module that learns at inference time by incorporating recent inputs into its parameters during the session rather than relying solely on pre-trained weights.
Problem
Pre-trained-only models can't learn within a session. Retraining is too slow and expensive to do per-session. RAG retrieves but doesn't internalize. The agent needs a way to memorize within a session that's faster than retraining but more integrated than retrieval.
Solution
Behrouz et al. 2024 — Titans architecture. A neural memory module sits alongside the main model; during a session, inputs trigger updates to the module's parameters (gradient steps at inference time). Later steps in the same session benefit from this in-session learning. Module state is per-session and ephemeral. Pair with episodic-memory, agentic-memory, landmark-attention, agent-resumption.
When to use
- Long single-session tasks where early inputs should shape late decisions.
- Compute budget allows test-time parameter updates.
- Research / experimental setting with model-side flexibility.
Open the full interactive page →
Diagram, neighbourhood map, code examples, related patterns and full provenance.