AI agents

AI agent memory guide: short-term state, long-term memory, and retrieval

A practical guide to AI agent memory: thread state, checkpoints, long-term user memory, retrieval, knowledge bases, privacy, consent, deletion, and evaluation for production agents.

Updated 2026-06-119 min readIntermediate

Read RAG chunk size guide Read enterprise RAG security checklist

Best for

Builders designing personal assistants, support agents, sales agents, or internal copilots
Teams deciding what an AI agent should remember across sessions
Product managers writing consent, deletion, and personalization requirements
Engineers separating checkpoints, conversation state, vector search, and user profiles

Not for

Storing every conversation forever because it feels useful
Using memory to bypass permissions, consent, or retention policy
Treating vector search, profile fields, and agent state as the same thing

Comparison

Choose by workflow, not brand

Option	Best for	Strengths	Tradeoffs	Use when
Short-term state	Current task progress, intermediate decisions, tool results, and workflow checkpoints	Keeps the agent coherent during one task or thread and supports recovery.	Should expire or reset when the task is finished.	The agent needs continuity inside a single workflow.
Long-term memory	Stable preferences, account facts, prior commitments, and user-specific context	Improves personalization and reduces repeated setup.	Requires consent, editing, deletion, access control, and evals for stale facts.	Remembering the fact clearly improves future sessions and is safe to retain.
Retrieval memory	Documents, tickets, CRM records, policies, code, and other source-grounded knowledge	Keeps answers anchored to updateable sources with citations and permissions.	Quality depends on chunking, embeddings, reranking, permissions, and refresh logic.	The agent needs facts from a changing knowledge base.

Separate state from memory

State is what the agent needs to finish the current run. Memory is what the product chooses to keep for future runs. Mixing them creates stale personalization, privacy risk, and hard-to-debug behavior.

Keep run state inspectable in traces and checkpoints.
Store durable memory in explicit fields or documents, not hidden prompt text.
Let users or admins remove memory that affects future behavior.

Use retrieval for facts

Knowledge bases are better than vague memory for policies, product docs, tickets, and code. Retrieval lets you update sources and apply permissions without rewriting the agent personality.

Use vector and keyword search for source-grounded facts.
Add citations or source IDs when the answer depends on a document.
Evaluate freshness, missing context, and permission leakage.

Evaluate memory like a feature

Memory failures are product failures. The agent can over-personalize, remember wrong facts, leak one user's context to another, or keep stale account data after a change.

Test create, update, recall, and deletion flows.
Add negative tests where the agent must not remember sensitive information.
Measure whether memory improves task success rather than just sounding personal.

Decision Rules

A practical checklist

Use short-term state for active workflow progress and tool outputs.

Use long-term memory only for durable facts with consent, edit, and delete paths.

Use RAG for source-grounded knowledge that changes or requires permissions.

Do not ship memory without tests for stale facts, privacy leakage, and cross-user isolation.

Related Guides

Continue the decision path

Read RAG chunk size guide

Tune retrieval quality before adding durable memory.

Open

Read enterprise RAG security checklist

Protect permissions and sensitive data before agents remember anything.

Open

RAG chunk size guide

Improve retrieval before expanding memory scope.

Open

Hybrid search RAG guide

Combine keyword and vector retrieval for stronger factual grounding.

Open

AI data residency guide

Plan where memory and retrieval data are stored.

Open

Chinese Archive

Aligned deeper reading

Dify knowledge-base archive

Chinese RAG and knowledge-base workflow materials.

Open

AI agent archive

Chinese AI agent notes and implementation examples.

Open

Topic Hubs

Explore the wider search cluster

Topic hub

Coding agents

Compare AI coding agents, repo-aware developer tools, app builders, agent frameworks, MCP servers, workflow automation, and practical engineering adoption paths.

Open

FAQ

Common questions

What is AI agent memory?

AI agent memory is the information an agent uses to maintain continuity. It can include short-term state for a task, long-term user or account facts, and retrieved knowledge from external sources.

Is vector search the same as memory?

No. Vector search is a retrieval method. Memory is a product decision about what context should influence future behavior.

What is the biggest risk of agent memory?

The biggest risks are privacy leakage, stale facts, cross-user contamination, and behavior that changes without a visible reason.

Source Links

Primary references used for this guide

Reference

LangChain memory concepts

LangChain documentation explaining short-term and long-term memory concepts.

Open

Reference

LangGraph overview

LangGraph docs covering persistence, human-in-the-loop, and memory.

Open

Reference

CrewAI documentation

CrewAI documentation for memory, knowledge, flows, guardrails, and observability.

Open

Reference

Agentforce context engineering guide

Salesforce guide to giving agents context, actions, and instructions.

Open

Build your own evaluation note

The strongest decision is always local to your workflow. Save the vendor links, define a representative task, record the exact prompt or command, and compare the final evidence instead of the marketing claim.

Return to the AI learning map