Store AI Agent Conversation State: Postgres vs Redis and Others

Store AI Agent Conversation State: Postgres vs Redis and Others

For most customer support agents, conversation history belongs in a relational database such as Postgres, which gives you durability, flexible queries and controllable retention. Redis works as a support layer for what is short-lived and fast, such as active sessions, concurrency locks and message deduplication, but not as the only place where the conversation exists. Other options make sense in specific cases, and the choice should start from what the business needs to query, for how long, and how much loss it can tolerate.

Key takeaways

• Separate history (the record of what was said) from state (which step the flow is on and what has already been collected). They have different storage requirements.
• Postgres is the safe default as a source of truth: transactions, queries by customer and period, and mature backup and access control.
• Redis is excellent for automatic expiry, locks and caching, but using it as the only record creates a risk of data loss and makes auditing harder.
• Expiry is a business and privacy-law decision (LGPD), not a technical detail. Define the retention period for each purpose before choosing the tool.
• The real operating cost is in who handles backup, cleanup, migration and monitoring, not just hosting.

What is conversation state, and why is it more than history?

When someone asks how to store a chatbot's conversation history, they are usually mixing four things that deserve separate treatment.

• Message history: the chronological log of what the customer, the agent and any human attendant said, with channel, time and message identifier.
• Flow state: the current step, the data already collected (for example, an order number or address), pending actions, and whether the conversation is with the agent or with a person.
• Summary memory: a summary or extracted facts from earlier conversations, used to continue without resending everything to the model.
• Control metadata: consent, language, preferences, the reason for handing off to a human, and a record of the agent's decisions.

History almost always needs to last and be searchable. Flow state changes with every message and can often expire. Summary memory needs to be correctable and deletable. Mixing all of it into a single structure is the source of most of the problems that show up later: stuck conversations, answers out of context, and data kept longer than it should be.

There is another important distinction: what you store is not what you send to the model. The database can keep the entire conversation while each call receives only a useful slice. This ties storage to the topic of the context window in AI, and the practice Anthropic describes in its writing on context engineering points the same way: select what goes into the context instead of dumping everything in.

When is Postgres the right choice?

Postgres is usually the most defensible starting point for support agents, for practical reasons.

• Durability: committed writes survive restarts, and backups and point-in-time recovery are well-known, documented features.
• Querying: you answer business questions with SQL, such as which conversations are still open, which were handed to a human, or what a specific customer said last week.
• Mixed structure: traditional columns for what is stable (customer, channel, time) and JSON fields for what varies (data collected per flow).
• Transactions: writing the message and updating the flow state atomically avoids the classic case where the message is saved and the state is not.
• Access control: role-based permissions and row-level isolation help when more than one team or client uses the same database.

The costs are real too. Someone has to operate the database, or pay for a managed service. Expiry is not automatic: you need a cleanup routine, and at higher volumes it is worth partitioning the messages table by period so that deleting old data is simple. Queries over free text inside messages need indexes or a dedicated search tool, otherwise they get slow.

For the typical small or mid-sized company that serves customers on WhatsApp, these costs are manageable and the gain in auditability is worth it. If you are still designing the channel integration, the guide on how to integrate AI into your company's WhatsApp helps you see where storage fits into the flow.

Where does Redis help, and where does it get in the way?

Redis keeps data in memory, with very low latency and native per-key expiry. That makes it ideal for anything ephemeral.

• Active session: the state of the conversation in progress, which can expire after a period of inactivity.
• Deduplication: messaging platforms can resend the same notification, and a short-lived key prevents answering twice.
• Per-conversation lock: prevents two executions from processing messages from the same customer at the same time and scrambling the reply.
• Message batching: when a customer sends several short messages in a row, the agent waits a moment and replies once.
• Cache for looked-up data, such as order status, so you do not hit the source system on every turn.

The problem appears when Redis becomes the only copy. Durability depends on the persistence configuration and the plan you pay for, and the data model was not built for questions like show me every conversation handed to a human this month. Memory is also a more expensive resource per volume than disk, so keeping months of history there is rarely a good trade. And without a durable record, it is hard to answer a data subject request, investigate a complaint, or evaluate the quality of the agent.

The simple rule: if losing the data is acceptable (a session that starts over), Redis is fine. If losing it would mean you cannot prove what was said to the customer, also write it to a durable database.

What about other options: MongoDB, DynamoDB, SQLite, files?

Nothing forbids using other tools. Each one has a fit.

• Document databases, such as MongoDB: a good match when each conversation is naturally a flexible document and the team already knows the tool. The trade-off is that reports crossing conversations with customers and orders are more work than in SQL.
• DynamoDB and similar cloud stores: they offer time-to-live expiry and scale without server administration. They suit key-based access well (fetch the conversation for one phone number), but exploratory queries require planning the access patterns up front.
• SQLite: works for prototypes, internal tools and agents running locally. When multiple instances write at the same time, it becomes a limitation.
• Object storage: good for keeping transcripts, audio and attachments for the long term at low cost, always with an index in a database pointing to them.
• Agent framework persistence: many libraries offer state checkpoints with adapters for Postgres, Redis and others. Use the adapter, but confirm where and for how long the data stays, because the default may be volatile memory.

If the company runs models locally for privacy reasons, the same reasoning extends to the database: keeping everything on your own infrastructure only makes sense if someone takes over operations. It is worth reading about running a local model without improvising before you decide.

How do you compare options on durability, querying, expiry and cost?

Use this decision guide as a starting point. It describes tendencies, not guarantees, and depends on configuration and vendor.

• Durability: Postgres and managed document or cloud databases are suitable as a record. Redis needs care with persistence and replicas. SQLite depends on a single file and on whoever backs it up.
• Querying: Postgres leads for varied questions and for joining with other data. DynamoDB and Redis are strong at key-based access and weak at exploration. Document databases sit in the middle.
• Expiry: Redis, DynamoDB and MongoDB have native time-to-live mechanisms. In Postgres you build the routine, preferably with partitions by period.
• Operating cost: a managed service reduces work but does not remove decisions about backup, access, retention and monitoring. One more Redis is one more piece to watch, and that counts.
• Auditing: anyone who needs to reconstruct the conversation, the tools called and the handoff to a human benefits from a single, queryable record.

A common architecture that keeps complexity low is Postgres as the source of truth and optional Redis for what is ephemeral. For small operations, Postgres alone is enough, and adding Redis can wait until there is a measured latency or concurrency problem.

How should you handle expiry, LGPD and sensitive data?

Expiry should not be set by what the database makes easy, but by what the purpose requires. A support conversation may need to stay available for warranty or dispute purposes, while the temporary state of a scheduling flow can disappear when it ends. Each category of data deserves a retention period and a recorded justification.

Some practical precautions:

• Keep the minimum: if the agent does not need the customer's ID document after verification, do not keep it in the history in plain text.
• Mask or separate sensitive data that customers type spontaneously, such as ID numbers and health information.
• Make sure you can locate and delete one person's data, including summaries and derived memories, not just the original messages.
• Treat audio transcripts as part of the history, with the same retention rules.
• Record where the database is hosted. If the provider or the model is outside Brazil, check what LGPD requires in international data transfer when using AI.
• Restrict who can read the history. Broad access to customer conversations is an internal risk, and it is also a matter of measuring productivity without turning it into surveillance when the team is being monitored.

For official guidance on data protection in Brazil, the ANPD website is the primary reference. Specific legal questions, such as the legal basis and the retention period, need validation from whoever is responsible for legal matters at your company.

How to choose in practice: map, limit, pilot, hand off

The working method we use for agents follows four moves, and they apply well to the storage decision.

• Map: list what the flow needs to remember during the conversation, what it needs to look up later, and who needs to read it. Note what happens today, whether in a spreadsheet, a CRM or the WhatsApp history.
• Limit: define retention periods, fields that will not be stored, and the size of the history sent to the model on each turn.
• Pilot: run one real flow with logs for each message, decision and handoff to a human, and a success metric agreed in advance. Measure whether the agent lost context, repeated a question or answered twice.
• Hand off: document the schema, the cleanup routine, the backup and who is responsible for each part. Only then decide whether to expand, adjust or stop.

Illustrative scenario: a service business handles quote requests on WhatsApp. The agent collects name, type of service and address, and hands off to a salesperson when the customer asks for something outside the standard offer. During mapping, the team realizes the salesperson needs to see the whole conversation and what the agent has already collected, and that the collection flow can be discarded some days after it is completed. The decision is to store messages and decisions in Postgres with a defined retention period, keep the flow state in a table with an expiry date, and leave Redis out of the pilot. If duplicate replies show up from resent notifications, then a short-lived deduplication key is added. The details here are fictional and only show the order of the decisions.

One detail that often goes unnoticed is idempotency: writing the same message twice after a retry produces a confusing history. Using the channel's message identifier as a unique key solves much of this, and the topic of retries is covered in error handling and retries in n8n.

Checklist before going to production

• Does the history have a durable source of truth, with a backup whose restore has been tested?
• Does each message have a unique channel identifier to prevent duplicates?
• Does the flow state have an expiry period and a clear path to resume or close stalled conversations?
• Is there a retention period for each type of data, and a routine that actually deletes?
• Can you locate and delete all of one person's data, including summaries?
• Is access to the history restricted by role and logged?
• Does the handoff to a human carry the context along, so the attendant does not have to ask everything again?
• Is there minimum observability: write errors, read latency and conversations where state was not recovered?

On that last point, the concept of observability and the OpenTelemetry conventions for generative AI telemetry are a useful basis for standardizing what you record.

Frequently asked questions

Can I keep the chatbot history only in Redis? You can for prototypes or for conversations where losing the session is acceptable. For real customer support, where you may need to prove what was said to the customer, we recommend a durable record in a database, using Redis only for what is temporary.

Do I need to send the whole history to the model on every message? No. The database can keep everything, but the model should receive only the slice it needs, such as the latest messages, the flow state and a summary. This lowers the cost per call and prevents irrelevant information from confusing the answer.

How long should I keep conversations? There is no single period. It depends on the purpose (support, warranty, disputes), on legal obligations in your industry, and on what you can justify. Define it per category of data, record the justification, and validate it with your legal or data protection lead.

Does a managed database solve operations? It reduces the work of maintaining the server, but it does not decide retention, access, tested backups and monitoring for you. Those remain the company's responsibility, which is why it is worth documenting them and naming an owner when the pilot is handed off.

Sources

• Anthropic: Effective context engineering for AI agents
• OpenTelemetry: observability concepts
• OpenTelemetry: semantic conventions for generative AI
• ANPD: Autoridade Nacional de Proteção de Dados