Langfuse vs LangSmith for Agent Observability: How to Choose

Langfuse vs LangSmith for Agent Observability: How to Choose

There is no universal winner in Langfuse vs LangSmith. The choice depends on three questions: where your trace data is allowed to live, how much personal data those traces contain, and which framework your agent is built on. If you need control over hosting and run a mixed stack, Langfuse usually goes on the shortlist first. If your agent is built on LangChain or LangGraph and the vendor's cloud is acceptable, LangSmith tends to create less friction.

Key takeaways

• Hosting and personal data matter more than the feature list. Decide where a trace can live before you compare dashboards.
• Langfuse is open source and can be run on your own infrastructure or used as a cloud service. LangSmith is, first of all, a service from the LangChain team, with self-hosting options subject to the vendor's conditions.
• A customer support agent's trace almost always contains personal data. Masking, retention and role-based access must be defined before the first day in production.
• The cheapest integration is the one that already works with your framework. OpenTelemetry support lowers the risk of being locked into one tool.
• Run a short pilot with one real workflow on both candidates and decide with written criteria, not a sales demo.

What an observability tool must give an agent

Observability, in plain terms, is the ability to answer questions about how a system behaves based on what it records. The OpenTelemetry observability primer organizes this into traces, metrics and logs. For AI agents, the trace is the central piece: it shows the sequence of model calls, tool use, retrieved data and the final response.

In practice, the process owner needs the tool to help answer:

• What did the agent receive, what did it decide and what did it send to the customer?
• At which step did the conversation go off track: context retrieval, a tool call or generation?
• How much time and how many tokens does each workflow consume?
• Did a prompt or model change improve or worsen the result?
• Who accessed these records and how long are they kept?

Langfuse and LangSmith both cover the core of this: trace capture, sessions, evaluation, prompt management and datasets for testing. The differences that change the decision are elsewhere.

Hosting: vendor cloud or your own infrastructure?

Here we separate what comes from official documentation from what is our own reading. According to each product's documentation, Langfuse has an open source core and can be run by you or purchased as a cloud service. LangSmith is offered as a cloud service by the vendor, and there is a deployment option inside the customer's own environment. The conditions, requirements and plan needed for that option should be confirmed directly with the vendor, because they change over time.

What this means for the decision:

• Vendor cloud: starts fast and needs little operational work. In exchange, traces leave your infrastructure and you depend on the contract, the storage region and the provider's guarantees.
• Self-hosting: gives you control over location, network and retention. In exchange, your team operates the database, queues, upgrades, backups and security of the tool. That is one more service to maintain.

A hypothesis that tends to hold up: small and mid-sized companies underestimate the cost of running the tool themselves and overestimate the risk of the cloud. Companies with contractual requirements from customers, or in regulated sectors, go the opposite way and find out late that the cloud was never an option. That is why the first step is to ask legal and security what is permitted, not to ask the technical team what is more convenient.

Personal data and LGPD: what goes into the trace

An agent that handles support on WhatsApp records names, phone numbers, addresses, orders and sometimes health or financial details, all inside free-text messages. If the trace stores full inputs and outputs, the observability tool becomes one more system that processes personal data. That puts it in scope for your data processing inventory, your contracts with processors and, when the risk is significant, an impact assessment. For that last point, see data protection impact assessments for AI projects. LGPD is Brazil's general data protection law.

Questions worth asking both vendors, and documenting the answers:

• Can fields be masked or removed before the data leaves your application, rather than after it reaches the tool?
• What is the retention policy, and can automatic deletion be configured per project?
• Is there role-based access control, so the development team does not see customer conversations without a need?
• How does a data subject's deletion request work when the data is spread across traces and evaluation datasets?
• Where is the data stored and which subprocessors are involved?
• Does the vendor use customer data for any other purpose? The answer should be in the contract, not only on the website.

Brazil's data protection authority (ANPD) is the official reference for guidance on data processing in the country. We will not cite legal articles from memory here: take these questions to your data protection officer or legal counsel and record the decision. If the agent makes or influences decisions about people, human review also comes into play, as we explain in automated decision-making under LGPD.

One point that holds regardless of the tool: the best protection is to record less. In many workflows it is enough to store the conversation ID, the steps executed, the tools called and the outcome, leaving the full text for controlled samples.

Integration: framework, SDK and OpenTelemetry

Integration determines how much extra work you face in the first month. According to public documentation, LangSmith integrates very directly with LangChain and LangGraph, and also offers SDKs to instrument code that does not use those frameworks. Langfuse offers SDKs and integrations with several frameworks and model providers, and also accepts data through OpenTelemetry. Confirm the current list of integrations on the official pages, because it changes often.

Why OpenTelemetry matters: it has semantic conventions for generative AI that standardize how model calls are described. If your agent emits traces in that format, switching tools later stops being a rewrite. It is protection against lock-in, and it is worth more the longer the system is expected to live.

Integration checks for the pilot:

• Is your main framework supported without fragile glue code?
• Do tool calls and retrieval steps appear as separate steps in the trace?
• Can you attach business metadata, such as channel, workflow type and prompt version, without exposing personal data?
• Does the tool let you connect evaluations to datasets and run them on every change? This ties into prompt regression testing in CI.
• What happens to the application if the observability tool goes down? Trace delivery should be asynchronous and must never take customer support offline.

A decision guide in list form

Use this as a starting point, not a fixed rule. These are implementation heuristics, not vendor claims.

• If customer data cannot leave your infrastructure and you have a team able to operate services: prioritize the self-hosting option and validate the operating effort.
• If you have no team to run one more service and legal accepts the cloud with a proper contract: prioritize the managed service and validate region, retention and subprocessors.
• If the agent is built on LangChain or LangGraph and the team already works in that ecosystem: include LangSmith in the first tests, because of its proximity to the framework.
• If your stack mixes frameworks, model provider SDKs and custom code: give weight to OpenTelemetry support and instrumentation flexibility.
• If continuous evaluation with human review is the main goal: compare how each tool handles datasets, annotations and comparison between versions.
• If neither option passes your data requirements: record only metadata for now and postpone the decision. Partial observability is better than an unmapped risk.

Illustrative scenario: an after-sales agent on WhatsApp

Illustrative scenario: a distributor of electrical supplies has an agent that answers after-sales questions on WhatsApp and hands warranty cases to a person. The owner wants to know why some conversations end in an unnecessary handoff.

The team lists what it needs to see: the customer's message, the policy excerpt consulted, the decision to hand off and the reason. It also decides what it does not need to see: full phone numbers and addresses. They define masking before sending, short retention for full text and access limited to two people.

Then they instrument the same workflow in both tools for a short period and compare: integration effort, how clearly the trace shows the reason for the handoff, data controls and how easy it is to build a set of test cases. The decision comes from a criteria sheet filled in by the team, together with legal's opinion on hosting. For context on the channel, see how to integrate AI into your company's WhatsApp.

How we apply the method: map, limit, pilot, hand over

The method we follow for agents also works for choosing the tool, without adding ceremony.

• Map: describe the current workflow, the data that passes through it and the questions observability needs to answer.
• Limit: define what will be recorded, for how long and who has access. Start with the minimum and expand only with a justification.
• Pilot: instrument one real workflow, with success metrics agreed in advance. Measuring an AI automation pilot helps you set the decision criteria.
• Hand over: deliver dashboards, a trace review routine and a named owner. Then decide whether to expand, adjust or stop.

Evaluating agent behavior is a separate subject, and Anthropic's engineering article on evaluating AI agents is a good basis for deciding what to measure with the traces you collect.

Frequently asked questions

Is Langfuse better than LangSmith? It depends on the criterion. For control over hosting and use with mixed stacks, Langfuse is usually more flexible. For teams already working with LangChain or LangGraph, LangSmith usually requires less setup. Test both with a real workflow before deciding.

Do I need observability if my agent is still a pilot? Yes, and that is the stage where it helps most. Without traces, you do not know why the agent got something wrong or whether a prompt change improved anything. The logging can be minimal, as long as it lets you reconstruct what happened in each conversation.

Are an agent's traces covered by LGPD? If they contain information about an identified or identifiable person, such as a name, phone number or message content, yes. That requires a legal basis, a clear purpose, access control and a defined retention period. Confirm the classification with your data protection officer or legal counsel.

Can I switch tools later? It is easier if you instrument with open standards such as OpenTelemetry and keep your test datasets and prompts versioned outside the tool. Historical data and vendor-specific configuration are usually the hardest parts to migrate, so plan the export from the start.

Sources

• OpenTelemetry: observability concepts
• OpenTelemetry: semantic conventions for generative AI
• ANPD: Autoridade Nacional de Proteção de Dados
• Anthropic: Demystifying evals for AI agents