What is Langfuse?
Langfuse is an open platform for tracing, evaluating, and improving AI agents and applications, aimed at developers and teams building with language models. It collects information about model calls, tool use, and retrieval steps so teams can inspect production behavior, track costs, and test changes. Teams can use it in the cloud or self-host it, and connect applications through SDKs, OpenTelemetry, APIs, or a model gateway.
Langfuse Features:
Trace Capture: Capture hierarchical traces of model calls, tool invocations, and retrieval steps through Langfuse SDKs, OpenTelemetry, or supported integrations.
Cost Tracking: Track token usage and costs across AI calls, with filters and dashboards for examining spend by user, model, or other dimensions.
Agent Evaluation: Evaluate production data or experiments with LLM-as-a-judge, heuristic functions, and human review, then record evaluation scores.
Prompt Management: Manage prompts separately from application code, deploy changes, roll back versions, and test prompts against real production inputs.
Custom Dashboards: Build dashboards from traces, observations, or scores, using filters, aggregations, and chart types such as line and bar charts.
Model Gateway: Route OpenAI and Anthropic calls through the Langfuse gateway, where calls are traced automatically and costs can be viewed by user or key.
Langfuse Benefits:
Faster Debugging: Seeing each step of an agent run in one trace helps developers pinpoint failed generations, slow operations, and problematic retrievals.
Focused Cost Reviews: Breaking usage down by model, user, or key makes it easier to find where spending is concentrated and decide what to investigate.
Safer Prompt Changes: Versioning, testing, and rollback give teams a way to refine prompts while reducing the risk of leaving an unwanted change in production.
Shared Quality Work: Evaluation and human annotation let engineering and review teams assess outputs together and turn reviewed examples into reusable datasets.
Flexible Adoption: OpenTelemetry support, integrations, APIs, and self-hosting options let teams work with existing instrumentation and retain control over their data.
Use Cases:
Production Debugging: Investigate an AI agent that returns failures or runs slowly by inspecting its model calls, tools, and retrieval steps.
Prompt Comparison: Test alternative prompts on real production inputs and compare model responses before choosing a version to deploy.
Spend Monitoring: Review model costs and latency across users or keys to understand how a production assistant is using AI services.
Regression Testing: Create a dataset from application examples, run experiments against it, and score prompt or model changes for quality.
Human Review: Set up a collaborative review process for agent traces, capture feedback, and build a collection of approved examples.