Langfuse in Practice: Make Slow, Expensive and Wrong AI Responses Diagnosable

After an AI application launches, the painful moment is often a user saying “it got this wrong again” while you cannot tell which step failed.

Did retrieval miss the document, or did the prompt omit a condition? Was the model slow, or did a tool time out and retry three times? Did a cheaper model save money per call but make the whole task more expensive?

Langfuse connects these questions to one task’s execution. Inspect the call tree, inputs, outputs, latency, usage and evaluation results, then decide what to change. A dashboard is the entry point; fixing problems is the benefit.

This article follows official documentation checked on October 11, 2026, and Python SDK 4.17.0. The example demonstrates instrumentation without calling a real model. No production benchmarks or deployment results are invented.

Distinguish traces, observations and sessions

Concept	What it records	Common mistake
Trace	One complete task: a chat turn or agent run	Putting a day’s requests into one trace
Observation	One step: retrieval, model invocation or tool execution	Recording only the final answer
Session	Related turns or successive tasks	Replacing individual task boundaries

Use one trace per question, with retrieval and generation as children. Link multiple conversation turns with a session. The current SDK uses OpenTelemetry; correct context propagation matters more than attractive names.

Do not wrap an entire agent loop in one generation. Record each model call separately and associate tools with the relevant step. Otherwise you see the total bill but cannot find the result that inflated the context.

Instrument one workflow first

One RAG question-answering endpoint is enough to start:

answer-question
├── retrieve-documents
└── draft-answer

Record the question and final result on the root, document IDs, versions and supplied passages on retrieval, and model inputs, outputs and actual usage on generations.

Keep names stable, such as retrieve-documents. Put order numbers, questions and request IDs in attributes, not names. Add release, environment and prompt version to make before-and-after comparisons possible.

A minimal Python example

This example pins Python SDK 4.17.0, requiring Python 3.10 or newer. Check SDK and server versions separately; do not mix methods from old tutorials.

python3 -m venv .venv
source .venv/bin/activate
python -m pip install "langfuse==4.17.0"

export LANGFUSE_PUBLIC_KEY="pk-lf-REPLACE_ME"
export LANGFUSE_SECRET_KEY="sk-lf-REPLACE_ME"
export LANGFUSE_BASE_URL="https://cloud.langfuse.com"

Get project keys in Langfuse settings. BASE_URL above is the EU Cloud endpoint; replace it for another region or self-hosting. Keep secrets in server-side environment variables. Merely writing a .env file does not make Python load it; these commands export variables explicitly.

Save this as trace_demo.py and run python trace_demo.py:

from langfuse import get_client

langfuse = get_client()
question = "How long are repeat article visits deduplicated?"

with langfuse.start_as_current_observation(
    as_type="span",
    name="answer-question",
    input={"question": question},
    metadata={"release": "demo-v1", "environment": "development"},
) as root:
    with langfuse.start_as_current_observation(
        as_type="span", name="retrieve-documents", input=question
    ) as retrieval:
        documents = [{"id": "privacy-v1", "text": "Repeat visits count once within 30 minutes."}]
        retrieval.update(output=documents)

    with langfuse.start_as_current_observation(
        as_type="generation",
        name="draft-answer",
        input={"question": question, "documents": documents},
        metadata={"simulation": True},
    ) as generation:
        answer = "Repeat visits count once within 30 minutes. [privacy-v1]"
        generation.update(output=answer)

    root.update(output={"answer": answer})

langfuse.flush()

Documents and answers are hard-coded, with simulation=True. The generation makes no model request and reports no invented tokens or costs. Replace both sections with your retrieval and model functions, record the actual model ID and copy real usage from the provider response.

Call flush() before a short-lived script exits. Handle long-running services through their lifecycle, rather than flushing after every token or tiny step. Verify propagation across async tasks, threads and queues: a context manager alone does not guarantee cross-process continuity.

What to inspect once traces arrive

Start with a request a user has confirmed is wrong:

Input conditions: missing product model, date or applicable version?
Retrieval: relevant, current documents with essential conditions preserved?
Model call: complete context and unexpected retries?
Tool result: did an HTTP success actually complete the business action?

Inspect step durations and total task time. Streaming also requires distinguishing first-token latency from completion. Count every call and retry when measuring cost. Estimates depend on model pricing configuration and usage; reconcile the provider’s bill separately.

Langfuse does not prove answer correctness or enforce business permissions. Showing an unauthorized attempt is not preventing it. Authentication, validation, idempotency and write confirmation remain application responsibilities.

Prompt management: preserve the version actually used

Moving text into an editable interface is only the start. Link versions to calls so failures can be reproduced and changes compared.

Create a text prompt named support-answer using {{question}} and {{context}}, then label an accepted version production:

prompt = langfuse.get_prompt("support-answer", label="production")
compiled = prompt.compile(question=question, context=documents[0]["text"])

compile substitutes variables; it does not invoke a model. Fetching production selects an intentionally released version. Creating a version should not automatically deploy it. Associate the returned prompt object with the generation through your integration.

SDK caching does not remove every dependency. First fetches, refreshes and failures still matter. Important workflows can prefetch at startup and retain a validated fallback prompt, with logs showing when fallback occurred.

Evaluation: turn failures into regression cases

Start with twenty or thirty real, sanitized cases: ordinary and ambiguous questions, missing evidence, tool timeouts and questions the system should not answer. Define expected behavior before comparing prompts, models or retrieval strategies.

Method	Useful for	Watch for
Deterministic code	JSON, required fields, business conditions	Correct format with wrong content
Human annotation	Expert judgments, edge cases, rubric calibration	Inconsistent standards and review costs
LLM-as-a-Judge	Batch scoring against a clear rubric	Bias, unstable scores and evaluator fees

Online evaluation monitors live traffic; offline experiments compare candidates on a fixed dataset. Use both and inspect important failures rather than hiding them in an average. Calibrate model judges against human labels; keep explicit code checks for permissions and critical rules.

Evaluation has its own cost. Budget sampling rate, judge model and frequency together instead of evaluating everything and discovering monitoring costs more than the application.

Cloud or self-hosting: count operations too

For a first trial, use a Cloud test project and sanitized samples; assess production-data requirements before uploading them. Self-hosting offers control over deployment and data flows, but is more than running one container.

The current architecture includes Web, Worker, PostgreSQL, ClickHouse, Redis/Valkey and S3/Blob storage. PostgreSQL holds transactional data, ClickHouse observability data, cache services the queue, and object storage raw events and multimodal attachments.

Official documentation positions Docker Compose for local use and testing, without built-in high availability, scaling or backups. Production operation needs TLS, authentication, database and object backups, queue monitoring, capacity, retention and upgrade recovery.

Distinguish the open-source core from Enterprise additions requiring licenses. Cloud limits, plans and self-hosting costs are different measures; this article does not substitute an unverified price for a total-cost calculation.

Mask before export and verify after integration

Traces can be more sensitive than ordinary logs: questions, document text, tool parameters and answers may appear together. Do not collect real email addresses, phone numbers, tokens and complete internal files by default.

Sanitize before data leaves the application. Current Python documentation recommends mask_otel_spans for new setups, covering exported OpenTelemetry attributes from SDK and third-party instrumentation; legacy mask has different coverage. Inspect automatic integrations as well as hand-written fields.

Verify normal requests, failures and script shutdown on one workflow. Review exported content and project permissions before expanding.

The first goal is modest: when a user reports a bad answer, find that request, explain the failure and turn it into a pre-release check. That makes Langfuse more than another AI dashboard.

Official sources

Checked October 11, 2026. Complete integration still needs your project credentials and acceptance testing.