Skip to content
Hybrid GraphRAG Wiki & Architecture

Langfuse Observability & OpenTelemetry Tracing

Overview

The platform embeds a self-hosted Langfuse v4 observability stack running locally inside the Docker Compose cluster (http://localhost:3001). Langfuse captures detailed execution traces for every agent consultation, vector search, graph query, and EvidenceGate filter decision.

Langfuse v4 Observability Console

Observability Infrastructure

The Langfuse v4 stack comprises five coordinated services:

  • langfuse: Web application and REST API on host port 3001.
  • langfuse-worker: Asynchronous task executor processing ingestion queues.
  • clickhouse: High-performance OLAP database storing trace events on host ports 8124 / 9001.
  • redis: Task queue and caching layer on host port 6389.
  • minio: Internal S3-compatible event and media storage on host ports 9092 / 9093.

Hierarchical Span Model

Every agent execution produces a structured trace tree in Langfuse:

Trace: [agent_run: RiskAssessmentAgent]
  |-- Span: Agent Observation (as_type="agent")
  |     - Prompt inputs, system code filter, foundation model parameters
  |
  |-- Span: Retriever Span (as_type="retriever")
  |     - Dense vector similarity queries (Qdrant)
  |     - Sparse keyword lexical scores (SPLADE)
  |     - Retrieved candidate chunk IDs and raw distances
  |
  |-- Span: Tool Span (as_type="tool")
  |     - Memgraph Cypher multi-hop graph traversals
  |     - PostgreSQL canonical record lookups
  |
  |-- Span: Guardrail Span (as_type="guardrail")
  |     - EvidenceGate pre-flight validation decisions
  |     - Rejected chunk hashes and supersession reasons
  |
  `-- Score Dispatches:
        - rtm_coverage_percentage: 1.00
        - part11_compliance_score: 0.98
        - csa_rpn_score: 12.0

Logging Scores and Evaluation Metrics

The backend tracking utility (src/observability/tracker.py) dispatches real-time compliance evaluation scores to the Langfuse API:

  • rtm_coverage_percentage: Calculated ratio of verified requirements to total requirements.
  • part11_compliance_score: Score measuring completeness of electronic signatures and audit trail entries.
  • csa_rpn_score: Maximum post-mitigation Risk Priority Number.
  • evaluation_latency_ms: Total end-to-end execution time.

Engineers can inspect these traces in the Langfuse console to analyze latency bottlenecks, monitor prompt token usage, and audit agent decision paths.