Observability

Spring AI Observability provides built-in instrumentation that helps developers understand how their AI applications behave in production. Built on top of Micrometer Observability and integrated with OpenTelemetry, it automatically captures metrics, traces, and structured attributes for common AI operations such as model calls, tool execution, and agent workflows.

With observability enabled, Spring AI can emit traces that describe key aspects of an AI interaction, including the model being used, prompts and LLM responses, tool invocation details, latency, and token usage. These traces can then be exported through the OpenTelemetry ecosystem and visualized in observability platforms such as Grafana Tempo, Jaeger, or Langfuse.

This capability is particularly valuable for modern AI applications that involve multiple components, including LLM calls, retrieval pipelines, and external tools. Because it provides end-to-end visibility into how a request flows through the system. By correlating traces, metrics, and logs, Spring AI observability helps developers diagnose failures, analyze model performance, monitor token consumption, and better understand the behavior of AI agents in real-world environments.

Here I provide an example of enabling observability for Spring AI with OpenTelemetry and Langfuse.

To enable OpenTelemetry, the following dependencies should be added to the Spring Boot project.

1 <dependency>
2     <groupId>org.springframework.boot</groupId>
3     <artifactId>spring-boot-starter-actuator</artifactId>
4 </dependency>
5 <dependency>
6     <groupId>org.springframework.boot</groupId>
7     <artifactId>spring-boot-starter-opentelemetry</artifactId>
8 </dependency>

After adding the dependencies, Spring Boot creates necessary beans to work with OpenTelemetry.

In the Spring AI application, we don’t send data directly to any observation platforms. Instead, the data is sent to OpenTelemetry collector. The collector can transform data and send to different upstream platforms.

OpenTelemetry collector is also started using Docker Compose.

Below is the configuration file of OpenTelemetry collector. It defines two exporters.

• otlp_http/langfuse: This exporter sends data to the OpenTelemetry API endpoint provided by Langfuse. This endpoint is available at path /api/public/otel. The Authorization header is required with the value of Langfuse API key.

• prometheus: This exporter exports data in the Prometheus format, which allows it to be scraped by a Prometheus server.

 1 receivers:
 2   otlp:
 3     protocols:
 4       grpc:
 5         endpoint: "0.0.0.0:4317"
 6       http:
 7         endpoint: "0.0.0.0:4318"
 8 processors:
 9   batch:
10 exporters:
11   otlp_http/langfuse:
12     endpoint: "http://langfuse-web:3000/api/public/otel"
13     headers:
14       Authorization: "Basic ${LANGFUSE_AUTH_STRING}"
15     tls:
16       insecure: true
17   prometheus:
18     endpoint: "otel-collector:10010"
19 service:
20   pipelines:
21     metrics:
22       receivers: [ otlp ]
23       processors: [ batch ]
24       exporters: [ prometheus ]
25     traces:
26       receivers: [ otlp ]
27       processors: [ batch ]
28       exporters: [ otlp_http/langfuse ]

In the configuration below, metrics and tracing data is sent to localhost:4318, which is the port of OpenTelemetry collector. management.tracing.sampling.probability is set to 1.0, so all tracing data will be collected and sent to OpenTelemetry collector.

 1 management:
 2   otlp:
 3     metrics:
 4       export:
 5         enabled: true
 6         url: http://localhost:4318/v1/metrics
 7     traces:
 8       export:
 9         enabled: true
10         url: http://localhost:4318/v1/traces
11   observations:
12     annotations:
13       enabled: true
14   tracing:
15     enabled: true
16     sampling:
17       probability: 1.0
18 otel:
19   logs:
20     exporter: none

Langfuse is an open-source observability platform designed for applications built with large language models (LLMs). It allows developers to monitor and analyze AI interactions by capturing structured traces that include prompts, responses, tool calls, latency, and token usage.

Langfuse integrates with standards such as OpenTelemetry, enabling applications to export telemetry data and visualize complete AI workflows in a dedicated dashboard. This helps developers debug model behavior, track performance in production, and continuously improve the quality of AI-powered applications.

Langfuse is started using Docker Compose. See the docker-compose.yaml for the complete Docker Compose file to start Langfuse and its required dependencies.

After starting the Spring AI app, we can interact with it by send some messages. Open Langfuse UI to view traces. Below is the screenshot of Langfuse UI.

MLflow is an open-source MLOps platform for managing the end-to-end machine learning lifecycle. It enables teams to track experiments, log parameters and metrics, package models, manage model versions, and deploy models across different environments. Designed to be framework-agnostic, MLflow integrates with popular machine learning libraries such as Scikit-learn, PyTorch, TensorFlow, XGBoost, and Hugging Face.

By providing a unified interface for experiment tracking, model registry, and deployment, MLflow helps improve reproducibility, collaboration, and operational efficiency in machine learning projects. Whether you’re developing traditional machine learning models or modern generative AI applications, MLflow simplifies the process of managing models from experimentation to production.

MLflow is started using Docker Compose. See the docker-compose.yaml for the complete Docker Compose file to start MLflow.

We can configure the OpenTelemetry collector to send tracing data to MLflow. The x-mlflow-experiment-id header specifies the experiment ID in MLflow. This header is required, otherwise the request will be rejected with 400 errors.

1 exporters:
2   otlp_http/mlflow:
3     endpoint: "http://mlflow:5000"
4     headers:
5       x-mlflow-experiment-id: "0"
6     tls:
7       insecure: true

Below is the screenshot of MLflow UI.