openllmetry
OpenLLMetry is an open-source extension set that uses the OpenTelemetry infrastructure to provide viability for productive AI and LLM applications. Because it produces standard OpenTelemetry data, it can be directly integrated with your existing observability tools. It includes standard tracking tools for LLM providers and vector databases.
traceloop · Observability / Datadog Alternatives
Common commercial products in this category: Datadog, New Relic, Dynatrace
- You can start LLM tracking in your Python code using Traceloop SDK.
- You can immediately view tracking data locally by disabling mass data sending.
- You can connect the data you produce to their existing systems, such as Datadog, Honeycomb, and Azure Application Insights.
- • High number of open issues (820)
- ↓ NegativeHacker News
Show HN: OpenLLMetry – OpenTelemetry-based observability for LLMs
“Just to add to Nir's answer here: Let's say your application takes several steps to build up a prompt dynamically, such as a RAG pipeline. You'll end up with a different prompt for potentially each user, depending on the application. The result is you've likely increased the…”
- • NeutralHacker News
Show HN: OpenLLMetry – OpenTelemetry-based observability for LLMs
“What is the difference from using OpenLLMetry versus using OTel directly? Is the issue that there aren't conventions for the needed attributes?”