Helicone

Helicone is designed for monitoring LLM applications at scale. It provides real-time insights into application performance, including user sessions, model usage, and costs. Key features include:

Easy integration with just two lines of code.
Support for any model and provider, including fine-tuned models.
Tools for segmenting requests, caching, rate limiting, retries, feedback, and secure key mapping.
Visualization of request chains and ETL capabilities with GraphQL.
Alert notifications for important events.
Designed for high-scale operations with minimal latency impact.