LLM Orchestration & Evaluation

ML

MLflow

4.7/5.01 verified review

Free·Open Source·Apache-2.0

Open source AI engineering platform for agents, LLMs, and machine learning models.

• Open source platform with over 30 million monthly downloads and widespread industry adoption.
• Native OpenTelemetry support and compatibility with any LLM provider or agent framework.
• Integrated capabilities for systematic evaluation using LLM judges and prompt management.

28.6kstars on GitHub

LA

Langfuse

4.7/5.01 verified review

Freemium

Open source observability, evaluation, and debugging platform for LLM applications.

• Open source architecture allowing complete self-hosting without vendor lock-in.
• Native integration between tracing, evaluation, and prompt management in a unified workflow.
• High performance in ingesting and analyzing large volumes of monthly observations.

37.0kstars on GitHub

OP

OpenRouter

4.6/5.01 verified review

Paid

Unified OpenAI-compatible API interface for hundreds of LLM models with routing and cost tracking.

• Single API endpoint compatible with OpenAI request format across hundreds of models.
• Automated fallback routing and failover handling when primary providers experience downtime.
• Customizable data privacy policies to restrict prompt routing to trusted model providers.

AF

Aigenos Flow

4.5/5.01 verified review

Free

Open-source framework for deterministic AI agent workflows using typed fields.

• Deterministic architecture separating flow control from LLM reasoning.
• Centralized workflow and policy definitions via Markdown configuration files.
• Open-source licensing under Apache-2.0 with no core commercial restrictions.

HE

Helicone

4.5/5.01 verified review

Freemium·Open Source·Apache-2.0

Open-source LLM observability platform and gateway for production AI applications.

• Granular token cost and latency monitoring per model
• Response caching to minimize redundant API calls
• Lightweight SDK integration for backend services

6.3kstars on GitHub

CL

Click

4.5/5.01 verified review

Freemium

Managed agents API with durable sessions and isolated sandboxes.

• API-driven architecture for serverless agent execution.
• Support for any language model via BYOK (Bring Your Own Key).
• Isolated sandboxes and built-in tools such as browser and code execution environments.

DA

Daily

4.5/5.01 verified review

Paid·Open Source·BSD-2-Clause

Global WebRTC infrastructure for real-time voice, video, and conversational AI agents.

• Ultra-low latency via optimized global WebRTC infrastructure.
• Pipecat framework specialized for conversational AI agent orchestration.
• Technical documentation written by authors of WebRTC standards.

627stars on GitHub

UC

UniverLab Canopy

4.4/5.01 verified review

Free·Open Source·Apache-2.0

A Rust DAG graph engine for AI coding agents unifying multiple harnesses and workflows.

• High-performance DAG graph engine written in Rust for minimal overhead.
• Agnostic architecture supporting multiple AI harnesses and model providers.
• Fully open-source software with no licensing fees or proprietary locks.

4stars on GitHub

OP

OpenLLM

4.4/5.01 verified review

Freemium

Local LLM routing layer for development tools and coding agents

• Unifies multiple AI providers and local models under a single routing interface
• Includes built-in initial inference ready to use without external credentials
• Supports configurable fallback chains to ensure continuous availability

RA

Raweki

4.4/5.01 verified review

Free

System of record and governance platform for enterprise AI agents and automated workforces

• Dedicated version control tracking for agent configuration changes
• Automated deployment gates to prevent regressions in production
• Centralized inventory and team ownership tracking

TE

telemetry.dev

4.4/5.01 verified review

Freemium

Native OpenTelemetry observability for AI applications and agents.

• Native OpenTelemetry integration for tracing models and agents.
• Official SDK support for TypeScript and Python.
• Detailed inspection of latency, costs, and token usage.

CO

Count

4.4/5.01 verified review

Freemium

AI-powered platform for analytics auditing and reasoning validation.

• Deconstructs AI responses into verifiable reasoning chains.
• Automatically detects hallucinations and implicit assumptions.
• Collaborative analytics canvas integrated with external data sources.