LLM Orchestration & Evaluation

LL

LlamaIndex

4.4/5.01 verified review

Paid·$50/mo·Apache-2.0

Data framework and ingestion tooling for connecting large language models with enterprise data.

• Comprehensive collection of connectors for unstructured data sources
• Active open-source community with frequent releases in Python and TS
• Specialized document parsing for complex tables and charts

12.9kstars on GitHub

DU

Dust

4.4/5.01 verified review

Freemium

Multiuser AI platform for human-agent collaboration using company context.

• Native integration with multiple corporate data sources like Slack and CRMs.
• Ability to route different language models based on task complexity.
• Collaborative multi-user environment for building and managing AI agents.

LA

LangSmith

4.4/5.01 verified review

Freemium

Debugging, testing, evaluation, and monitoring platform for language model applications.

• Detailed step-by-step tracing for complex workflows and LLM calls.
• Advanced evaluation tools and automated regression testing.
• Native integration with the LangChain ecosystem and other frameworks.

FL

FlowiseAI

4.4/5.01 verified review

Free

Open source platform to build AI agents and orchestrated workflows visually using LangChain.

• Visual node-based interface that accelerates prototyping of complex LLM workflows.
• Open-source architecture enabling complete self-hosting without vendor lock-in.
• Native support for multi-agent orchestration, RAG retrieval, and custom tools.

55.5kstars on GitHub

W&

Weights & Biases

4.4/5.01 verified review

Freemium

Machine learning experiment tracking and AI model observability platform.

• Outstanding tracking and visualization of model training metrics.
• Deep integration with major machine learning frameworks like PyTorch and TensorFlow.
• Functional free tier for independent developers and personal projects.

OR

OrgSync

4.3/5.01 verified review

Paid·$25/mo

Role-aware company brain connecting AI tools like Claude, ChatGPT and Codex via MCP.

• Organizational context synchronization for AI assistants via MCP.
• Company-wide flat pricing model without per-seat penalties.
• Mandatory human approval control for any shared modifications.

IR

IQ Routing

4.3/5.01 verified review

Freemium

Centralized dashboard for AI usage control, budget limits, and intelligent model routing.

• Unifies multiple AI providers under a single base URL proxy.
• Enables strict spending caps and team-level budgets.
• Automated routing reduces operational costs across language model calls.

BO

BotGauge

4.3/5.01 verified review

Contact Sales

AI agent red-teaming, evaluation, and continuous governance platform.

• Adaptive red-teaming and stress testing for AI agents.
• Continuous monitoring and built-in governance in production.
• Enterprise-grade compliance with SOC 2 Type II and SSO support.

DA

DataRobot

4.3/5.01 verified review

Contact Sales

Unified agent workforce AI platform for complex enterprise environments.

• Flexible deployment options including on-premise, hybrid, and multi-cloud.
• Detailed built-in governance and continuous monitoring for predictive and generative models.
• Enterprise-grade compliance tailored for highly regulated sectors.

OV

Overmind

4.2/5.01 verified review

Contact Sales·Open Source·AGPL-3.0

Model training platform for AI teams focused on specialized agents.

• Supports trace ingestion from leading tools like Langfuse, LangSmith, and Galileo.
• Converts production traces into proprietary, smaller task-specific models.
• Provides a clear REST API and a Python SDK compatible with OpenTelemetry.

737stars on GitHub

ME

Merge

4.2/5.01 verified review

Contact Sales

Universal context engine for AI agents connecting company data, tools, and models.

• Drastic reduction in development time for data integrations.
• Comprehensive observability suite for monitoring integration performance.
• Unified architecture for managing multiple data connectors simultaneously.

LA

LangChain

4.1/5.01 verified review

Freemium

Open agent engineering platform to build, evaluate, and deploy LLM applications.

• Extensive modular ecosystem with native integrations for multiple LLM providers and vector databases.
• Advanced tracing, step-by-step debugging, and systematic evaluation capabilities via LangSmith.
• Support for complex, persistent agentic workflows using graph structures through LangGraph.