# ARBR > ARBR observes LLM workloads to surface model-switching opportunities and builds evaluation datasets from representative traffic. It lets teams compare candidates on cost, quality, and latency, then roll out approved changes under human control. Th... - Type: AI tool - Page: https://completeaitraining.com/ai-tools/arbr/ - Published: 2026-08-25 - Updated: 2026-09-04 - Author: Jeroen Erne - Jobs: IT and development, Operations - Topics: Developer Tools, Open Source, AI-Detection, Analytics, Research - Reading time: 2 min ### About ARBR ARBR is an open-source, self-hosted control layer that sits between applications and AI models. It presents an OpenAI-compatible endpoint so teams can route, govern, observe, evaluate, and deploy across multiple providers without changing existing code. The tool is MIT-licensed and provider-neutral, built to run as a standalone gateway or on top of LiteLLM. ### Review ARBR tackles a specific production problem: knowing which workloads can safely move to a different model, with evidence to back the decision, and confirming the result after rollout. It observes traffic patterns, surfaces model-switching opportunities, and builds evaluation datasets from representative requests. The routing logic follows explicit, auditable rules - models that are explicitly pinned stay pinned, while workloads marked "auto" follow only the policies the team has enabled. #### Key Features - Observes live workloads and identifies which ones are candidates for switching to a different model - Builds evaluation datasets from actual traffic and compares candidate models across quality, cost, latency, format adherence, and critical failures - Supports a staged deployment pipeline: shadow testing against mirrored traffic, guarded canary with automatic rollback, and human-promoted full rollout - Automatic canary monitoring that recomputes candidate-vs-baseline metrics every 5 minutes over a trailing 60-minute window, with rollback triggered on guardrail breaches - Demo mode that lets users explore the full workflow without adding a provider key #### Pricing and Value ARBR is free and open source under the MIT License. The team has not published any paid tiers, enterprise plans, or managed hosting options. Self-hosting is the only deployment model described in the current documentation. #### Pros - Keeps the final model-change decision human-controlled, with an audit trail recording who promoted a swap - Rollback triggers combine production telemetry (error rate, p95 latency, cost savings) with one eval-based metric, rather than relying solely on LLM-judged scores - OpenAI-compatible endpoint means applications don't need rewrites to connect - Works as a standalone gateway or layers above an existing LiteLLM setup - MIT license imposes no restrictions on modification or commercial use #### Cons - Requires self-hosting, so teams without infrastructure to run Docker containers will find setup heavier than a managed service - Provider integrations are limited to what LiteLLM supports; teams using niche or proprietary APIs may hit coverage gaps - Not well suited for teams that don't run LLM workloads in production yet - the tool's evaluation and canary pipeline assumes live traffic to observe and compare ARBR fits teams already running LLM workloads in production who want a structured, evidence-based process for model changes without vendor lock-in. The staged rollout pipeline gives operators room to test swaps on shadow and canary traffic before committing. Organizations that prefer a fully managed SaaS control plane, or that don't have production traffic to observe yet, won't find those use cases addressed in the current release. --- This is the Markdown form of https://completeaitraining.com/ai-tools/arbr/. The whole site is mapped at https://completeaitraining.com/llms.txt.