Databricks Agent Bricks Simplifies Enterprise AI Development with TAO and ALHF
Databricks has launched Agent Bricks, a platform that streamlines the creation of domain-specific AI agents for enterprises. It reduces the complexity of agent development by letting teams focus on defining the agent's goals and providing natural language feedback to guide quality. The platform then automatically handles evaluation and optimization without manual intervention.
Agent Bricks automates the entire workflow, including generating task-specific evaluation suites, creating synthetic data similar to customer data, and applying various optimization techniques to improve agent performance.
Four-Step Automated Workflow
- Task Declaration: Users begin by selecting their objective, describing in natural language what they want the agent to achieve, and connecting relevant data sources.
- Automatic Evaluation: The platform builds custom evaluation benchmarks, which can include synthetic data generation and custom large language model (LLM) judges tailored to the task.
- Automatic Optimization: Agent Bricks intelligently explores and combines optimization methods such as prompt engineering, model fine-tuning, reward models, and test-adaptive optimization (TAO) to boost quality.
- Cost and Quality Balancing: The system ensures agents are both effective and cost-efficient, giving users the option to prioritize cost or quality. Often, this approach delivers better results at a lower cost than typical DIY methods.
Agent Learning from Human Feedback (ALHF)
One key innovation behind Agent Bricks is Agent Learning from Human Feedback (ALHF). Traditional feedback methods—such as simple thumbs up or down—lack the detail needed to fine-tune complex agent behavior. Additionally, packing all instructions into a single large prompt is brittle and doesn’t scale well for advanced systems.
ALHF improves this by using rich natural language guidance and algorithms that translate that guidance into specific technical adjustments. These may include refining retrieval algorithms, enhancing prompts, filtering vector databases, or modifying agent patterns.
Test-time Adaptive Optimization (TAO)
TAO is a new model tuning method introduced by Databricks that requires only unlabeled usage data. It enables enterprises to improve model quality and reduce costs by leveraging test-time compute and reinforcement learning. This method scales with a compute budget, avoiding the need for expensive human labeling.
TAO can outperform traditional fine-tuning and bring affordable open-source models like Llama close to the quality of expensive proprietary models such as GPT-4o and o3-mini.
Mosaic AI Agent Evaluation
Databricks also offers Mosaic AI Agent Evaluation, a tool that assesses the quality, cost, and latency of AI applications, including retrieval-augmented generation (RAG) and agent chains. It identifies quality issues and their root causes throughout development, staging, and production phases, logging all evaluation metrics in MLflow Runs.
This tool maintains consistency between development and production environments, allowing teams to iterate and deploy agentic applications efficiently. The key difference is the availability of ground-truth labels in development, enabling more detailed quality metrics.
Agent Bricks Use Cases Across Industries
- Information Extraction Agent: Converts unstructured documents like emails and PDFs into structured data fields. For example, retail companies can extract product details from supplier documents regardless of format complexity.
- Knowledge Assistant Agent: Provides fast, accurate answers based on enterprise data. Manufacturing technicians can query SOPs and manuals for instant, cited responses.
- Multi-Agent Supervisor: Coordinates multiple agents and tools to handle tasks such as intent detection, document retrieval, and compliance checks. Financial services can orchestrate agents across different spaces effectively.
- Custom LLM Agent: Tailors text generation for industry-specific purposes, helping marketing teams produce content aligned with brand guidelines.
A Shift Toward Declarative AI Development
Matei Zaharia, CTO at Databricks and a computer science professor, emphasized that Agent Bricks is the result of a collaborative effort between engineering and research teams. The platform’s declarative approach allows domain experts to improve AI systems without needing deep technical AI infrastructure knowledge.
This could change how enterprises develop and maintain AI agents by making the process more accessible and integrated with business goals.
Learn More and Explore Agent Bricks
For IT and development professionals interested in the technical details and multi-agent system development, Databricks offers a session from their Data AI Summit on building multi-agent systems with both structured and unstructured data. A video demonstration showcases Agent Bricks’ automated optimization workflow and real-world applications.
Explore further AI development training and resources at Complete AI Training.
Your membership also unlocks:
