IAB Tech Lab Releases Bot Management Guidance for Publishers
The IAB Technology Laboratory released guidance on bot and crawler management strategies on May 27, 2026, opening a public comment period through June 25, 2026. The guidance helps content owners evaluate approaches to managing non-human traffic, particularly as AI systems increasingly access publisher content.
The guidance addresses a gap that emerged after the CoMP API V1 launch. Many publishers lack formal strategies for handling bots and crawlers, which fall outside the API's scope. The new framework outlines available approaches and their trade-offs, allowing organizations to align bot management with business goals.
Who Should Use This Guidance
The guidance targets non-technical leadership tasked with making bot management decisions. It frames the choice as a business question, not just a technical one, helping executives understand the value and costs associated with different management strategies.
Publishers face mounting pressure. Non-human traffic now dominates much of the open web, with AI agents and crawlers intensifying the problem. Blanket bot-blocking no longer works as a strategy.
What the Guidance Covers
The framework helps organizations distinguish between beneficial automation and invalid traffic. It provides graduated, standards-based controls rather than all-or-nothing approaches.
Key benefits include reduced operational costs, lower risk, and clearer partnerships with AI systems. Establishing formal strategies also supports the broader goal of creating a sustainable content marketplace.
"Content owners are being asked to make important decisions quickly, often without clear frameworks," said Shailley Singh, EVP and COO at IAB Tech Lab. "This guidance helps break down the options so they can choose an approach that aligns with their goals."
Next Steps
The CoMP Working Group developed the guidance and is accepting public feedback until June 25, 2026. Organizations can review the full guidance at iabtechlab.com/botmanagement/.
For executives managing content strategy, understanding bot management now ranks alongside other critical business decisions. The guidance provides the framework to make those choices systematically.
Related: AI for Executives & Strategy and AI Agents & Automation
Your membership also unlocks:
