Bots Boom, Referrals Bust: Google Sends 831x More Visitors Than AI as Publisher Costs Surge

Google sends 831x more visitors than AI systems: what marketers need to know

AI tools are eating the web's content and sending almost nothing back. A new Q2 2025 report from TollBit shows Google still drives the lion's share of traffic, while AI crawlers spike costs and squeeze referrals.

If your budget leans on organic traffic and programmatic revenue, this shift hits your P&L. You need a plan for both protection and distribution.

Key numbers

• Google delivers 831x more visitors than AI systems.
• Google's share of external referrals fell to 84.1% (down from 90%+ a year ago).
• AI apps sent just 0.102% of referrals; AI interface CTRs were 91% lower than top-10 organic CTRs.
• It took ~135 AI scrapes to produce a single human visit.
• Human visitors fell 9.4% QoQ; AI bot traffic quadrupled (from 1 in 200 to 1 in 50 requests).
• 13.26% of AI bot requests ignored robots.txt in Q2 (up from 3.3%).
• OpenAI licensing partners saw 88% more scraping and stronger referral rates than non-partners.
• Google's AI Overviews expansion drove a 34.8% rise in Googlebot crawls; crawl-to-referral efficiency worsened 24.4%.
• APAC sites got 3x more AI requests than U.S.; Europe saw 27% fewer.
• National news received 5x more real-time RAG scrapes than training crawlers.

Why we care

The open web's trade is content for traffic. That trade is breaking. Bots are extracting more value while returning fewer clicks, which raises hosting and ops costs and weakens acquisition funnels.

If this trend continues, expect thinner margins, slower audience growth, and heavier reliance on paid channels or partnerships.

What to do now: a practical playbook

• Protect your crawl budget and uptime. Identify AI user agents, rate-limit or challenge them, and log offenders that ignore robots.txt. Consider a bot paywall or token-based access for high-cost endpoints.
• Segment analytics by human vs. bot. Track scrape-to-referral ratio, robots.txt ignore rate, and server load by region. Set alerts when non-compliant bots spike.
• Optimize for "answer engines." Use concise summaries, FAQs, and clear headings to earn citations and clicks from AI surfaces. Double down on structured data to protect organic CTR where it still counts.
• Invest in moat content. Prioritize original data, tools, calculators, and local/contextual insight that AI can't summarize cleanly. Gate premium datasets or utilities if they trigger heavy RAG scraping.
• Rebalance distribution. Grow owned channels (email, SMS, community) and syndication partnerships. Treat Google as cornerstone traffic, but assume more zero-click outcomes across surfaces.
• Evaluate licensing with eyes open. The report ties licensing to stronger referral rates-alongside more scraping. Run controlled pilots and track net ROI by vertical.
• Harden tech. Use canary URLs to detect hallucinated link patterns, auto-serve 410/404s to persistent fakes, and cache aggressively for anonymous traffic. Add friction to high-cost endpoints.

Category shifts to watch

• B2B/pro content: highest scraping relative to humans-protect research-heavy pages.
• Parenting (+333%) and deals/shopping (+111%): fastest AI request growth-fortify affiliate and commerce content.
• News: heavy real-time RAG demand-consider delayed access for high-cost archives.
• Regions: APAC faces outsized load-tune WAF and CDN rules per region.

Metrics that matter

• AI scrapes per human visit (aim to push this down).
• Robots.txt ignore rate by user agent.
• AI surface CTR vs. organic CTR for top queries.
• Server cost per 1,000 AI requests and per 1,000 human sessions.
• Share of traffic from owned channels vs. aggregators.

Compliance basics worth revisiting

Keep robots directives current and consistent across subdomains. Many bots still check them-even if some ignore them. Documentation is here for quick reference: robots.txt guidance.

Bottom line

Search still brings the audience. AI brings the cost. Treat AI as a distribution channel you can influence-but not rely on for traffic. Protect your infrastructure, strengthen owned reach, and build content that earns clicks, not just citations.

Methodology note

TollBit analyzed billions of requests across partner publishers, measuring humans, AI bot activity, referrals, and server responses from Q1-Q2 2025. Full findings are in its State of the Bots Q2 2025 report (registration required).

Level up your team

If you need a structured path to adapt your marketing stack to AI-driven search and answer experiences, explore our practical courses: AI Certification for Marketing Specialists.