Bot Management In Search 2026 – Chris Green youtube.com

In this expert session from Search With Sean Live 2026, Chris Green breaks down the modern challenges of bot management in an era dominated by AI. Moving beyond simple ‘block or allow’ tactics, Chris explores the nuances of managing diverse traffic—from search engine crawlers and AI training bots to autonomous agents—and why a static approach is no longer sufficient.

Key takeaways include:

Decoding Bot Taxonomy: How to distinguish between different types of AI crawlers (e.g., OpenAI’s specific bot signals) versus general scrapers.

Beyond Robots.txt: Why relying solely on robots.txt is an outdated strategy and how to use CDN-level controls for more robust enforcement.

Strategic Policy: How to build a defensible, flexible bot policy that protects your proprietary content while ensuring your site remains visible in AI-driven search results and agentic workflows.

Analytics Integrity: Understanding the risks of agentic browsing to your data and how to prevent ‘bot contamination’ in your reporting.