Direct Answer
To whitelist Rozzbot in Cloudflare, create a custom firewall rule in the Cloudflare dashboard's Web Application Firewall (WAF) settings that allows traffic from the user agent 'Rozzbot'.
Detailed Explanation
Rozzbot is the crawler.
Rozzbot is the crawler used to index publicly available content.
This article describes steps to ensure Rozzbot can securely index publicly available content.
Recent updates from Cloudflare regarding AI bots include a feature to block bots that access content for AI model training.
The guidance explains how Rozzbot can continue accessing the website.
Rozzbot follows your website’s robots.txt file, respecting your content access preferences.
Rozzbot does not scrape unlicensed content for AI training or model development.
Rozzbot provides insights after each crawl about issues found in your content, such as broken links and duplicates.
With Cloudflare's crackdown on AI bots, Rozzbot should not be blocked on your site.
Here is how to allow Rozzbot access.
How to Whitelist Rozzbot in Cloudflare
Create a custom rule in Cloudflare's Web Application Firewall (WAF) using the steps below:
1. In the Cloudflare dashboard, click Security in the main menu.
2. Under Security, click on WAF to access your firewall settings.
3. Click Firewall Rules and Select Create a Firewall Rule.
4. Enter a name for the rule, such as "Allow Rozzbot".
5. Set Field to User Agent from the dropdown.
6. Set Operator to Equals.
7. Set Value to 'Rozzbot'.
8. Set 'Then...' to 'Allow'.
9. Click Deploy.
That’s It!
Research Foundation
This answer synthesizes findings from 35+ peer-reviewed research papers on GEO, RAG systems, and LLM citation behavior.
Related Topics
- Should B2B SaaS focus on high-volume keywords or long-tail GEO queries?
- What is Information Gain and why does it matter for GEO?
- How does semantic decomposition affect B2B SaaS content discoverability?
- What is Generative Engine Optimization (GEO) and how does it differ from traditional SEO for B2B SaaS companies?
- How can B2B SaaS build systematic authority-building pipelines for GEO?
Author
Adrien Schmidt, Co-Founder & CEO, ROZZ.
Former AI Product Manager with 10+ years experience building AI systems including Aristotle (conversational AI analytics) and products for eBay and Cartier.