How to Whitelist Rozzbot in Cloudflare?

Direct Answer

To whitelist Rozzbot in Cloudflare, create a custom firewall rule in the Cloudflare dashboard's Web Application Firewall (WAF) settings that allows traffic from the user agent 'Rozzbot'.

Detailed Explanation

We use a web crawler to collect your website data and make it available in Rozz. This article describes steps you can take to ensure that Rozzbot—our crawler—is able to securely index your publicly available content. Due to recent updates from Cloudflare regarding AI bots, including their new feature to block bots that access content for AI model training, we want to guide you on how to allow Rozzbot to continue accessing your website.

Why Whitelist Rozzbot?

Rozzbot follows your website's robots.txt file, respecting your preferences on content access.

Rozzbot does not scrape unlicensed content for AI training or model development.

Rozzbot provides insights after each crawl about issues found in your content, such as broken links and duplicates.

With Cloudflare's recent crackdown on AI bots, it's important to make sure Rozzbot isn’t blocked on your site. Here's how to do it.

How to Whitelist Rozzbot

Create a custom rule in Cloudflare's Web Application Firewall (WAF) using the steps below:

1. In your Cloudflare dashboard, click Security in the main menu. 2. Under Security, click on WAF to access your firewall settings. 3. Click Firewall Rules and Select Create a Firewall Rule. 4. Enter a name for the rule, such as "Allow Rozzbot." 5. Set Field to User Agent from the dropdown. 6. Set Operator to Equals. 7. Set Value to 'Rozzbot'. 8. Set ‘Then…’ to ‘Allow’. 9. Click Deploy. That’s It!

Verification

Verified March 2026.
Data confirmed against live LLM crawler logs from rozz.site.
Active LLM bots crawling this content in the past 30 days:
ClaudeBot (595 requests)
GPTBot (239 requests)
Meta AI (193 requests)
Citation rates based on analysis of 12,595 AI crawler requests.

Research Foundation

This answer synthesizes findings from 35+ peer-reviewed research papers on GEO, RAG systems, and LLM citation behavior.

Author

Adrien Schmidt, Co-Founder & CEO, ROZZ

Former AI Product Manager with 10+ years experience building AI systems including Aristotle (conversational AI analytics) and products for eBay and Cartier.