The Crawler Logs

Insights

Total LLM bot requests: 22,679.
Citation events: 9,250.
Days of data: 94.
Log entries: 10.

The Crawler Logs document real AI crawler behavior on production AI sites built by ROZZ. It uses CloudFront access logs and User-Agent classification to track how GPTBot, BingBot, ClaudeBot, ChatGPT-User, PerplexityBot, and other LLM bots discover, crawl, and cite GEO-optimized content. Across 94 days of observation on rozz.genymotion.com, 22,679 total LLM bot requests were recorded and 9,250 citation events occurred—real users receiving Genymotion content in AI conversations.

Three bots, three structural triggers

GPTBot — January 7 — 547 requests
Trigger: Sitemap discovery
Description: Found site via sitemap, mass-crawled content

PerplexityBot — March 10 — 511 requests
Trigger: Index page revamp
Description: Product description + topic directory

ClaudeBot — March 20 — 577 requests
Trigger: Per-topic sitemapindex
Description: Topic-organized sitemaps + simplified robots.txt

There is the note that content remains the same with the same Schema.org markup and the same topic taxonomy. All three bots arrived on their own schedules.

Log Entries

Entry #10 — Apr 8, 2026

Title: Three Citation Bots. Ten Articles. What We Learned Building an AI Site.
Summary: Three AI platforms—ChatGPT-User, Claude-User, Perplexity-User—now retrieve content from the same AI site during live user sessions.
rozz.genymotion.com · Jan 8 – Apr 8, 2026 · 22,679 total LLM bot requests (90-day cumulative)
9,250 citation requests across 90 days
Three months ago there were zero
Per perplexity: Perplexity-User first appearance (Apr 5) — all three major citation pipelines now active
376 commits across 90 days; structural fixes (sitemaps, robots.txt, topic taxonomy) drove every breakthrough
Q&A pages drive 66–75% of citations; CLI runbooks open a developer-tool sales channel

Entry #9 — Mar 31, 2026

Title: Selling in Claude Code: From Pricing to Implementation in 10 Seconds with an AI Site
Summary: A developer asked Claude Code how much Genymotion costs. Ten seconds later, it was showing them how to set it up. Both answers came from our AI site.
Claude-User: 14 requests; Claude Code terminal sessions: 12
rozz.genymotion.com · Mar 24–31, 2026 · 1,612 total LLM bot requests
Claude pipeline: ClaudeBot crawl → 5 days → Claude-User live retrieval
8x increase
ClaudeBot: 0 Q&A pages, 0 Claude-SearchBot traffic—still in discovery mode after 3 weeks

Entry #8 — Mar 24, 2026

Title: ClaudeBot: 958 requests
Summary: ClaudeBot made 958 requests in one week, up from 123 the week before.
503 GEO pages and 162 Q&A pages—the largest ClaudeBot crawl since December
Six hours after deploying per-topic sitemaps, ClaudeBot came back
rozz.genymotion.com · Mar 17–24, 2026 · 2,446 total LLM bot requests
ClaudeBot 8x increase triggered the day per-topic sitemapindex was deployed
March 20: 577 ClaudeBot requests in a single day—largest since December
Every major AI crawler has now completed a deep indexing event on the AI site

Entry #7 — Mar 17, 2026

Title: PerplexityBot: 511
Summary: PerplexityBot made 511 requests in one week, up from 42 the week before.
PerplexityBot: 511 requests; 172 Q&A pages and 256 GEO pages crawled
rozz.genymotion.com · Mar 10–17, 2026 · 2,532 total LLM bot requests
PerplexityBot 12x increase (42 → 511) triggered the day after index page redesign
84% of PerplexityBot requests hit content pages—Q&A and GEO pages, no homepage
ClaudeBot: 0 Q&A pages, 0 Claude-SearchBot traffic—still in discovery mode after 3 weeks

Entry #6 — Mar 10, 2026

Title: Sessions
Summary: What the AI Site reveals about AI-mediated discovery
ChatGPT-User made 681 visits in one week
By grouping visits into sessions using IP hashes and timing, 168 sessions were reconstructed
rozz.genymotion.com · Mar 3–10, 2026 · 681 ChatGPT-User visits
4.6 pages fetched per turn—ChatGPT-User verifies across multiple sources, not just one
28% of sessions hit only the index page and stopped—led to index redesign
30% of sessions are multi-turn: actual ChatGPT conversations reconstructed
83%

Entry #5 — Mar 3, 2026

Title: The platforms that crawl your AI site the most cite you the most (except one)
Summary: ChatGPT cites Genymotion 83% of the time; Claude 21%; Perplexity 17%; Gemini 4%.
rozz.genymotion.com · Feb 24 – Mar 3, 2026 · 24 queries × 4 platforms
ChatGPT: 83% citation rate, up from 14% before the AI site launched
ClaudeBot jumped 24x (21 to 505 requests)—reads topic taxonomy, not individual pages
Gemini does not crawl the AI site at all

Entry #4 — Feb 24, 2026

Title: Bing just found Genymotion: 1,556 BingBot requests and what it means
Summary: BingBot made 1,556 requests this week—more than any other bot, including ChatGPT-User.
rozz.genymotion.com · Feb 17–24, 2026 · 3,188 total requests
BingBot: 1,556 requests—largest single bot category
ChatGPT-User citations: 1,329, up from 1,077 the prior week (+23%)
Six platforms now crawling: OpenAI, Microsoft, Anthropic, Meta, ByteDance, Perplexity

Entry #3 — Feb 17, 2026

Title: 3x Week-Over-Week: What Sustained ChatGPT Citation Growth Looks Like
Summary: ChatGPT citations hit 1,077 this week—3x the 345 reported last week.
Six-week trajectory: 42 → 345 → 1,077
Q&A pages drive 66% of all citations
rozz.genymotion.com · Feb 10–17, 2026 · 2,606 total requests
1,077 ChatGPT citations in 7 days—3x the previous week’s 345
Q&A pages remain the citation driver: 66% of all ChatGPT requests
PerplexityBot activity up 5x: 33 requests vs 7 the previous week

Entry #2 — Feb 10, 2026

Title: 16x Citation Growth in 7 Days: What ChatGPT Users Are Actually Asking
Summary: ChatGPT citations grew from 7 to 116 in one week.
345 citation events, 161 unique sessions, and 75% of requests hitting Q&A pages—not traditional content
rozz.genymotion.com · Feb 2–9, 2026 · 2,195 total requests
16x daily citation growth: from 7 on Feb 2 to 116 on Feb 9
75% of ChatGPT requests landed on Q&A pages, not traditional content
Q&A pages cited 10x more than traditional GEO content pages

Entry #1 — Feb 3, 2026

Title: 547 Requests in One Day: What Happens When GPTBot Discovers Your Mirror Site
Summary: GPTBot made 547 requests in a single day—47% of all training bot activity in 30 days.
Three weeks later, ChatGPT users were receiving content in their conversations
rozz.genymotion.com · Jan 3–Feb 2, 2026 · 1,280 total requests
GPTBot made 547 requests on January 7—47% of 30-day training activity in one day
42 citation events recorded; concentrated on high-intent pages (requirements, compatibility)
~3 weeks from major crawl to first ChatGPT citations

---

Methodology

Data source: CloudFront access logs for rozz.genymotion.com.
Bot classification based on User-Agent strings.
Training bots (GPTBot, ClaudeBot) are distinguished from search index bots (OAI-SearchBot) and citation events (ChatGPT-User).
Citation events represent real user conversations where ChatGPT retrieved and cited mirror site content.
All data is from a single production mirror site; results may vary by domain, content volume, and vertical.

---

Get Your Own Crawler Logs

ROZZ builds mirror sites with full CloudFront logging. See exactly which AI bots crawl your content, when they crawl it, and when citations begin.

$997/month | Mirror site + crawler analytics

→ Book a call | → See how it works | → rozz@rozz.site

---

Author

Adrien Schmidt, CEO, ROZZ

Serial tech entrepreneur with 10+ years experience building AI systems including Aristotle (conversational AI analytics) and products for eBay and Cartier.