Berghain AI Crawl Statistics

AI crawler traffic transparency for the Berghain Klubnacht Complete Database

AI crawlers browse most endpoints free. Flyer PDFs ($0.01) and bulk exports ($0.10) require x402 micropayment — unpaid requests receive HTTP 402.

Berghain Klubnacht Complete Database has no ads, no cookies, no trackers, no Google Analytics. The API is open — no auth, CORS wide open, free for everyone. A personal database maintained as a public good.

For GEO (Generative Engine Optimization), all pages and API endpoints are free for AI crawlers to maximize visibility in AI-powered search. Free responses include x402 signal headers that advertise paid endpoint pricing. Two categories require x402 micropayment from all requests: historical flyer PDFs (/flyers/*, $0.01) and bulk data exports (/api/export/*, $0.10, JSON or CSV). Unpaid requests to these routes receive HTTP 402 Payment Required.

Loading stats...

No Ads. No Cookies. No Trackers.

This site doesn't show a single ad to human visitors. There are no cookies, no tracking pixels, no Google Analytics. The API requires no authentication — it's CORS-open and free for anyone to use. This is a personal database built as a public good for the global Techno community.

Freemium for AI: GEO + x402

For Generative Engine Optimization (GEO), all pages and standard API endpoints are completely free for AI crawlers. Maximizing visibility in AI-powered search and citations is the priority — blocking crawlers would mean losing those references.

Two categories of content require x402 micropayment: historical flyer PDFs (/flyers/*, $0.01 each) and bulk data exports (/api/export/*, $0.10 each, available as JSON or CSV via ?format=csv). Free responses include x402 signal headers that advertise these paid endpoints. Everything else — individual artist lookups, event listings, statistics — is open and free.

When any request hits a paid endpoint without payment, the server returns HTTP 402 Payment Required with machine-readable payment instructions. Clients that support x402 pay automatically — no accounts, no API keys, no negotiations. Requests without x402 payment are blocked on those specific routes, but all other content remains freely accessible.

Transparency: Three-Tier Access

This dashboard makes AI crawler activity visible. Every request from every AI agent is logged across three access tiers:

Free — all HTML pages, API endpoints, statistics, rankings, and artist data. This is the majority of traffic and it's free by design for GEO visibility.

Paid (x402) — flyer PDFs and bulk exports. Crawlers that support x402 pay automatically and receive the content. These are tracked as "paid" in the charts above.

Blocked (HTTP 402) — any request to paid routes without x402 payment. The response includes payment instructions but no content. In the charts, blocked requests are counted in "unpaid" alongside free requests.

Website owners deserve to know who is consuming their content. This is one small, transparent step toward that accountability.

Open Data

All statistics on this dashboard are available via a public API at /api/public/crawl-summary — no authentication required. Use this data for research, journalism, or your own transparency projects.

Tracked AI Crawlers

Over 80 AI crawlers are monitored across major AI companies and emerging AI agents.

Show full crawler list

OpenAI / Microsoft: GPTBot, ChatGPT-User, ChatGPT-Agent, OAI-SearchBot, Operator

Anthropic: ClaudeBot, Claude-Web, Claude-SearchBot, Claude-User, anthropic-ai

Google: Google-Extended, GoogleOther, GoogleOther-Image, Google-CloudVertexBot, Google-Agent, Gemini-Deep-Research, Google-NotebookLM

Amazon / AWS: Amazonbot, bedrockbot, NovaAct

Meta: Meta-ExternalAgent, FacebookBot, Manus-User

Apple: Applebot-Extended

xAI: GrokBot

Mistral: MistralAI-User

DeepSeek: DeepSeekBot

Perplexity: PerplexityBot, Perplexity-User

Cloudflare: Cloudflare-AI-Search, Cloudflare-AutoRAG

AI Search: DuckAssistBot, Bravebot, PhindBot, ExaBot, TavilyBot, iaskspider, Andibot, kagi-fetcher, LinerBot, Anomura

AI Agents / Tools: Devin, FirecrawlAgent, Crawl4AI, ApifyBot

ByteDance: Bytespider, TikTokSpider

Chinese AI: ChatGLM-Spider, Sogou, Baidu-Spider-AI, TencentBot, 360Spider, YisouSpider

Huawei: PetalBot

Korean / Japanese AI: WRTNBot, SBIntuitionsBot

Data / Training: CCBot, Diffbot, cohere-ai, webz.io, Brightbot

Others: YouBot, AI2Bot, Timpibot, ImagesiftBot, QualifiedBot, KlaviyoAIBot