Berghain AI Crawl Statistics

No Ads. No Cookies. No Trackers.

This site doesn't show a single ad to human visitors. There are no cookies, no tracking pixels, no Google Analytics. The API requires no authentication — it's CORS-open and free for anyone to use. This is a personal database built as a public good for the global Techno community.

Human users have never been charged a cent. Yet AI companies routinely send crawlers to extract this carefully structured data — feeding it into training datasets and search results without asking, attributing, or paying.

The Third Option

You can block AI crawlers via robots.txt. But that also means losing visibility in AI-powered search results and citations. The choice between "block everything" and "give it all away for free" is a false dilemma.

This site chose a third option: charge AI crawlers a fair micropayment through the HTTP specification itself. The x402 protocol turns HTTP 402 Payment Required — a status code reserved since 1997 — into a real, automated payment flow using USDC stablecoin on Solana.

When an AI crawler hits a paid endpoint, the server returns 402 with machine-readable payment instructions. Agents that support x402 pay automatically — no accounts, no API keys, no negotiations. It's permissionless, instant, and built on open standards.

Pricing is minimal: $0.01–$0.10 per request. The point isn't to make money. It's to establish the principle that structured data has value, and the exchange between creators and AI systems should be fair and automatic.

Transparency: Who Pays and Who Doesn't

This dashboard makes AI crawler behavior visible. Every request from every AI agent is logged and categorized — you can see exactly which companies pay for data access and which just take it.

The "paying" vs "blocked" breakdown above is real production data, updated daily. Most AI crawlers don't support x402 yet. They hit the paywall, receive a 402, and leave empty-handed. A few have started paying. This dashboard tracks that evolution in real time.

Website owners deserve to know who is consuming their content. AI companies should be accountable for how they acquire training data. This is one small, transparent step toward that accountability.

Open Data

All statistics on this dashboard are available via a public API at /api/public/crawl-summary — no authentication required. Use this data for research, journalism, or your own transparency projects.

Tracked AI Crawlers

Over 80 AI crawlers are monitored across major AI companies and emerging AI agents.