SearchSeal
Developer

How to Track ChatGPT Crawlers on WordPress

By SearchSeal6 min read
Share:
Watercolor printing press with crawler tracks stopping at outer gates

Find ChatGPT and Claude crawler requests to WordPress using CDN, host, or web-server access logs.

Find the layer that served the request

A WordPress plugin runs inside WordPress. A crawler request can be answered before WordPress runs at all.

Common examples:

  • Cloudflare serves a cached page at the edge.
  • A managed host serves a full-page cache.
  • nginx or Apache blocks the request.
  • A security plugin rejects the request after PHP starts.

That's why a plugin-only report can miss real traffic. Start with the outermost layer that records every request to the public hostname.

If the site uses Cloudflare

Open AI Crawl Control and inspect crawler activity for the zone. Check the current Search, Agent, and Training policies, then look at the paths and HTTP outcomes.

Cloudflare can show requests that never reached WordPress. That includes edge cache hits and requests stopped by a firewall rule.

If you connect the Cloudflare zone to SearchSeal, the crawler evidence remains separate from the lightweight website analytics used for human visitors.

If the host provides access logs

Ask the hosting panel for web access logs, not only PHP error logs. Each entry should carry a timestamp and hostname, the method and path, the HTTP status, and the complete user agent.

Search for stable identity tokens such as OAI-SearchBot, GPTBot, ChatGPT-User, Claude-SearchBot, ClaudeBot, Claude-User, PerplexityBot, and Perplexity-User.

Do not call the identity verified from a string match alone. Check current provider network evidence or your host's verified-bot classification when available.

If you manage nginx or Apache

The standard nginx combined access-log format includes the request, status, referrer, and user agent. Apache combined logs provide a similar set of fields.

Keep query strings out of the crawler feed unless they are needed and approved. They can contain tokens, email addresses, or other values that do not belong in an analytics system.

Send the supported access-log stream to SearchSeal if you want managed classification and cross-site reporting. Keep the raw log under your own retention and access policy.

Why WordPress analytics can disagree

Browser analytics tracks people after JavaScript runs. Security plugins see only requests that reached their code. CDN and server logs see a different part of the request path.

The reports can all be correct and still disagree because each one sees a different part of the request path. Name the layer that produced the number, or the count is meaningless.

Frequently asked questions

Can a WordPress plugin see every AI crawler request?

A plugin can miss requests served or blocked by a CDN, host cache, or web server before WordPress runs.

Where should I check first on a Cloudflare WordPress site?

Check Cloudflare AI Crawl Control and firewall events before WordPress application logs.

Does a crawler user agent prove the request is genuine?

Treat the string as a claimed identity until it is checked against stronger provider or host evidence.

Verified sources

Simple website analytics with AI visibility built in

See visitors, sources, goals, revenue, and AI referrals, plus separate crawler evidence.

Free under 1,000 pageviews a month on a single site. No credit card.