GPTBot, OAI-SearchBot, and ChatGPT-User

GPTBot is OpenAI's training crawler. OAI-SearchBot supports ChatGPT Search. ChatGPT-User fetches pages for some user-initiated actions. They are separate identities with separate purposes. A single allow or block rule does not express the same policy for all three.

Quick comparison

Identity	Purpose	Automatic crawl	robots.txt
OAI-SearchBot	Surface sites in ChatGPT Search	Yes	Use its named group
GPTBot	Collect content that may be used to train foundation models	Yes	Use its named group
ChatGPT-User	Fetch a page for some user actions in ChatGPT or a custom GPT	No	Rules may not apply

OpenAI documents these identities on its crawler reference. The version numbers inside full user-agent strings may change. Match the stable product token, not a hard-coded browser or bot version.

OAI-SearchBot is the search crawler

OAI-SearchBot is the control for automatic crawling used to surface websites in ChatGPT Search. OpenAI says sites that opt out will not be shown in ChatGPT search answers, although a site may still appear as a navigational link.

Allowing this crawler does not prove that a page will be indexed, selected, quoted, or cited. It only removes one policy barrier. The request can still fail at a CDN, WAF, origin, redirect, or application route.

OpenAI publishes the crawler's current network ranges at openai.com/searchbot.json. Use that feed when you need stronger verification than a user-agent string.

GPTBot is the training crawler

GPTBot is used to crawl content that may be used to train OpenAI's generative AI foundation models. Blocking GPTBot is a training preference. It is not the ChatGPT Search opt-out.

OpenAI treats the OAI-SearchBot and GPTBot settings independently, so a publisher can allow discovery in ChatGPT Search while disallowing training. See "Allow search, block training" below. OpenAI publishes GPTBot ranges at openai.com/gptbot.json.

ChatGPT-User is a user-triggered fetcher

ChatGPT-User can appear when a person asks ChatGPT or a custom GPT to visit a page. It is not used for automatic web crawling and it does not determine whether a page can appear in ChatGPT Search.

OpenAI states that robots.txt rules may not apply because the action was initiated by a user. Adding this group does not guarantee that every user-triggered request will be allowed or blocked:

User-agent: ChatGPT-User
Allow: /

Treat this as an expressed preference, then enforce sensitive access at the application or network layer. OpenAI publishes current ranges at openai.com/chatgpt-user.json.

Choose a policy by purpose

Allow search, block training

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /

This is the common split when a site wants potential inclusion in ChatGPT Search without allowing GPTBot training crawl.

Allow both automatic crawlers

User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Allow: /

You can still disallow private or operational paths inside each named group. Copy the path rules into both groups if both identities need the same result.

Block both automatic crawlers

User-agent: OAI-SearchBot
Disallow: /

User-agent: GPTBot
Disallow: /

This covers the two automatic OpenAI crawler identities. It is not authentication and it does not secure a public page from browsers, proxies, or uncooperative clients.

Verify the request, not just the name

An HTTP User-Agent header is a claim supplied by the requester. Any client can send GPTBot as text. For higher-confidence classification, combine several fields:

1. Preserve the complete user-agent header.
2. Match the stable product token case-insensitively.
3. Compare the source IP with the current vendor-published ranges.
4. Record the requested host, path, method, time, and HTTP outcome.
5. Keep the raw evidence so a classification can be corrected later.

Do not copy IP addresses from a blog post into a permanent firewall rule. Fetch the vendor's JSON source and retain when it was checked. Network ranges change.

Read the outcome correctly

A 200 request proves that your infrastructure served a response to that request. A 403 proves that some layer refused it. A 404 may reveal a stale link or a route the crawler discovered elsewhere. None of those events proves that a model used the content.

SearchSeal records identified AI crawler requests from connected website infrastructure, including requested paths and HTTP outcomes when available. It does not turn a request into an indexing or citation claim.

Debug in this order

1. Policy: fetch the public /robots.txt for the exact host.
2. Identity: inspect the raw user agent and source network evidence.
3. Edge: check CDN, bot management, and WAF decisions.
4. Origin: inspect redirects, status codes, and rendering failures.
5. Interpretation: report the request as observed evidence, not proof of use.

For a wider policy template, read robots.txt for AI crawlers. For the cross-vendor identity list, use the AI crawler user-agent reference.