# GPTBot, OAI-SearchBot, and ChatGPT-User

> How GPTBot, OAI-SearchBot, and ChatGPT-User differ, which robots.txt rules apply, and how to verify requests in server logs.

Published: 2026-07-20
Category: Developer
Reading time: 4 min read

**GPTBot is OpenAI's training crawler. OAI-SearchBot supports ChatGPT Search. ChatGPT-User fetches pages for some user-initiated actions.** They are separate identities with separate purposes. A single allow or block rule does not express the same policy for all three.

## Quick comparison

| Identity | Purpose | Automatic crawl | robots.txt |
| --- | --- | --- | --- |
| `OAI-SearchBot` | Surface sites in ChatGPT Search | Yes | Use its named group |
| `GPTBot` | Collect content that may be used to train foundation models | Yes | Use its named group |
| `ChatGPT-User` | Fetch a page for some user actions in ChatGPT or a custom GPT | No | Rules may not apply |

OpenAI documents these identities on its [crawler reference](https://developers.openai.com/api/docs/bots). The version numbers inside full user-agent strings may change. Match the stable product token, not a hard-coded browser or bot version.

## OAI-SearchBot is the search crawler

`OAI-SearchBot` is the control for automatic crawling used to surface websites in ChatGPT Search. OpenAI says sites that opt out will not be shown in ChatGPT search answers, although a site may still appear as a navigational link.

Allowing this crawler does not prove that a page will be indexed, selected, quoted, or cited. It only removes one policy barrier. The request can still fail at a CDN, WAF, origin, redirect, or application route.

OpenAI publishes the crawler's current network ranges at [openai.com/searchbot.json](https://openai.com/searchbot.json). Use that feed when you need stronger verification than a user-agent string.

## GPTBot is the training crawler

`GPTBot` is used to crawl content that may be used to train OpenAI's generative AI foundation models. Blocking GPTBot is a training preference. It is not the ChatGPT Search opt-out.

OpenAI treats the OAI-SearchBot and GPTBot settings independently, so a publisher can allow discovery in ChatGPT Search while disallowing training. See "Allow search, block training" below. OpenAI publishes GPTBot ranges at [openai.com/gptbot.json](https://openai.com/gptbot.json).

## ChatGPT-User is a user-triggered fetcher

`ChatGPT-User` can appear when a person asks ChatGPT or a custom GPT to visit a page. It is not used for automatic web crawling and it does not determine whether a page can appear in ChatGPT Search.

OpenAI states that robots.txt rules may not apply because the action was initiated by a user. Adding this group does not guarantee that every user-triggered request will be allowed or blocked:

```
User-agent: ChatGPT-User
Allow: /
```

Treat this as an expressed preference, then enforce sensitive access at the application or network layer. OpenAI publishes current ranges at [openai.com/chatgpt-user.json](https://openai.com/chatgpt-user.json).

## Choose a policy by purpose

### Allow search, block training

```
User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Disallow: /
```

This is the common split when a site wants potential inclusion in ChatGPT Search without allowing GPTBot training crawl.

### Allow both automatic crawlers

```
User-agent: OAI-SearchBot
Allow: /

User-agent: GPTBot
Allow: /
```

You can still disallow private or operational paths inside each named group. Copy the path rules into both groups if both identities need the same result.

### Block both automatic crawlers

```
User-agent: OAI-SearchBot
Disallow: /

User-agent: GPTBot
Disallow: /
```

This covers the two automatic OpenAI crawler identities. It is not authentication and it does not secure a public page from browsers, proxies, or uncooperative clients.

## Verify the request, not just the name

An HTTP `User-Agent` header is a claim supplied by the requester. Any client can send `GPTBot` as text. For higher-confidence classification, combine several fields:

1. Preserve the complete user-agent header.
2. Match the stable product token case-insensitively.
3. Compare the source IP with the current vendor-published ranges.
4. Record the requested host, path, method, time, and HTTP outcome.
5. Keep the raw evidence so a classification can be corrected later.

Do not copy IP addresses from a blog post into a permanent firewall rule. Fetch the vendor's JSON source and retain when it was checked. Network ranges change.

## Read the outcome correctly

A `200` request proves that your infrastructure served a response to that request. A `403` proves that some layer refused it. A `404` may reveal a stale link or a route the crawler discovered elsewhere. None of those events proves that a model used the content.

SearchSeal records identified AI crawler requests from connected website infrastructure, including requested paths and HTTP outcomes when available. It does not turn a request into an indexing or citation claim.

## Debug in this order

1. **Policy:** fetch the public `/robots.txt` for the exact host.
2. **Identity:** inspect the raw user agent and source network evidence.
3. **Edge:** check CDN, bot management, and WAF decisions.
4. **Origin:** inspect redirects, status codes, and rendering failures.
5. **Interpretation:** report the request as observed evidence, not proof of use.

For a wider policy template, read [robots.txt for AI crawlers](/blog/robots-txt-ai-crawlers). For the cross-vendor identity list, use the [AI crawler user-agent reference](/blog/ai-crawler-user-agents).