G
GEO Toolbox

AI Crawlers

GPTBot

Also: OpenAI GPTBot

GPTBot is OpenAI's web crawler that gathers publicly available content which may be used to train its models. You control it through robots.txt. It is separate from OAI-SearchBot, the crawler that surfaces pages in ChatGPT's search answers, so blocking GPTBot opts you out of training without removing you from ChatGPT search.

Updated

GPTBot identifies itself with the user agent GPTBot and respects robots.txt. Allowing it lets OpenAI include your content in training data; disallowing it keeps your pages out of future model training.

The common mistake is assuming GPTBot controls whether you appear in ChatGPT. It does not. ChatGPT's live search answers are gathered by a different crawler, OAI-SearchBot. If you block GPTBot to stay out of training but leave OAI-SearchBot allowed, you can still be cited in ChatGPT search. A broad User-agent: * block, however, can catch both.