G
GEO Toolbox

AI Crawlers

PerplexityBot

Also: Perplexity crawler

PerplexityBot is Perplexity's crawler, designed to surface and link websites in Perplexity's search results. Perplexity states it is not used to train foundation models and recommends allowing it in robots.txt. Blocking it removes you from the index Perplexity builds answers from.

Updated

Perplexity explicitly recommends allowing PerplexityBot, because it powers the citations Perplexity shows in answers. It is a search crawler, not a training crawler.

Pair it with Perplexity-User, the agent that fetches a page live when a user's query requires it. PerplexityBot respects robots.txt; Perplexity-User, being user-initiated, generally does not.