OAI-SearchBot is the bot OpenAI uses for search. According to the overview of OpenAI crawlers, it makes it possible for websites to appear in ChatGPT’s search features. A site that blocks OAI-SearchBot will not appear in ChatGPT’s search answers, but can still be shown as a navigational link. The bot follows robots.txt; its token is OAI-SearchBot.
Why does it matter?
If you want ChatGPT to be able to link to your pages for a search question, this bot needs to be allowed to fetch your site. OpenAI writes that it can take about 24 hours for a change in your robots.txt to take effect in search results.
How do you check it?
Search your server logs for OAI-SearchBot. The part of the user agent that identifies it looks like this (the version number may change):
compatible; OAI-SearchBot/1.4; +https://openai.com/searchbotAlso check your robots.txt and any bot protection at your hosting provider or CDN. A firewall that keeps out unknown bots can block this bot as well.
Common mistake
Confusing OAI-SearchBot with GPTBot. GPTBot is for training, OAI-SearchBot for search. OpenAI describes the settings as independent of each other. A third bot, ChatGPT-User, fetches pages for certain user actions; according to OpenAI, robots.txt rules may not apply to it.
More in How do you become visible in ChatGPT? and Allowing or blocking AI crawlers with robots.txt.