Usually because a different crawler is involved. Google finding your site says nothing about whether OpenAI or Microsoft can. Each runs its own bots, reads your robots.txt separately, and builds its own picture of your site. If you never gave those bots permission, you are invisible to them.
Most teams we work with have spent years tuning for Googlebot and have never once opened Bing Webmaster Tools or checked which AI crawlers their robots.txt allows. That was a reasonable trade when Google was the only door. It is not reasonable now, because a growing share of buying research starts inside a chat window instead of a search box.
The good news is that this is mostly a configuration problem, not a content problem. The fixes are small, they are verifiable, and you can do most of them in an afternoon.
Through a dedicated crawler called OAI-SearchBot. OpenAI documents four separate bots, and only this one governs whether your site can appear in ChatGPT search answers. It is a completely separate permission from the one that lets OpenAI train on your content, which is the part most site owners get wrong.
In OpenAI's own developer documentation, the four bots are OAI-SearchBot, which is "used to surface websites in search results in ChatGPT's search features", GPTBot, which is "used to make our generative AI foundation models more useful and safe", ChatGPT-User, which handles "certain user actions in ChatGPT and Custom GPTs", and OAI-AdsBot, which validates pages submitted as ads.
OpenAI is explicit about the consequence of blocking the search crawler. Its documentation states that "sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links." That is the whole game in one sentence. Block that bot and you are not in the answer.
Worth being honest about what is not documented. OpenAI does not publish the full makeup of the index behind ChatGPT search, and it has never committed publicly to a single upstream provider. So we do not tell clients "ChatGPT runs on Bing" as if it were settled. We tell them to satisfy both OpenAI's crawler and Bing's crawler, because that covers the ground either way.
For most businesses, yes. This is the setting that lets you appear in ChatGPT answers without handing your content over for model training. OpenAI supports the split directly. Its documentation confirms that "a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot."
We think this is the right default for a marketing site. You publish content so people find you. Being cited in a ChatGPT answer with your name and link attached is exactly what you wanted. Being absorbed into training data with no attribution is a different transaction, and you are allowed to decline it.
The mistake we see most often is a blanket block. Someone reads an article about AI scraping, adds a rule that disallows every bot with "GPT" or "AI" in the name, and quietly removes their company from ChatGPT search results. Nobody notices for months because nothing in Google Analytics changes. We have covered the full robots.txt setup in our guide to controlling AI crawlers with robots.txt, and the short version is to be deliberate about each bot rather than swinging at all of them.
One more detail from OpenAI's docs that matters. For ChatGPT-User, which fires when a person asks ChatGPT to go look at a specific page, OpenAI notes that "robots.txt rules may not apply" because the action is user-initiated. So your robots.txt is not a wall against everything.
Yes, and the share number undersells it. Statcounter put Google at 91.32% of worldwide search in July 2026, with Bing at 4.46%, Yahoo at 1.24%, Yandex at 0.99%, and DuckDuckGo at 0.65%. On raw traffic, Bing is small. On strategic value, it is worth far more than 4.46% of your attention.
The reason is that browser share is the wrong way to size an index. Microsoft has built AI answer surfaces on top of Bing, and Bing itself publishes webmaster guidance about AI powered search, so the same index reaches people who never visit bing.com. Getting into it is cheap, so the return does not need to be large to justify the work.
There is also a competitive argument. Almost nobody optimizes for Bing. Every competitor you have is fighting for the same Google positions, and most of them have never verified their site in Bing Webmaster Tools. That makes Bing one of the few places left where basic diligence still produces an outsized result.
Submit a sitemap and use IndexNow. Bing recommends putting your sitemap location in robots.txt for automatic discovery and submitting it directly in Bing Webmaster Tools. In its July 2025 post on sitemaps in AI powered search, Bing says it will fetch a submitted sitemap immediately and then revisit it regularly, typically at least once per day.
IndexNow is the faster half. It is a simple protocol where you ping participating engines the moment a URL changes, instead of waiting for a crawler to notice. The official IndexNow FAQ lists Bing, Yandex, Naver, Seznam.cz, Yep, and Amazon as participating engines, with Cloudflare offering support at the CDN level. It also confirms you can submit up to 10,000 URLs per POST request.
Be realistic about what it buys you. The IndexNow FAQ is blunt that "submitting a URL does not guarantee immediate indexing" and that "the search engine evaluates whether it should crawl the URL based on its crawl quota, scheduling logic, and quality signals." It moves you up the queue. It does not skip the queue. Our explainer on IndexNow walks through the setup end to end.
In Bing's own framing from that July 2025 post, the combination is the point. Sitemaps give comprehensive site coverage and IndexNow gives fast URL-level submission, and using both is what keeps content discoverable in traditional and AI powered search alike.
Two meta tags, announced in September 2023. NOCACHE lets your content appear in Bing Chat answers but limits what is shown to the URL, snippet, and title. NOARCHIVE keeps your content out of Bing Chat answers entirely, and out of links in those answers. Both tags leave normal Bing search results untouched.
This is a more granular set of controls than most site owners realize exists, and it is a useful model for thinking about the whole category. You are not choosing between total openness and total invisibility. You are choosing, per surface, how much of your page a machine may reproduce.
Our advice for a typical services business is to use neither tag. If you are publishing content specifically so that buyers find you, appearing in an AI answer with a link back is the outcome you were paying for. The tags earn their place when you have content you want indexed for search but not summarized, which is a narrower case than it sounds.
Less different than the folklore suggests, but the emphasis shifts. In our work the things that move Bing are the mechanical, verifiable ones: clean crawlability, a current sitemap, fast server responses, unambiguous title tags, and real schema markup. The judgment-heavy signals Google leans on carry less weight.
That is our read, not a claim from Microsoft, and we hold it loosely. What we can say with confidence is that the work is not additive in the way people fear. There is no separate Bing content strategy to write. A site that is technically clean for Google is most of the way to being clean for Bing, and the remaining gap is usually setup rather than substance.
The one place we do change our approach is exact-match clarity. We keep page titles and H1s more literal on Bing-facing pages than we might for Google, where semantic understanding is stronger. Naming the thing plainly costs nothing and removes ambiguity for every crawler reading the page.
Four things, in order. Open your robots.txt and confirm OAI-SearchBot is not disallowed. Verify your domain in Bing Webmaster Tools. Make sure your sitemap URL is declared in robots.txt. Then set up IndexNow so new pages get announced instead of waiting to be found.
Each of these is a yes or no answer, which is why we start here on every audit. There is no strategy to debate and no content to write. You either allow the crawler or you do not, and you either announce your URLs or you leave them sitting there hoping to be discovered.
After that, the work becomes ordinary content work again. Being crawlable gets you eligible. Being the clearest answer on the page is what gets you cited, and we go deep on that in our guide to getting cited by AI search engines. Access first, then quality. Doing it in the other order wastes the quality.
Treat it as a separate access problem and the same content problem. The permissions are genuinely distinct and need their own check, because OpenAI's crawler and Google's crawler answer to different rules. The writing that earns a citation is the same writing that earns a ranking: specific, current, and clearly structured.
What we would not do is build a parallel content operation for AI search. We have not seen anything that suggests a separate body of work is required, and the sites doing well in AI answers in our experience are the ones that were already publishing precise, well-organized pages. The AI surfaces rewarded them for it.
If you want a second set of eyes on your robots.txt and your crawler access before you invest in more content, we are happy to walk through it. It is a short review and it usually turns up at least one rule blocking something you did not mean to block. Reach out through phoenix.studio and we will take a look.
Tell us where you want to go. We'll tell you how we'd get you there.