How do we rank in ChatGPT?

The companies running these systems publish inclusion, not order

Start with what each publisher actually documents. The gap between them is the whole finding.

Google folds AI traffic into ordinary reporting rather than breaking it out. Its words: "sites appearing in AI features (such as AI Overviews and AI Mode) are included in the overall search traffic in Search Console."

There is no separate scoreboard, because there is no separate contest.

Microsoft went the other way, and the contrast is the most useful thing here. When it introduced an AI performance view in Bing Webmaster Tools, it did break the numbers out. Then it spent the announcement explaining what they do not mean.

On citation counts, the view shows how often content is referenced "without indicating placement or presentation within a specific answer."

On averages, the figures reflect overall patterns and "does not indicate ranking, authority, or the role of any page within an individual answer."

On page level data: "This reflects how often pages are cited, not page importance, ranking, or placement."

Three disclaimers in one announcement. One engine will not separate AI traffic from search traffic at all. The other separates it, then says three times that the separated number is not a ranking.

Microsoft states the limit of the data too: "The data shown represents a sample of overall citation activity." So the count is an estimate, and it is described as one.

A reader who knows this cannot be sold an AI ranking by anybody, because the two companies best placed to publish one have each declined in their own way.

OpenAI documents something narrower still. Its crawler page describes what a site owner can decide. Every decision on it is binary. You may allow the crawler that feeds search answers, or you may refuse it.

Neither publisher describes a position, a score, or a way to move up. That is not an oversight you can work around. It is the shape of what has been made public, and the shape is the same at both companies.

So the ranking question has no published answer. The eligibility question has a documented one. Those are different questions, and only the second has a lever attached to it.

No new file or format is required, whatever you have been offered

This is the claim to check any proposal against, because Google answers it directly.

On files and markup: "You don't need to create new machine readable files, AI text files, or markup to appear in these features. There's also no special schema.org structured data that you need to add."

Two sentences, and together they rule out most of what gets sold as AI optimisation. No new file type. No new markup. No special structured data.

What Google lists instead is unremarkable. Allow crawling. Link your pages to each other. Keep important content in text rather than locked inside images. Keep your structured data matching what a reader can see.

That last one is worth pausing on. The instruction is to make structured data match the visible text, which is a fidelity rule rather than a volume rule. Adding more markup is not the goal. Agreeing with the page is the goal.

None of this is exciting, which is exactly why it gets skipped. An owner who is told the fundamentals still apply tends to hear nothing new. The correct response is relief, because it means the work you already paid for is the work that counts.

The one control is a switch with three separate positions

Here is where real damage gets done, usually by somebody trying to help.

OpenAI names different agents for different jobs, and states plainly that "Each setting is independent of the others", so one can be allowed while another is refused.

One crawler supports search answers. Another gathers content used for training. A third fetches a page when somebody asks about it directly.

Allowing one and refusing another is explicitly supported: "a webmaster can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot to indicate that crawled content should not be used for training OpenAI's generative AI foundation models."

So a site can be findable in ChatGPT answers without its content feeding model training. Those are separate decisions. Treating them as one decision is the common error.

The cost of getting it wrong is stated too: "Sites that are opted out of OAI-SearchBot will not be shown in ChatGPT search answers, though can still appear as navigational links."

Read the second half of that sentence carefully. Opting out is not total disappearance. It changes what kind of thing your site can be inside an answer. Being a link somebody may follow is a different outcome from being a source the answer was built on.

One more detail stops a wrong reading of your server logs. The agent used when a person asks about a specific page works like this: "ChatGPT-User is not used for crawling the web in an automatic fashion."

A visit from it is closer to a click than to a crawl. Seeing it rarely is not evidence that you have been shut out.

The switch is a request, and the standard says so

This is the limit nobody selling AI visibility will mention. It comes from neither company.

The control OpenAI documents is the Robots Exclusion Protocol, standardised in RFC 9309. The standard describes its purpose as letting service owners "control how content served by their services may be accessed, if at all, by automatic clients known as crawlers."

Then it describes the nature of that control. One verb carries the weight. These are rules "that crawlers are requested to honor when accessing URIs."

Requested. Not required. The standard is blunt about what follows: "These rules are not a form of access authorization."

It repeats the point where the consequences are worst: "The Robots Exclusion Protocol is not a substitute for valid content security measures."

So an opt-out is a notice. Well behaved automated clients are asked to respect it. Large operators publish their agent names and addresses so their behaviour can be checked, which is a reason to trust those operators rather than a reason to assume every client behaves alike.

If something must not be read by anyone at all, a line in a text file is the wrong tool. The standard says so plainly instead of leaving you to find out later.

That reframes the question. What you control is a request about inclusion. What you cannot control is order, because no one has described it.

What to check, in order

Five checks, and every one of them is free.

• Read your robots.txt before changing anything. The agents named in it already decide whether you are eligible to appear, and most owners have never opened the file.
• Answer the search question and the training question separately. They are independent settings. One blanket rule is how sites remove themselves from answers while meaning to protect their content.
• Wait a day before testing a change. OpenAI notes it can take around 24 hours from a robots.txt update for its systems to adjust, so a same day test measures the old state.
• Treat AI traffic as search traffic in your reporting, because that is how Google files it. Hunting for a separate AI number means hunting for something nobody publishes.
• Only then look at the content, using ordinary search fundamentals. Google says the usual best practices remain the relevant ones.

Now the arithmetic, because it turns a vague worry into a decision you can act on. There are three independently controlled agents. Each is either allowed or refused. That gives 2 x 2 x 2, which is eight possible configurations of your site.

Exactly one of those eight is the blanket block. It is the one people reach for when they decide they want nothing to do with AI. It also removes the site from search answers, because one of the three agents is the agent that puts you there.

The figures are only a count of the options. The point survives anyway. A single decision, made once, in a file most owners have never opened, selects among eight outcomes. The intuitive choice is the one that costs the most visibility.

Work out which of the eight you are in before anybody argues about content. It takes five minutes, it costs nothing, and it settles a question that otherwise gets answered with guesses.

Reading these rules so a site is eligible everywhere its owner wants to be, and absent where they do not, is part of what we do on search.

Terms used on this page

• robots.txt
• AI Overviews
• Google-Extended
• AI performance report

Sources

• AI features and your website (Google Search Central)
• Overview of OpenAI Crawlers (OpenAI Platform documentation)
• RFC 9309, Robots Exclusion Protocol (RFC Editor, Proposed Standard)
• Introducing AI Performance in Bing Webmaster Tools (Bing Webmaster Blog, Microsoft)

Last reviewed 2026-09-12.