Why ChatGPT Recommendations Change, and How to Test Fairly

Published October 16, 202613 min readUpdated October 16, 2026

ChatGPT builds a fresh answer every time someone asks, so the businesses it names shift from one run to the next. BrightLocal's 2026 study found two runs of the same local question shared only 23% of their names. A miss today is not a verdict, and a hit is not a promise.

Say you asked ChatGPT for the best dentist in your town on Monday, and it named your practice. On Thursday you asked again, and a rival took your spot. Nothing broke, and nobody paid to push you out.

ChatGPT recommendations change from one ask to the next because of how the tool builds each answer. This guide explains what the numbers mean, why the list moves, how to run a fair test over one week, and what raises your odds of being named more often.

Key takeaways

Two ChatGPT answers to the same local question share fewer than one in four of the businesses they name.

A business that shows up once comes back in about half of later answers, so one check is close to a coin flip.

Where the person asks from changes the answer, even inside one town.

A fair test is about 20 answers: five real customer questions, asked four times each, over a week.

Nobody can make ChatGPT name you every time. Be wary of anyone who says they can.

Owner comparing two phones where ChatGPT shows two different short lists of local businesses

What does a 23% overlap between ChatGPT answers mean?

A 23% overlap means two ChatGPT answers to the same local question share fewer than one in four of the businesses they name. That figure comes from BrightLocal's Local AI Visibility Study (September 2026), which tracked 200,085 local searches on ChatGPT, Google AI Mode and Google AI Overviews over 60 days.

Overlap is the share of names two answers have in common. Google AI Mode scored 21% and Google AI Overviews scored 33%, so ChatGPT is not alone. Every AI tool BrightLocal tested moved a lot.

The second number matters more for you: how often a business that showed up once comes back. For ChatGPT, it was about half the time. In BrightLocal's own words, if you run a prompt four times, a single business is likely to appear in two of them.

So a typical business is in about half the answers, not all of them. BrightLocal found 71-80% of businesses appeared in half the attempts or fewer. ChatGPT named 4.1 businesses per answer on average, and 10% of its answers named no business at all.

Here is what that can look like. This is a made-up example, not real data, for the question "best family dentist in [your town]":

Practice	Run 1	Run 2	Run 3	Run 4
Practice A (you)	Named	Not named	Named	Not named
Practice B	Named	Named	Named	Named
Practice C	Named	Not named	Not named	Named
Practice D	Not named	Named	Not named	Not named
Practice E	Not named	Named	Named	Not named
Practice F	Not named	Not named	Named	Named

If you only checked on run 2, you would think you were invisible. If you only checked on run 1, you would think you had it made. Both would be wrong.

Google Maps is far steadier. BrightLocal found Maps results agreed 99% of the time across attempts, against 91-96% for the AI tools. If you grew up on Maps rankings, ChatGPT will feel like a slot machine by comparison. It is not random, but it is not fixed either.

See what AI says about your business: get a free AI audit → Enter your website and we show you how AI search reads it and what to fix first.

Why do ChatGPT recommendations change between runs?

ChatGPT recommendations change between runs because ChatGPT writes each answer fresh, often after a new web search, and small differences in the question, the searcher's location and the pages it finds all shift which businesses make the short list. There is no saved ranking of businesses in your town for it to look up.

The biggest test of this so far looked at brands, not local shops. In January 2026, SparkToro had 600 volunteers run 12 prompts a combined 2,961 times. AI tools gave the same list of brands less than 1 in 100 times, and the same list in the same order less than 1 in 1,000 times.

Local answers move too. In July 2026, Whito asked ChatGPT the same plumber question 30 times across Nottingham, Luton and Preston. The answers named 51 different businesses, and only one business was named every time. Changing how the question was worded changed roughly 70% of the names.

Here are the main causes, and what you can do about each one.

Cause	What it looks like	What you can do
A fresh answer every time	Same question, different names an hour later	Judge yourself on many answers, not one
A new web search each time	Different websites quoted as sources	Make your own site clear enough to be one of them
Small changes in wording	"Emergency plumber" and "plumber open tonight" give different lists	Test the five ways customers ask
Where the person is asking from	A friend across town sees other names	Test from more than one place
Your details change or clash	Old hours or a wrong phone number on a directory	Make your name, address, phone and hours match everywhere

A 2025 test of product questions by Amplifying also found ChatGPT answers built from a live web search were less consistent than answers built from what the model already knew.

Different AI tools disagree with each other as well. BrightLocal found ChatGPT's named businesses overlapped only 20% with Google AI Overviews and 19% with Google AI Mode. Being named in one tool tells you little about the others.

Why does ChatGPT name you in one part of town but not another?

ChatGPT names different businesses for people in different parts of the same town because it factors in where the person is asking from. BrightLocal found ChatGPT results overlapped only 36.2% across points in the same town, the lowest of the three AI tools it tested.

Only 3.5% of the businesses ChatGPT named showed up at every point on the map. ChatGPT also pulls in businesses from much farther away than Google's AI tools do.

That has two lessons for your testing. First, your own phone in your own shop is one point on the map, not the whole town. Second, a customer across town may see a different list than you do.

Service-area businesses feel this most. A service-area business is one that goes to the customer, like a roofer, a plumber or a mobile dog groomer, instead of having customers walk in. BrightLocal found storefront businesses were mentioned about 34% of the time in AI answers, against about 29% for service-area businesses.

Picture a roofer based in one suburb who works in six. If the website only names the home suburb, ChatGPT has little to match when someone in the other five asks. A short page for each area you serve, with the town name, the jobs you do there and how fast you can get there, gives it something to work with.

How do you test ChatGPT fairly?

You test ChatGPT fairly by asking the questions real customers ask, in a clean chat, several times over several days, and writing every result down. About 20 answers is enough to trust: five customer questions, each asked four times, over a week or more.

One check tells you almost nothing, because a business named half the time will miss about half of all single checks. SparkToro's researchers suggest running each prompt 60 to 100 times for a research study. You run a business, so here is a scale that fits a busy week.

How many answers	What it tells you	Good for
1	Almost nothing. A hit or a miss can be luck	Seeing what the answer looks like
5 (one question, five times)	Whether you show up for that one way of asking	Spotting a total blank
20 (five questions, four times each, over a week)	A fair score, like "named in 9 of 20"	A starting point you can compare against
20 again each month	Whether your score is going up or down	Checking whether your fixes are helping

"Named in 9 of 20" is a useful number. "Third in the list on Tuesday" is not, because the order changes even more than the names do. The six steps below fit into one normal week.

Step 1: Write five questions your customers ask

Write five questions in your customers' words, not your own, and never include your business name. Mix in the words that change the answer, like "tonight", "near me", "cheap" or "open now".

Here are starter sets you can copy and adjust:

Dentist: "emergency dentist open Saturday in [town]", "best family dentist in [town] that takes new patients", "how much is teeth whitening in [town]"
Plumber: "plumber who can come tonight in [town]", "best rated plumber near [neighborhood]", "who fixes water heaters in [town]"
Lawyer: "divorce lawyer in [town] with free consultation", "best DUI attorney in [county]"
Med spa: "best place for Botox in [town]", "med spa near [neighborhood] with good reviews"
Roofer: "roof leak repair [town] same week", "who replaces roofs in [suburb]"

Step 2: Use a temporary chat

Open a Temporary Chat in ChatGPT (or a logged-out window) for each test. That way your past chats and saved memory do not nudge the answer toward businesses you have talked about before.

Step 3: Ask each question four times over a week

Ask each of your five questions four times, on different days and at different times. That gives you 20 answers. If you can, have someone in another part of town run a few of them too.

Step 4: Log every answer in one sheet

Write down every answer, even the ones where you are missing. A simple sheet with these columns is enough:

Date	Question	Named? (Y/N)	Who else was named	Sources ChatGPT showed	Were your details right?
Mon 3pm	emergency dentist open Saturday in [town]	N	Practice B, Practice C	Yelp, Practice B website	n/a
Wed 9am	emergency dentist open Saturday in [town]	Y	Practice B, Practice E	Your website, Yelp	Phone was old number

The "who else" and "sources" columns matter most. They show which rivals keep coming up and which sites ChatGPT trusts for your trade. Our guide on how to check if ChatGPT recommends your competitors shows how to turn that list into a plan.

Step 5: Check the details ChatGPT gives about you

When you are named, check every detail: phone, address, hours, services. In a July 2026 benchmark of 10,000 US local businesses, Insites found only 55.58% of ChatGPT results fully matched the business's Google Business Profile, and a wrong phone number was the most common error (30.09% of ChatGPT results). A mention with the wrong number can send the call to nobody.

Step 6: Repeat on other AI tools, then once a month

Run the same five questions on Google AI Mode, Microsoft Copilot and Claude. Then repeat the full test once a month and compare your score out of 20, not single answers.

Want it done for you? Get your AI SEO setup for $349 → We add the code AI reads, fix AI crawler access and your business info. Done in 48 hours, with a 30-day money-back guarantee.

What makes ChatGPT name your business more often?

Nobody can switch ChatGPT on for your business, but the data points to a few things that businesses named more often tend to have: more reviews, details that match everywhere, a clear website ChatGPT can read, and no block on ChatGPT's search crawler. Each one makes you an easier, safer pick.

Reviews come first in the data. Insites found businesses surfaced by both ChatGPT and Perplexity averaged 133.4 Google reviews, against 10.7 for businesses surfaced by neither. It called that the strongest signal in its whole dataset. According to the SOCi Local Visibility Index 2026, via BrightLocal, the local businesses ChatGPT recommends average 4.3 stars.

A review ask can be one text. Try: "Thanks for having us out today, [name]. If you were happy with the repair, a quick review helps other people in [town] find us: [link]."

Your website is the top source. BrightLocal found business websites made up 42% of all citations in AI answers, ahead of Google Business Profile (about 29%) and Yelp (9.5%). Pages that answer the exact question help most: your hours, whether you take emergencies, the areas you serve and rough prices.

Matching details remove doubt. The SOCi Local Visibility Index 2026 found only 68% of business contact details shown by ChatGPT and Perplexity matched the Google Business Profile. If your old phone number lives on three directories, ChatGPT may get your number wrong, and Insites found exactly that in 30.09% of ChatGPT results.

ChatGPT's crawler has to get in. OpenAI's crawler documentation says OAI-SearchBot is the bot it uses to surface websites in ChatGPT's search features. If your robots.txt file (the small file that tells bots where they can go) blocks it, ChatGPT cannot read your site. Our guide to the hidden file that can block ChatGPT shows how to check.

Here is the honest limit. Insites found structured data (the code that labels your business details for machines) had little effect on whether AI tools knew a business. Code is tidy-up work, not a switch. It helps ChatGPT read your details correctly, but it will not beat a rival with ten times your reviews.

If you would rather not do the website side yourself, our done-for-you setup checks how ChatGPT, Claude, Google AI and Copilot see you, fixes crawler access, makes your business details match, and adds the code AI reads. It is $349 one-time (normally $497), done in 48 hours, with a 30-day money-back guarantee.

What should you ask anyone who promises ChatGPT results?

Ask anyone who promises ChatGPT results how they will measure it, because when the list changes this much between answers, "we will get you to number one in ChatGPT" is a promise nobody can check, let alone keep. A fair provider talks about making you easier to pick and counting how often you are named over many answers.

SparkToro's Rand Fishkin put it bluntly: any tool that gives a "ranking position in AI" is "full of baloney." That goes double for a sales pitch built on one screenshot.

What they say	What it means	Ask this
"We'll get you #1 in ChatGPT"	There is no fixed #1 to hold	"How do you measure #1 when the list changes every time?"
"Here's a screenshot of a client being named"	One answer out of many	"How many runs was that, and how many named them?"
"Guaranteed AI recommendations"	Nobody controls what ChatGPT says	"What happens if it doesn't happen?"
"We track your AI ranking daily"	Daily order changes are mostly noise	"Do you count how often I'm named across many questions?"

Before you sign anything, ask these five questions:

Which customer questions will you test, and how many ways of asking?
How many times will you run each one?
Where will the tests run from: my town, or your office?
Which AI tools will you check besides ChatGPT?
What exactly do I get if my score does not move?

We do not promise that ChatGPT or any AI tool will name your business either. What anyone can honestly do is fix the things that make you hard to pick.

Frequently asked questions

Why did ChatGPT stop recommending my business?

ChatGPT usually stops naming a business for a while because of normal churn between answers, not because something broke. Lists change a lot from one ask to the next. Before you change anything, run the 20-answer test: five customer questions, four times each, over a week. If you are named in some of them, nothing is wrong.

Is ChatGPT showing me different results because I am logged in?

Being logged in can shape ChatGPT's answers, because saved memory and past chats can nudge what it says. Use a Temporary Chat or a logged-out window when you test, so the answer is closer to what a new customer sees.

How long do changes take to show up in ChatGPT?

Some changes show up quickly and some take weeks. OpenAI says it can take about 24 hours for its search systems to adjust after a robots.txt update. Reviews, listings and website changes have no fixed timeline, so re-run your 20-answer test monthly and watch the trend.

Does Google AI name the same businesses as ChatGPT?

Google's AI tools and ChatGPT name mostly different businesses. BrightLocal's 2026 study found little overlap between them, so test each tool on its own.

Should I pay for an AI visibility tracking tool?

A paid tracking tool is worth it only if it counts how often you are named across many questions and runs, not a single "AI ranking." For most small businesses, a free monthly 20-answer test in a sheet is enough to see whether things are getting better.

Why does ChatGPT show the wrong phone number for my business?

ChatGPT shows a wrong phone number when it finds old or clashing details on the web. Insites found a wrong phone number was the most common error in ChatGPT's local results in 2026. Fix the number on your website, Google Business Profile, Yelp and every directory so they all match.

The short version

ChatGPT recommendations change because ChatGPT writes every answer fresh, so stop judging your business on one check. Ask five real customer questions four times each in a clean chat, and count how many of the 20 answers name you.

Then work on what makes you an easy pick: more reviews, matching details, a clear website and a crawler that can get in. Our guide on how ChatGPT recommends local businesses covers the full picture. Nobody can promise you a spot, but you can stop being easy to skip.

Want to see where you stand first? Get a free AI audit. Ready to have it done for you? Get your AI SEO setup for $349, done in 48 hours.