Asking ChatGPT "Do You Recommend Me?" Once Tells You Nothing
You typed your business name into ChatGPT, asked if it would recommend you, and got an answer. Maybe it named you. Maybe it didn't. Either way, that single check told you almost nothing true, because AI tools hand back a different answer nearly every time you ask. The only honest number for AI visibility is how often you show up across dozens of asks, not where you landed in one.
This matters because a one-time check feels like a measurement and acts like a coin flip. A vendor will sell you a "your AI ranking" dashboard built on that coin flip, and you will make real decisions on noise. Stop ranking yourself in a single answer. Start counting how frequently you appear across many.
One ChatGPT check is a coin flip dressed up as a measurement
In numbers, that randomness looks like this. In research by Rand Fishkin (SparkToro) with Gumshoe.ai in late January 2026, 600 volunteers ran 12 prompts across ChatGPT, Claude, and Google AI a combined 2,961 times. The finding: there is less than a 1 in 100 chance that two responses return the same list of brands, and less than a 1 in 1,000 chance of the same list in the same order. Fishkin describes this work as not peer-reviewed, so treat it as a large field test rather than a lab result, but the size of the effect is hard to wave away.
Read that again as a business owner. If two answers almost never match, then the one answer you checked on Tuesday morning is a single draw from a deck that reshuffles itself constantly. You would not judge your Google rankings by refreshing the page once and screenshotting it. Yet that is exactly what a one-time AI check is, except worse, because the AI is built to vary on purpose.
The cost of treating that one draw as truth is concrete. You panic over a "miss" that was random, or you celebrate a "win" that was random, and you spend budget chasing a number that was never stable to begin with. A specialty bakery owner who checks once, sees a competitor named instead, and rushes to hire an "AI optimization" firm is reacting to a dice roll. The money is real. The signal that triggered it was not.
This is a different problem from your visibility shifting when a model gets updated, which I covered in how AI visibility swings when models update. That piece is about version changes over weeks and months. This one is about the randomness inside the same model on the same afternoon. Two separate forces. Both punish anyone reading a single answer as gospel.
Owners get this wrong because the tools look like search and act like slot machines
You have twenty years of muscle memory that says a search box returns a stable, rankable list. Type a query, get results, results sit in an order, the order means something. AI chat boxes look identical. They are not. They are probability engines, which means each answer is freshly generated text, not a lookup from a fixed list.
Fishkin put it plainly:
"These tools are probability engines: they're designed to generate unique answers every time. Thinking of them as sources of truth or consistency is provably nonsensical."
Rand Fishkin, SparkToro
The second trap is the dashboard. A tool shows you a tidy "AI ranking position: #4" and your brain accepts it, because a number in a box reads as fact. But that number was generated from a handful of asks, and the next handful would produce a different figure. Fishkin's verdict on those products is blunt:
"any tool that gives a 'ranking position in AI' is full of baloney."
Rand Fishkin, SparkToro
The third reason owners get fooled is that a single check is free and easy, and a proper count is tedious. Asking once takes ten seconds. Asking thirty times across fresh sessions and tallying the results takes an afternoon. Human nature picks the ten seconds, then trusts the ten-second answer because the effort of the alternative makes the shortcut feel sufficient. It is not.
The data says count your hit rate, not your rank
The same SparkToro research that exposed the randomness also showed the right metric hiding in plain sight. One hospital, City of Hope, appeared in 69 of 71 ChatGPT answers, roughly 97 percent of the time. It was ranked first in only about 25 of those answers. So its "rank" bounced all over the place while its presence was nearly guaranteed.
That gap is the whole point. If you had measured City of Hope by its position in any single answer, you would have gotten a number somewhere between first and tenth, basically at random, and called it a result. Measure it by presence across many asks and you get 97 percent, a figure stable enough to actually steer decisions. Presence is the signal. Position is the noise.
Fishkin even hands you the practical consequence of the randomness, almost as a dare:
"If you don't like an answer, or your brand doesn't show up where you want it to, just ask a few more times."
Rand Fishkin, SparkToro
If asking a few more times changes the answer, then the answer was never a measurement. The frequency across all those asks is. So why does any of this presence stuff matter for revenue, when most clicks still happen the old way? Because being named inside the AI box is worth real traffic when the click does happen.
Seer Interactive looked at 53 brands across 5.47 million queries in April 2026 and found that being cited inside a Google AI Overview delivered about 120 percent more organic clicks per impression than appearing on the same results page without being cited. Translation for your week: getting named in the AI summary is worth more than doubling your click rate from that page, which is why presence frequency is the thing to grow, not vanity rank.
Now temper that with how often people click at all when a summary shows up. Pew Research Center, publishing in July 2025 from March 2025 browsing data on 900 US adults, found that when an AI summary appeared, users clicked a result 8 percent of the time versus 15 percent without a summary, and only 1 percent clicked a link inside the summary. That data is from early 2025 and the tools have moved, so read it as direction, not gospel: AI summaries depress clicking overall, which means being named inside the answer matters more precisely because fewer people are leaving to click anything.
Put the two together. Clicks are getting scarcer when AI answers appear, and the clicks that survive flow heavily to cited brands. Your job is not to win a phantom rank in one answer. It is to be the brand the AI names again and again, so you catch the shrinking pool of clicks that is left.
The fix is a hit-rate baseline plus the boring fundamentals that move it
This is not complicated, it is just disciplined. The routine below is built for someone who runs a business and does not have a data team.
Pick your three most important questions. Not twenty. Three. The "best [category] in [place]" or "best [category] for [specific need]" questions your real customers would type. A pediatric dentist in Sacramento picks "best pediatric dentist in Sacramento," not "best dentist in California." Specific beats broad, because that is how customers actually ask.
Ask each question 10 times in fresh or incognito sessions. Open ChatGPT or Google AI Mode in an incognito or logged-out window so past chats do not bias the result, ask, note whether your brand appears, close it, repeat. Do all ten for one question, then move to the next. Tally how many times out of ten you showed up.
Write down the fraction. That fraction is your baseline. "4 out of 10" or "9 out of 10" is your real AI visibility for that question. Not a rank. A hit rate. City of Hope's honest number was 69 of 71, and yours is whatever your tally says, no dashboard required.
Spend your effort on the fundamentals that actually move the hit rate. The factors that drive AI citations are mostly the unglamorous ones you already half-know. Cyrus Shepard of Zyppy synthesized 54 experiments on May 7, 2026 into scored citation factors: URL accessibility scored 9.5, traditional search rank scored 9.4, and "fan-out rank" (how well you cover the related sub-questions an AI breaks a query into) scored 9.3. A dedicated "LLMs.txt" file, the kind of trick vendors love to sell, scored 2.0, near the bottom. So the decision is simple: make your pages reliably reachable, rank well in normal search, cover the sub-questions thoroughly, and ignore the gimmick files.
Shepard's own summary of the takeaway:
"win SEO, win AI citations (most of the time, with extra steps.)"
Cyrus Shepard, Zyppy
That is the whole strategy compressed into a sentence. The work that earns AI citations is the same precise SEO work that has always earned visibility, which I argued in AI citation is just precise SEO. There is no separate AI playbook waiting to be bought. There is the regular playbook, executed well.
Re-run the tally monthly. Same three questions, same ten asks each, same incognito sessions, first week of every month. Watch the fraction, not any single answer. If "4 out of 10" becomes "7 out of 10" over a quarter, your fundamentals are working. If it sits flat, the problem is upstream, and you may be working the wrong layer entirely, which is the trap I broke down in why your AI visibility work isn't moving the needle and which layer to fix first.
Measure the hit rate monthly and refuse the metrics that lie to you
The thing worth tracking is one number per question: appearances out of ten, checked monthly. That is it. A pizza shop owner tracking "best pizza in Tucson" should be able to say "we went from 3 of 10 in April to 6 of 10 in June," and that sentence should be the entire dashboard.
Three vanity metrics will try to pull you off that simple number. The first is rank in a single answer. You now know why it is garbage: it changes when you ask again, so any "we're #2 in ChatGPT" claim is a screenshot of one dice roll. If a report leads with a rank position, distrust the report.
The second trap is a vendor's proprietary "AI visibility score." If you cannot reproduce a number yourself by opening an incognito window and counting, you cannot trust it and you certainly cannot act on it. Your hand-tallied fraction is less polished and far more honest. A score you cannot recreate is a score you are renting on faith.
The third trap is chasing every model at once. You do not need ChatGPT, Claude, Gemini, and every other tool tracked in a spreadsheet. Pick the one or two your customers actually use, usually ChatGPT or Google AI Mode for most local and small businesses, and measure those well. Breadth across tools you have no time to act on is just more noise to drown in.
The cadence keeps you honest in the other direction too. Monthly is frequent enough to catch a real trend and slow enough that you are not reacting to randomness. If you check weekly, you will see the fraction wobble from sampling noise and start tinkering. Hold the line at monthly. Let the boring fundamentals compound between checks.
Measuring well is one piece of the work. I put it in context in my complete guide to AI search visibility.
Frequently Asked Questions
Why does ChatGPT give me a different answer every time I ask if it recommends my business?
Because these tools are built to generate fresh text on every request, not to look up a fixed list. In research by Rand Fishkin (SparkToro) with Gumshoe.ai, across 2,961 runs there was less than a 1 in 100 chance that two responses returned the same list of brands. So a single check is one random draw, not a measurement. The honest number is how often you appear across many asks, not what one answer said.
How do I actually measure my AI visibility without buying a tool?
Pick your three most important "best [category] in [place or for X]" questions. Ask each one 10 times in fresh or incognito ChatGPT or Google AI Mode sessions and tally how many times out of 10 your brand appears. That fraction is your real baseline, the same way one hospital in the SparkToro study showed up in 69 of 71 answers. Re-run the same tally monthly and watch the fraction move, not any single answer.
Does showing up in AI answers even matter if most people don't click?
It matters more, not less, because clicks are scarcer when an AI answer appears. Pew Research Center found that with an AI summary present, users clicked a result 8 percent of the time versus 15 percent without one, though that data is from early 2025. Meanwhile Seer Interactive found that being cited inside a Google AI Overview delivered about 120 percent more organic clicks per impression than appearing on the same page uncited. Fewer clicks overall, far more of them going to the brands the AI names.
You came in wanting to know your rank in ChatGPT. The truthful answer is that you do not have a rank, you have a hit rate, and the only way to know it is to count. The good news buried in all this randomness is that the work that raises the count is the same unglamorous SEO you could have started years ago. The unsettling part is simpler: every confident "AI ranking" number you have ever been shown was one roll of the dice, framed as a fact.
