Buzzwords Lose, Receipts Win: How to Vet an AI Search Vendor
You are about to pay someone to get your business named inside ChatGPT, and every proposal on your desk reads like it was written by the same three people. Entity graphs. Retrieval pipelines. Definition-lead architecture. I sell this work, and it does not change the rule: the fancier a pitch sounds, the less you should trust it, and the vendor worth hiring is the one who hands you something you can check without them in the room. What follows is the vetting guide agencies do not write, because writing it disarms their own pitch, and it ends with the same test pointed at your own website, where your customers are already running it on you.
The three pages you read before hiring were written by people who want the job
Search "how to choose an AI search agency" and read the results the way a buyer actually reads them: one after another, on a phone, at nine at night. Every one is published by an agency. Every one carries a list of things to look for, and every item on that list is something the author happens to sell. You finish all three knowing more vocabulary than you did an hour ago. You still cannot tell an operator from a salesperson, which was the only thing you needed.
That gap is why this page exists. I have been doing search work since 2008, seventeen years, fifteen of them in SEO, across more than 300 businesses in the US, Canada, the UK, Singapore, Australia and New Zealand. I am also one of the vendors you would be vetting. So the check I am handing you is the one I have to pass.
Language is the first tell. Fractl surveyed 343 US marketing decision-makers responsible for AI visibility and released the findings on July 21, 2026. 81% of them still call this work SEO. Only 19% have adopted GEO, short for generative engine optimization, which means work aimed at getting your business named inside AI answers. Four out of five buyers do not use the word half the proposals are built around.
A Search Engine Land write-up of the same research, published August 24, 2026, reported a finer cut from what looks like a differently worded question: 46% preferred "AI search optimization" and 24% said "SEO." Two questions, two pictures. Survey wording moves answers, which is its own small lesson about any number that arrives without a method attached to it.
The same research found decision-makers put about 24% of their search and content budget into AI visibility work. A quarter of the budget, aimed at a discipline with no agreed name, sold by people who all sound identical. Hire wrong and the cost is not only the retainer. It is the time you spend not fixing whatever was actually broken, while a competitor's name is the one the assistant says out loud.
The fanciest pitch on your desk is the one to walk from
Ask buyers what turns them off most in an AI-search pitch and the top answer is not price and not inexperience. 36% named heavy buzzwords with no clear explanation, more than any other complaint. Missing case studies or trackable results came second at 21%. Vague, unsubstantiated performance claims came third at 20%. SEO repackaged as AI came fourth at 16%.
Those four complaints are one complaint wearing four coats: I cannot check anything you just said. A pitch loaded with technical language does not read as expertise to the person signing. It reads as a wall built between you and the part where someone shows their work.
Jargon survives because it works on the wrong audience. It impresses other marketers, it fills a slide, and it makes an unverifiable promise sound like engineering. Anyone who does this work can explain it to a business owner in plain sentences. A vendor who cannot either does not understand it or would rather you did not.
So translate the pitch back into English before you decide anything. Sit with the proposal and put every phrase through the middle column below. If the vendor struggles to say the middle column out loud, you have your finding.
| What the pitch says | What it means in plain English | What to ask next |
|---|---|---|
| "We optimize your entity graph." | We make your business name, services and locations consistent everywhere machines read them. | Show me two pages you changed and tell me exactly what changed on them. |
| "Our proprietary RAG pipeline." | RAG means retrieval augmented generation: the assistant looking things up before it answers. That is how the assistant works, not something a vendor owns. | Which part of that do you control on my site? |
| "Definition-lead architecture." | We put a plain one-sentence answer at the top of the page. | Show me one of those pages before and after. |
| "We stack schema for AI retrieval." | Schema is code that labels what a page is about so machines classify it correctly. | Which types, on which pages, and what did you watch change afterward? |
| "Full-funnel generative visibility strategy." | Usually a content calendar. | What gets published in the first 30 days, and who writes it? |
Buyers already voted on what counts as proof, and vocabulary finished last
Put the same question to those 343 decision-makers from the other direction, what makes a vendor credible, and the ranking is blunt. Case studies with measurable, verifiable results: 34%. A clear explanation of methodology: 22%. Team expertise and track record: 15%. Fluency in the terminology: 9%.
Add the first three and 71% of buyers are choosing on something they can check. Fewer than one in ten are choosing on how the vendor talks. If most of the pitch goes to how the technology works and almost none to a past client, the meeting was built backwards. Flip it. Spend the first twenty minutes on one client, one outcome, one date, and let the technology explain itself through that story.
| What buyers say makes a vendor credible | Share | What that looks like in the meeting |
|---|---|---|
| Case studies with measurable, verifiable results | 34% | A dated starting point, a named change, and an outcome in the client's own numbers. |
| A clear explanation of methodology | 22% | They describe the order of the work and why step two follows step one. |
| Team expertise and track record | 15% | You meet the person doing the work, not only the person selling it. |
| Fluency in the terminology | 9% | The thing most pitch decks are built on. |
Read the bottom row again, because it is the expensive one. The signal vendors optimize for hardest is the signal buyers rank last. That mismatch is your advantage in a negotiation. When the room fills with acronyms, you are watching effort go into the 9% while the 34% sits untouched in a folder nobody opened.
A receipt has four parts, and most case studies have one
The most common claim in this market is a percentage: we drive a share of your AI citations. It sounds like measurement, and it has the shape of a number, and it means nothing until someone tells you how it was produced. Five questions take it apart. Which assistants. How many prompts. Written by whom. Run how often. And what exactly counts as a citation, a link, a brand name in the text, or the business being recommended. A vendor who chose the prompts chose the score.
Guaranteed placement is the other pattern, and it is simpler to judge. Nobody controls what a language model says, not the vendor and not the model's own maker on a given day. So a guaranteed spot inside ChatGPT's answers is either a promise the seller cannot keep or a plan to buy their way in, which is the same trade as the emails offering to add your business to ChatGPT's index for a fee. Those are two different problems and they need different responses. The outright con you delete. A real agency doing shallow work is fixable, but only with a scope that names deliverables and dates.
Then there is the screenshot. One prompt, one day, one account with memory switched on and the vendor's own history behind it. A screenshot proves a model said something once, to someone, under conditions you cannot see. It belongs in a pitch the way a photo of one good day belongs in a weather forecast.
I will hold my own two to the same standard. An ADHD clinic in London ended up booked solid for three straight months, and the demand pushed them to hire more specialists and outsource the overflow. You cannot verify that from this page, and I am not going to pretend otherwise. What you can do is ask me on a call to walk through what changed and in what order, then judge whether the sequence sounds like someone who did the work or someone who read about it.
The second one is better precisely because it needs no trust from you. This site scored 59 on PageSpeed before I rebuilt it. Run the test yourself, right now, on the page you are reading, and see where it lands today. Nothing about that depends on my honesty. That is what a receipt is, and it is the standard you should hold every vendor to, including this one.
Five questions that cost you nothing and end a bad meeting early
Ask these before the deck opens, in the first ten minutes, in whatever order feels natural. None of them require you to understand how a language model works. All of them are hard to fake, which is the entire design. A real operator answers four of the five without breaking stride and says "let me check" on the fifth.
There is a sixth I would ask if I were sitting on your side of the table. What changed in the last 60 days in how these assistants pick sources, and how did you find out? Speed of change was the top challenge decision-makers named in that survey, at 28%, which makes a vendor's reading habit part of the deliverable. A shrug means you are buying last year's snapshot. A specific answer, with where they saw it, means someone is paying attention on your behalf.
You can also run the cheapest test of all before any meeting happens. Open ChatGPT and ask the question a customer would ask, something like who is the best AI search consultant for a dental practice in Leeds, and see whether the vendor's own name comes up anywhere in the answer. It is not a disqualifier on its own, since a busy operator can be quietly booked out. It is a conversation worth having out loud.
Your customers run this test on your website, and most sites fail it
Turn that checklist around and it stops being about hiring. The buyer reading your homepage is doing what you just did to that proposal: deleting the adjectives and looking at what survives. "Award-winning." "Trusted by hundreds." "Patient-first philosophy." Those are your entity graphs. They feel like proof, check out to nothing, and cost you the sale in the seconds before a thumb keeps scrolling.
Picture a physiotherapy clinic in Manchester. The homepage says award-winning care and a patient-first approach, which is what every clinic within six miles also says. The version that wins names the thing: we see runners with knee pain, here is the treatment window, here is what happened to the last group of them, here is the month it happened. The clinic owner already knows those numbers. The front desk quotes them on the phone every day. They are simply not on the website, where the deciding actually happens.
The survey's four signals map straight onto your own pages. Verifiable results, 34%: do you have one page with a real outcome and a date on it? Clear methodology, 22%: do you explain how you work, in order? Named expertise, 15%: is there a human on the page with a track record a stranger could check? Terminology, 9%: the industry words you agonize over matter least. Most sites are built in exactly the reverse order.
There is a second payoff, and it is machine-shaped. The page that convinces a skeptical buyer is also the page an assistant can quote, because a model looking for a source needs something specific to lift and adjectives give it nothing to hold. Whatever you leave blank gets filled with whoever else published something concrete, which is the case for publishing verifiable proof before AI fills that gap with a competitor. One piece of work, two audiences, and the human one is still the audience that pays.
The report that flatters you is not the report that pays you
Ninety days in, a deck arrives with a line going up and to the right. Before you feel good about it, ask what would have to be true for that line to go up while nothing changed for your business. Mentions counted with no fixed prompt list grow by writing more prompts. Share of voice is a grade set by whoever wrote the exam. Averages across a dozen assistants hide the only one your customers use.
That last one has teeth. Decision-makers said they prioritize ChatGPT at 34%, then Gemini at 16%, Claude at 6%, and Copilot or Bing at 5%. If your customers ask ChatGPT, a blended score across five assistants can climb while ChatGPT quietly stops naming you. Ask for the platform split, every time, and read the column that matters instead of the average that soothes.
| The number in the report | Why it moves | Ask for this instead |
|---|---|---|
| Total AI mentions | Rises whenever someone adds prompts to the tracker. | Hit rate on a fixed prompt list that has not changed since month one. |
| A screenshot of one great answer | One run, one day, one logged-in account with its own history. | The same prompts run on a schedule, logged out, with dates attached. |
| Share of voice, as a percentage | Depends entirely on which prompts were chosen, and by whom. | The prompt list itself, written and frozen before the work started. |
| AI referral traffic, on its own | Some assistants send few clicks even when they recommend you by name. | Calls and bookings, plus what people say when you ask how they found you. |
Hit rate is the honest version of all of this: out of a fixed set of runs, on the questions your customers actually ask, how many name you. It moves slowly, it embarrasses everybody at first, and it cannot be inflated by adding prompts. That is also why asking ChatGPT once whether it recommends you tells you nothing, and why a boring schedule beats an exciting screenshot every quarter of the year.
Set the timing expectation before you sign so nobody has to argue about it later. Page work shows up the week it ships and you can look at it yourself. Citations move over months, because the sources these assistants lean on have to notice you first. And accuracy deserves its own line in the report: when an assistant does name your business, check what it says about your prices, your service area and your hours, because a confident wrong answer sends the customer to a competitor with your recommendation attached.
Frequently Asked Questions
How do I know if an AI search agency is legit?
Ask for one case study with a date, a named starting point, and a result in the kind of numbers your business runs on, then ask to see a page they changed so you can look at the actual work. Ask how they measure: which assistants, which prompts, who wrote them, and how often they run. Verifiable case studies were the top credibility signal in Fractl's July 2026 survey of 343 US marketing decision-makers, at 34%, with clear methodology second at 22%. Walk away from anyone guaranteeing placement inside ChatGPT, because no vendor controls what a model says on a given day.
Is GEO just SEO with a new name?
The label is mostly noise. GEO stands for generative engine optimization, meaning work aimed at getting your business named inside AI answers, and in the same 2026 survey 81% of marketers still call the whole discipline SEO while only 19% have adopted GEO. The underlying craft overlaps heavily: clear pages, real proof, consistent business facts, and a technically healthy site. What matters is whether the vendor can show you work, not which acronym is printed on the invoice.
What should an AI visibility report actually show me?
The prompt list first, written before the work started and unchanged since, so nobody can grow the score by adding questions. Then your hit rate on that fixed list, run on a schedule under the same conditions, with dates. Then accuracy: what the assistant says about your prices, service area and hours when it does name you. Finally your own business numbers, calls and bookings, and what customers say when you ask how they found you. If the report leads with a big total and never shows the prompt list, you are reading marketing, not measurement.
Every vendor in this market, me included, is asking you to believe a story about work you will never sit and watch happen. A receipt is the only defense you have, and it costs one question to ask for. If you want a second pair of eyes on a proposal already sitting in your inbox, bring it to a call and we will read it line by line, including the parts that would fail my own test: book an AI visibility call. Before that, though, open your own homepage and delete the adjectives, because the customer who left last Tuesday already did.
