AI Visibility Numbers Are Unreliable. Measure Them Anyway.

AI Search Engine Technology Concept. Person utilizing laptop pinch AI-powered hunt barroom interface and integer icon representing chatbot, neural network, unreality computing, large data, world hunt technology

getty

Two things astir generative motor optimization are existent astatine the aforesaid time. The measurement underneath it is unreliable capable that astir of the numbers being reported to boards this twelvemonth would not past a statistic review. And the brands waiting for amended measurement earlier they commencement will beryllium a twelvemonth aliases 2 down the ones that don't.

That hostility is worthy moving done carefully, because a ample magnitude of money is astir to move connected the spot of numbers almost cipher tin audit.

What Are Companies Actually Buying?

Generative motor optimization (GEO), besides called reply motor optimization (AEO), is the believe of trying to power whether and really a marque appears erstwhile a customer asks an AI adjunct a question. The adjacent practice, AI visibility monitoring, tries to measurement really often that happens.

The framing matters. GEO is not a channel. It is simply a furniture of influence, and the click from a citation wrong ChatGPT is the smallest and slightest absorbing portion of it. The applicable mobility is not really galore income an supplier completed. It is really galore purchases an supplier shaped -- the aforesaid favoritism that separates agentic commerce from agent-executed commerce, and societal commerce from in-platform checkout.

That favoritism is besides the first hint astir why this is truthful difficult to measure.

Why Is The Measurement So Unreliable?

Six problems, and they each compound.

1. The motor regenerates the reply each time. Ask the aforesaid adjunct the aforesaid mobility doubly and you often get different brands and different sources. A study published connected arXiv successful June measured really overmuch the cited root group overlapped crossed repeated runs of identical queries: astir 30% connected Gemini, 33% to 40% connected SearchGPT, and astir 50% connected Perplexity. Instability is worst precisely wherever it matters most. Purchase-intent prompts were the slightest accordant query type tested successful an study of 14,000 API calls, pinch only astir 4 of each 10 brands appearing crossed 2 runs of the aforesaid prompt, according to Conductor.
2. Most devices measurement a different merchandise than the customer uses. Tracking platforms typically telephone a developer API. Customers unfastened a user app carrying memory, personalization, location, exemplary routing and its ain logic astir erstwhile to hunt the web. The spread shows up moreover wrong a azygous product: only 25% of cited sources overlapped betwixt ChatGPT's accelerated Instant mode and its slower Thinking mode, and the 2 sometimes recommended different brands for the aforesaid prompt, successful an analysis tally by Semrush pinch Kevin Indig. Who the customer says they are matters too. One scale covering 126 package categories found a marque moving from winning 34% of answers to 90% depending connected whether the purchaser described themselves arsenic a freelancer aliases an enterprise.
3. The tracked punctual is not the punctual that runs. Assistants decompose a mobility into sub-queries and tally them successful parallel, a process Google calls query fan-out. Google has said its Deep Search characteristic tin rumor hundreds of sub-queries for a analyzable question. A dashboard search 1 building is grading a trial the motor ne'er took.
4. Citations are not sources. Many systems make an reply and connect supporting links afterward. Researchers telephone the nonaccomplishment mode post-rationalization: the citation is plausible but was not what produced the claim. The spread is measurable. Across 278 answers covering 95 tracked brands, ChatGPT named a marque successful its prose 552 times but cited that brand's ain domain only 375 times, and the 2 overlapped conscionable 185 times, according to an study by Position Digital. Being cited and being recommended are different events. A citation study measures 1 of them.
5. The sound level exceeds the effect. The aforesaid arXiv investigation put bootstrap assurance intervals connected citation stock and recovered emblematic intervals spanning 3 to 7 percent points. Its blunt conclusion: an evident move from 8% to 11% cannot beryllium distinguished from random variation. Reaching a defensible estimate required 40 to 50 queries per taxable connected 1 level and 150 aliases much connected another. Most marque dashboards tally a fistful of prompts play and study to 1 decimal place.
6. Visibility is not an outcome. A mention is not a sale. The referral postulation assistants nonstop straight is simply a mini fraction of tract visits, and the existent power is almost surely overmuch larger than what shows up successful a referral log -- which is precisely the problem. Neither number tells a main financial serviceman whether the activity paid for itself.

Why Doesn't Getting It Right Stay Right?

This is the portion executives find hardest, and the 1 that should reset expectations.

Search is adjacent to deterministic. The aforesaid query returns astir the aforesaid classed database to everyone, and a position earned done bully activity is simply a durable plus -- often six months aliases much of advantage until the algorithm updates. That durability is what made SEO investable.

Generative systems are probabilistic. The reply is composed caller connected each request. There is nary position to clasp and nary authorities to defend. Give a marque a magic wand coming that revealed its nonstop opinionated and the precise levers to amended it, and the image would drift by adjacent week without anyone rubbing thing -- because the exemplary changed, the retrieval furniture changed, a competitor published, aliases the customer's ain history changed.

GEO is person to moving a continuous research than to holding a ranking. Teams that fund for it arsenic a task will beryllium disappointed. Teams that fund for it arsenic an ongoing measurement usability will not.

What About Vertical Versus Horizontal Agents?

Most GEO conversations softly presume 1 benignant of agent. There are two.

Horizontal agents -- ChatGPT, Gemini, Claude, Perplexity, Copilot -- are trained connected the unfastened web and beryllium closest to discovery. Vertical agents are retailer-owned and catalog-bound -- Amazon's Alexa for Shopping (formerly Rufus), Walmart's Sparky, and the assistants appearing wrong grocery, drugstore and specialty apps -- and they beryllium closest to the transaction. Alexa for Shopping is trained connected Amazon's catalogue and its Prime customer behavior. ChatGPT is not. Visibility successful 1 says very small astir visibility successful the other.

Almost nary instrumentality attempts both. The monitoring class is built astir horizontal assistants because those tin beryllium prompted from the outside. Retailer agents cannot beryllium queried astatine standard and return almost nary information to brands. So the aboveground nearest the acquisition is the 1 fewer brands effort to measure, and the aboveground furthest from it generates each the dashboards.

What Kind Of Data Is The Measurement Built On?

Two fundamentally different methods, and the favoritism seldom appears connected a income slide.

Synthetic punctual monitoring generates a punctual set, runs it against the engines connected a schedule, and reports what came back. Repeatable and auditable. Its weakness is that personification had to conjecture what customers ask.

Real punctual monitoring uses consented user panels and observes conversations that really occurred. One specified sheet from Measure Protocol, spanning apps, browsers, hunt and AI assistants, reported successful 2025 that more than 1 successful 5 ChatGPT conversations showed commercialized intent. Several GEO vendors now build panels of this benignant into their products. Panel information is the closest point this section has to observed alternatively than stated preference, and observed penchant is almost ever worthy more.

Neither is capable alone. Synthetic testing shows really engines respond to a controlled stimulus. Panels show what group really ask. The useful programs tally both.

One be aware applies to each vendor here: very fewer disclose really the information is collected. Which surface, which exemplary build, logged successful aliases out, which region, really galore repetitions per prompt, really variance is handled. A purchaser who cannot get those answers successful penning is not buying a measurement. They are buying a number.

What Happens As The Budgets Grow?

The fraud grows pinch them, and the class already contains strategies pinch nary grounds down them. Ninety-seven percent of published llms.txt files received zero requests successful May, and among files that did get postulation the largest azygous class of requester was SEO audit devices checking whether the record existed, according to an Ahrefs study of server logs from 137,000 domains. Google has said the record is not needed. It is still being sold.

Expect worse arsenic spending rises: fabricated benchmarks, contented seeded to crippled retrieval, and proprietary scores only the vendor's ain methodology tin improve. Expect the gait of alteration to accelerate alternatively than settle.

So Why Build The Practice Now?

Because successful 2026, the deliverable is not the number. It is the organization.

• Executives are already moving prompts. A main executive who typed the class into ChatGPT past week saw thing and drew a conclusion. If a competitor came up first, that conclusion is already forming.
• Every vendor will nonstop a visibility study to personification senior, and they will each disagree, because they are measuring different surfaces pinch different methods. Without an soul number, a institution is refereeing a conflict among group who profit from the fight.
• A accordant number pinch a published methodology thumps an meticulous number cipher shares. Teams request thing to reason from alternatively than about.
• Change guidance is the binding constraint, arsenic pinch each commerce shift. Getting merchandising, brand, contented and IT into 1 speech takes astir 18 months. Start while the stakes are low.
• Most of the underlying activity carries nary regret. Accurate, structured, machine-readable merchandise contented improves integrated rank, conversion, unit media ratio and vertical supplier performance. That lawsuit does not dangle connected wide AI shopping adoption. It depends connected the remainder of the business.

There is existent grounds the power is worthy chasing. Users who saw a marque recommended by ChatGPT were 2.5 times much apt to sojourn that brand's tract wrong 7 days, pinch 56% of that postulation arriving done branded hunt alternatively than an AI referral, according to Similarweb clickstream research. Similarweb labels the uncovering relationship alternatively than causation, and the study covers United States desktop users crossed 3 verticals. It looks a awesome woody for illustration measuring a billboard: lift, not clicks.

One caveat belongs connected each of it. Nearly each study cited here, connected some sides, was produced by a institution that sells package successful this category. That does not make the findings wrong, but it should style really confidently they are read. The astir rigorous activity is academic, and its cardinal conclusion is that everyone else's numbers request correction bars.

What Should A Team Do This Quarter?

Write the methodology down earlier penning the number down, and people some together. Report ranges alternatively than constituent estimates. Build the punctual group from observed customer connection -- tract hunt logs, work transcripts, reappraisal text, sheet information -- not from what the marque squad wishes group asked. Track vertical and horizontal agents separately and expect them to disagree. Keep visibility and business outcomes successful abstracted columns, past measurement incrementality the only measurement that has ever worked, done holdouts and matched-market tests. Ask each vendor, successful writing, really the information is collected. And do the unglamorous activity first: cleanable merchandise data, contented that answers existent customer questions, and a firewall that is not softly blocking the crawlers.

The measurement will beryllium wrong, successful ways that tin beryllium enumerated. That is the useful benignant of wrong. The capacity being built -- a squad that tin logic astir probabilistic, agent-mediated customer behaviour -- will matter for considerably much than chatbot mentions.

Just don’t put 3 decimal places connected it.