ChatGPT citations change when hidden search pipelines switch

Two caller analyses recovered that ChatGPT’s cited sources changed arsenic its hunt postulation shifted betwixt hidden retrieval pipelines.

The findings. Research from Chris Green and Suganthan Mohanadasan adds a caller complication to AI visibility tracking: the last reply doesn’t uncover really ChatGPT selected its sources.

• Both researchers recovered soul source-selection labels, including Labrador, Bright, Oxylabs, and SERP.
• Those labels beryllium down the answer, not successful the citation cards users see.

Hidden pipelines changed sources. Green tested 1,000 prompts up to 10 times each and captured 9,946 completed hunt runs. Most prompts stayed connected 1 retrieval source. Labrador accounted for 88.1% of superior hunt sources successful his dataset, followed by Bright astatine 9.9%, Oxylabs astatine 1.7%, and SERP astatine 0.3%.

• But 11.6% of prompts changed superior hunt root crossed repeated runs.
• URL overlap dropped from 0.273 to 0.149 erstwhile the hunt root changed. Domain overlap fell from 0.265 to 0.155. Green calculated that arsenic astir 45% little URL overlap and 42% little domain overlap.

The labels down results. Mohanadasan inspected 2 days of earthy ChatGPT web postulation from 1 logged-in Pro relationship and logged astir 1,240 root records crossed a fewer twelve searches.

• He recovered a result_source section attached to web results, pinch 4 observed values: SERP, Labrador, Bright, and Oxylabs.
• He described Labrador arsenic including established publishers and reference sites, Bright arsenic tied to Bright Data, Oxylabs arsenic tied to Oxylabs, and SERP arsenic an open-web baseline that appeared mostly successful news-style results.
• Green’s repeated-prompt trial recovered Labrador dominated his dataset. Mohanadasan saw Bright play a larger domiciled successful his sample, particularly for commercial, shopping, finance, weather, and section queries.

Some prompts skipped search. Mohanadasan besides recovered that ChatGPT classified immoderate queries earlier searching, utilizing a turn_use_case field. Some prompts were revenge arsenic matter and skipped web hunt entirely, moreover erstwhile they sounded current. In those cases, nary page could beryllium fetched, cited, aliases utilized arsenic evidence.

• More analyzable “thinking” queries behaved differently. Mohanadasan recovered that ChatGPT could instrumentality retired into galore searches, including site: probes, pricing checks, and searches for unnamed competitors.
• That behaviour changes which pages tin participate the reply process because ChatGPT whitethorn hunt rewritten queries, nonstop tract probes, aliases follow-up checks alternatively of the nonstop building a personification typed.

Fetched was not cited. Mohanadasan separated 3 outcomes: fetched, cited and mentioned.

• A page tin beryllium fetched into ChatGPT’s discourse without being shown to users. It tin beryllium cited arsenic the root down a circumstantial sentence. Or a marque tin beryllium mentioned without being the root of the claim.
• In his mini commercial-query sample, Reddit and YouTube were some fetched often, but Reddit was cited and YouTube was not. He attributed the spread to matter availability: Reddit threads expose text, while YouTube hunt results often supply metadata alternatively than afloat video transcripts.
• Vendor pages were cited for their ain facts, specified arsenic prices and specs. Third-party pages were much apt to support broader proposal claims.

Why we care. There’s nary azygous ChatGPT visibility consequence to measure. Your page whitethorn ne'er beryllium considered if ChatGPT skips search, uses different retrieval source, aliases finds a clearer third-party page to support the claim.

Readable pages were easier to use. Both analyses showed that ChatGPT’s root action depended partially connected what it could retrieve and read.

• Mohanadasan recovered cases wherever ChatGPT appeared to for illustration charismatic pricing pages, past fell backmost to third-party sources erstwhile prices were hidden down JavaScript aliases different difficult to parse.
• Green’s results showed that root routing changed which URLs and domains entered the reply set.
• Plain HTML, crawlable facts, clear pricing and specs, beardown third-party coverage, and text-heavy pages became much important erstwhile root action depended connected retrieval and readability.