Only 40% of AI-cited statistics trace back to a study

Hacker News by 15 min read 78x views
Only 40% of AI-cited statistics trace back to a study

Share Post

One number, followed.

We asked ChatGPT, Perplexity, Gemini, Claude and Google’s AI Overviews 200 questions that call for a number, the benevolent group ask all day. Then we clicked the citations, and kept clicking. Here is one trail, from the answer to the end.

Google AI Overviews

What percent of salespeople provision up following one follow-up?

44% of salespeople provision up following fair one follow-up attempt.

A revenue firm’s stats page2020

44% of salespeople provision up following one follow-up attempt (source).

A freelance writer’s example articleby 2017

80 Percent of Sales Require 5 Follow-up Calls following Meeting; 44 Percent Salespeople Give Up After One Follow-up

No study rearward itNo survey, no sample, no date.

  1. In the answer

    Every aide got the identical question: “What percent of salespeople provision up following one follow-up?” All five assistants gave the identical figure, and lone ChatGPT stated the evidence rearward it was unclear.

  2. One click in

    Google AI Overviews cited a revenue firm’s stats page, from 2020. The figure is correct there. Its “source” nexus goes to a penning sample.

  3. Two clicks in

    That leads to a freelance writer’s example article, online by 2017. The figure is in its headline. It lists two glide decks, a blog catalog and two news stories. None is a study of follow-ups.

  4. The end of the trail

    That’s anywhere it ends. No survey, no sample, no study: all another path to this figure we tried stopped at a leaf that cites no research. It’s a figure motionless strolling about alongside nothing underneath it. That’s a zombie stat.

Most numbers aren’t zombies. But you can’t inform from the answer: a zombie arrives alongside the identical confidence and the identical citation, and here, alongside all five assistants agreeing. So we did the identical for 1,285 statistics.

Fewer than fractional get there.

We took a random example of the data all aide cited, 1,285 in all, and walked all trail the identical way: the cited page, afterward that page’s source, up to five pages deep. Where a leaf named a origin without a link, we searched for it.

40% reached a study that says the identical thing. Another 8% reached the study, but the figure or its environment had changed on the way. The remainder never got there: 40% stopped at a origin we couldn’t check, since it was locked, gone or silent on the number, and 11% led nowhere at all.

Four in ten trails reached a study that says the identical thing

Each quadrate is one in a hundred of the 1,285 data we traced, sorted by anywhere its trail ended.

  • 40%SourcedWe established the study, and it says the identical thing.
  • 8%ChangedWe established the study, but the number, who was asked, anywhere or whenever is different.
  • 40%Can’t checkA leaf names a genuine source, but the figure isn’t in it, or the origin is locked or gone.
  • 11%NowhereNo study anyplace on the trail. Every leaf repeats the figure and credits no one.

The final 1%: trails whose pages wouldn’t burden for us.

ChatGPT did best, and quoted the fewest numbers: 1.3 per answer, against 6.8 for Claude. The another four landed between 38% and 46%, too near to rank.

ChatGPT quoted the fewest numbers, and originated the most

The identical hundred squares for all assistant, from its own example of cited statistics.

  1. ChatGPT

    57% sourced

    1.3 data per answer

  2. Perplexity

    46% sourced

    2.2 data per answer

  3. AI Overviews

    39% sourced

    4.8 data per answer

  4. Gemini

    38% sourced

    6.2 data per answer

  5. Claude

    38% sourced

    6.8 data per answer

It isn’t a quirk of our questions. Of 300 data in Google AI Overviews answers to genuine mundane searches, 40% were sourced, the identical share.

AI cites the copy, not the study.

Most of the time, the AI wasn’t making numbers up: the leaf it cited had the fig 82% of the time. The difficulty is which page. Only 21% of the pages cited for a figure were the investigation itself.

More than fractional were business blogs. Add news stories and data roundups, and most citations district on pages built to collect figures from elsewhere, frequently without a nexus back.

Where the trail starts decides anywhere it ends

Every leaf an aide cited archetypal for a number, by kind. Each column’s width is its portion of those citations. Green is how frequently its trails reached a study that says the identical thing.

62%

Research firm13% of citations

Research resolute or survey: 13% of archetypal citations, 62% of them sourced.

61%

Government7% of citations

Government or authoritative body: 7% of archetypal citations, 61% of them sourced.

37%

Company blog54% of citations

Company blog: 54% of archetypal citations, 37% of them sourced.

37%

News10% of citations

News or magazine: 10% of archetypal citations, 37% of them sourced.

35%

Stats roundup8% of citations

Stats roundup: 8% of archetypal citations, 35% of them sourced.

41%

Other8% of citations

Other: 8% of archetypal citations, 41% of them sourced.

Where the trail starts decides anywhere it ends. From a investigation resolute or a authorities page, it reached a matching study 61% of the time. From a business blog, 37%; from a data roundup, 35%. Even investigation firms cut abbreviated 38% of the time, frequently since the complete study sat rearward a form.

A citation tells you anywhere the AI established a number, not anywhere the figure came from.

Numbers alter on the way.

Reaching the study didn’t continually average the figure held. Of the data that got there, 17% had changed on the way. The digits moved lone concerning one period in four. Usually it was the context: what was measured, the year, the source, or who was asked.

What changed, throughout all alter we logged

The figure changed26%

What was measured changed20%

The twelvemonth changed17%

The origin changed16%

Who was asked changed14%

The location changed4%

Fifteen women judging health websites rotate into all website’s archetypal impression. Three companies rotate into a regulation of business. Here are the clearest trails we followed, all one checked by hand.

Watch a figure endure its context

Each trail runs from the first study to an answer an aide gave in October 2026. Step alongside it: the figure stays, and the words about it change.

AskedWhat percent of archetypal impressions of a website are design-related?

The study · Sillence and colleagues, CHI, 2004

Of the comments 15 women made about rejecting or mistrusting health websites, 94% were about design.

A 2004 study watched 15 British women fairness health websites, and 94% of their comments concerning rejecting a location were concerning design. Twenty years of retelling made it “94% of archetypal impressions are design-related”, and Google AI Overviews states it as fact.

Step 1 of 4

The zombies.

Ask five assistants the identical inquiry and you’ll frequently get five distinct numbers. On 28 of the 71 questions anywhere all five gave a percentage, we got four or five distinct figures.

One question, five answers, from 49% to 96%

Asked “What percent of marketers use promotion automation tools?”, term for word, on the identical day.

0%100%

49–51%AI Overviews

62%ChatGPT

70–79%Gemini

81%Perplexity

96%Claude

  • ChatGPTThere isn’t one worldwide figure, it varies by study and what counts as “marketing automation.” One commonly cited study established 62% of U.S. digital marketers used automation software, but that data is from 2020, so it may not indicate current adoption.

  • PerplexityApproximately 81% of promotion organizations use promotion automation.

  • GeminiAcross important industry benchmarks, between 70% and 79% of marketers and companies use promotion automation tools in several capacity

  • Claude96% of marketers have used a promotion automation phase or scheme to use one

  • AI OverviewsRoughly 49% to 51% of companies and promotion teams currently use promotion automation tools, according to industry benchmarks from Moosend and connected investigation studies.

It wasn’t one odd question. The representative gap between the highest fig and the lowest was 16 points.

On 31 of 71 questions, the five figures were 20 points or additional apart

Every inquiry anywhere all five gave a percentage, one row each, from the lowest fig to the highest. Closest at the top.

What percent of searches contain local intent specified as nearby or near me? 46% to 46%How many social media users prosecute at smallest one brand? 90% to 90%What percent of online buying carts are abandoned? 70% to 70%What is the average conversion charge for ecommerce websites? 3% to 3%What percent of students obtain at smallest one online way during their studies? 54% to 54%What percent of social media users admission platforms on mobile devices? 99% to 99%What is the average conversion charge for websites? 3% to 4%What percent of revenue leads change to customers on average? 4% to 5%What percent of small businesses last their archetypal five years? 49% to 51%What is the average involvement charge for integrated social media posts? 0% to 2%What percent of workforce activity remotely at smallest part time? 22% to 24%How many group transport credit cardstock balances duration to month? 45% to 48%What percent of small businesses are owned by women? 40% to 43%What percent of web traffic comes from mobile devices? 57% to 60%What percent of organizations study AI has improved decision making? 50% to 53%What percent of users use ad blockers on websites? 30% to 33%What is the average completion charge for enormous open online courses? 7% to 10%How frequently do users click the archetypal integrated result? 25% to 29%What percent of households own a smartphone? 87% to 91%What percent of group have pupil debt debt? 16% to 21%How frequently do websites use receptive design? 90% to 95%How many households have no emergency savings? 21% to 27%What percent of guidance positions are held by women? 27% to 33%How many students autumn out before completing their flat or certificate? 30% to 39%What percent of ecommerce income is generated by reiterate customers? 40% to 50%How many small businesses have a website? 73% to 83%What percent of group live paycheck to paycheck? 48% to 59%What percent of a salesperson's period is spent really selling? 29% to 40%What percent of workers execute gig or freelance activity in supplement to a chief job? 8% to 20%What percent of promotion budgets is typically spent on digital channels? 55% to 68%How frequently do users click on featured snippets or answer boxes? 9% to 22%What percent of shoppers forsake checkout since of shipping costs? 40% to 54%What percent of searches are performed on mobile devices? 53% to 67%What is the average percent of business prosperity spent on AI? 2% to 17%What percent of application features are rarely or never used? 64% to 80%What percent of data breaches are caused by individual error? 60% to 78%What percent of adults use several form of medication medication regularly? 50% to 68%What is the average win charge for outbound leads? 2% to 20%What percent of emails are opened on mobile devices? 42% to 61%How many adults cognition sleep disorders specified as insomnia? 13% to 33%What percent of liberated test users change to paying customers? 5% to 25%What percent of SaaS expend goes on unused licences? 30% to 50%What percent of students study concerns concerning mental health during studies? 63% to 85%How many marketers study using email promotion regularly? 63% to 86%What percent of marketers use social media for promotion? 73% to 97%What percent of adults use social media? 70% to 95%What percent of adults are classified as obese? 16% to 42%What is the average win charge for inbound leads? 4% to 30%What percent of customers volition pay additional for a improved experience? 59% to 86%What percent of group peruse the conditions and conditions before accepting them? 9% to 37%What percent of application betterment teams use AI code assistants? 66% to 97%What percent of households have net access? 58% to 93%How many group have credit cardstock debt? 23% to 60%How many AI projects neglect to attain production? 46% to 88%How many online purchases are made via marketplaces versus idiosyncratic stores? 30% to 72%How many online shoppers use social media to detect products? 39% to 82%What percent of marketers use data analytics for run decisions? 44% to 90%What percent of marketers use promotion automation tools? 50% to 96%How frequently do organizations cognition ransomware attacks all year? 25% to 72%What percent of companies use AI in their operations? 37% to 88%How frequently do group inspect social media all day? 33% to 85%How frequently do workforce study emotion burned out? 27% to 82%What percent of consumers use social media to investigation purchases? 26% to 82%What percent of students enroll in higher learning following finishing secondary school? 44% to 100%What percent of consumers disregard online ads? 37% to 93%What percent of consumers rely AI recommendations for purchases? 33% to 89%What percent of small businesses market products or services online? 23% to 85%How much additional apt are customers to buy whenever liberated shipping is offered? 20% to 82%What percent of data breaches affect compromised credentials? 22% to 90%

Closest20 points or additional apartFurthest apart

Agreement feels akin proof. It isn’t. All five gave the identical fig on 10 questions, and of the 5 we traced, 2 were sourced. The data they repeated most, quoted by three or more, did no better: 11 of 29.

And four were quoted by all five assistants, alongside no study anyplace rearward them.

Quoted by all five. Sourced by none.

Four data all aide gave us, all alongside a trail that ends at a leaf citing no research.

  • 44%

    Salespeople who provision up following one follow-up

    Every trail ends at a leaf that cites no research, one of them a freelance writer’s example article. Its complete trail opens division 1.

    Quoted by all five

  • 90%

    Social media users who prosecute at smallest one brand

    Stated alongside no source. No study alongside this fig was found.

    Quoted by all five

  • 75%

    Searchers who never scroll former the archetypal leaf of results

    Repeated alongside no source, frequently credited to an old promotion roundup that no longer carries it. No study was found.

    Quoted by all five

  • 5 to 25 times

    How much additional it expenses to win a client than to keep one

    A 2014 Harvard Business Review part gives the range “depending on which study you believe”, and names none.

    Quoted by all five

The 29 data AI repeats most

Each was quoted by at smallest three of the five assistants, most frequently first, and its trail checked by hand.

  1. 95%

    Generative AI pilots that neglect to provision a measurable financial return

    MIT NANDA’s 2025 study says 95% of organizations got no come back from generative AI. That isn’t the identical as 95% of pilots failing.

    5 of 5ChangedStudy from 2025
  2. 44%

    Salespeople who provision up following one follow-up

    Every trail ends at a leaf that cites no research, one of them a freelance writer’s example article. Its complete trail opens division 1.

    5 of 5Nowhere
  3. 4% to 10%

    Unhappy customers who really complain

    Goes rear to TARP’s 1979 investigation for the White House person office. Its author says rates that low use to small purchases; in 2016, 22% to 41% complained concerning their most grave problem.

    5 of 5ChangedStudy from 1979
  4. 2 hours 21 minutes

    Time group expend on social media all day

    DataReportal’s Digital 2025 report, from GWI study data: the representative net person spends 2 hours 21 minutes a day on social media.

    5 of 5SourcedStudy from 2025
  5. 46%

    Google searches alongside local intent

    Traces to a 2018 social article concerning a glide at a hunt conference. Google never published it, as far as we could find.

    5 of 5Can’t check
  6. 90%

    Social media users who prosecute at smallest one brand

    Stated alongside no source. No study alongside this fig was found.

    5 of 5Nowhere
  7. 94%

    First impressions of a website that are concerning its design

    Began as a 2004 study of how 15 women judged health websites. Its complete trail is in division 4.

    5 of 5ChangedStudy from 2004
  8. 75%

    Searchers who never scroll former the archetypal leaf of results

    Repeated alongside no source, frequently credited to an old promotion roundup that no longer carries it. No study was found.

    5 of 5Nowhere

What to do alongside an AI statistic.

Treat it as a lead, not a source. Six habits, three for all flank of the citation.

If you use them

  1. Follow the citation, afterward prosecute it again.

    If the cited leaf is a catalog of data or a business blog, use its own nexus to go one leaf further. Keep going until you attain the study itself, or until you can’t.

  2. Check who was asked, how many, and when.

    Most of the changes we established were in the context, not the digits: one country’s poll, a ten-year-old study, fifteen people.

  3. Don’t obtain accord as a check.

    Five assistants giving the identical figure isn’t five sources. The numbers they repeated most were no improved originated than the rest.

If you publish them

  1. Put the method next to the number.

    The sample, the date and a nexus to the source, on the identical leaf as the figure, so the next page, and the next answer, can trace it rear to you.

  2. Keep the findings in the open.

    Reports rearward a form were a average deceased end: equal trails that started at a investigation resolute cut abbreviated 38% of the time. Put the header numbers and the method on an open page.

  3. See which leaf AI cites for your numbers.

    It is frequently a roundup or a blog that copied you, sometimes alongside an older figure. That leaf decides what group hear.

Other Article Hacker News
↑
Close Right Ads
Close Left Ads