ChatGPT, Perplexity, Grok, and Google AI all cited posts concerning a made-up idea inside seven days of the posts going up, according to a paper from researchers at Fudan University.
The researchers published the posts from average accounts on nine sites the tested tools already cited often. On two another topics, an opinion inquiry alongside existing conversation and a made-up product, the identical four tools picked up small in the identical seven days.
The document uses the tag “Google AI” and links to Google’s broad AI site, but it doesn’t define which Google merchandise was tested. The tests were conducted in March and April, and the document was posted on arXiv on Oct. 8.
How The Test Worked
In March, the Fudan squad ran 120 questions through 10 AI hunt tools and recorded the sites all tool cited. The researchers categorize ChatGPT, Perplexity, Grok, and Google AI as English-web tools; the another six, including Doubao and DeepSeek, are Chinese-web tools.
Across those questions, Reddit was the most-cited location for Google AI and Grok, YouTube for Perplexity, and Wikipedia for ChatGPT. Six to 10 of all tool’s 10 most-cited sites accepted posts from average users.
In April tests of 22 posting platforms, the researchers rated Medium, GitHub, LinkedIn, and Facebook uncomplicated to associate and article on, during Reddit’s karma requirements and strict deletions made posting harder.
For all tool, the squad afterward picked its most-cited location that a new document could article on, which gave them nine sites, including Medium, YouTube, and Sohu.
The investigation squad posted articles concerning the fictional “Cactus Gravity Protocol,” whether AI would supplant younger programmers inner of five years, and a fictional keyboard. The archetypal drafts of all part were created using ChatGPT, afterward all was manually revised and carried planted particulars (e.g., fabricated names and numbers) to authorize the authors to acknowledge if the AI tools repeated that information.
The researchers ran checks before posting to verify that none of the tools provided an informative reply to either of the fictional topics or repeated the planted information. Following posting, the researchers ran the identical five questions for all of the three topics on all tools regular for one week.
In June I covered a Cornell Tech paper on planted content that tested open-source investigation agents in a simulation. This squad posted on the live web and queried the tools through their community interfaces.
What The Four English-Web Tools Cited
Eight of the 10 tools cited the made-up concept, the Cactus Gravity Protocol, inside a week, including all four English-web tools. Counting answers from the archetypal day all tool cited the posts through the seventh day following posting, Grok connected to a article in 30 of 32 answers, Google AI in 15 of 33, Perplexity in 13 of 35, and ChatGPT in 7 of 20.
Fewer responses to questions included a planted item from the origin posts. Of Grok’s responses (n = 32), 53% did; for ChatGPT (n = 20), 10%; for Google AI (n = 33), 6%, and Perplexity (n = 35) none. In total, among all 10 tools and all three topics, there were links from 176 of the 1,004 answers to the posts during the week.
The lone English-web tool that cited an “opinion” article was Perplexity, in 11.4% of its responses (n = 35), and none of the four cited keyboard posts inside the week. The authors do not isolate which of the topic, competition, and category of inquiry caused the differences throughout the three topics, since all three changed together.
In a follow-up, the squad put one article all on Sohu, YouTube, and CSDN for 10 additional made-up topics. Nine drew at smallest one citation or planted item throughout the 10 tools, and Perplexity was among the three tools that cited the most of them. The document doesn’t interrupt out the another English-web tools there.
Placement Beat Volume In These Tests
To measure location, the researchers created paired postings concerning 5 comparatively unidentified holiday destinations. One article was located on a origin frequently referenced by the tools; the second article was placed on a location that was extremely rarely, if ever, referenced. In all cases, the article on the frequently referenced origin produced the archetypal reference/citation or planted detail.
The finest case on the lesser-referenced flank required 20 postings and 40 postings on a self-hosted WordPress location received zero references inside seven days. These are presented by the authors as results for particular topics, not as broad rules.
A distinct test tested six fictional subjects using varying numbers of Sohu postings (between 1-20). The total figure of citations throughout all ten tools risen from 24 to 128. However, ChatGPT never generated references to those posts and Google AI lone did so whenever citing the three lowest-volume topic topics. Since all of the volumes used a distinctive subject, the researchers referred to this addition as an affiliation fairly than an consequence of expanding the figure of posts.
Lastly, the investigation squad purchased a Chinese web posting bundle via a GEO vendor for $14. All 13 postings were delivered to the web inside about 28 minutes following purchase. A Doubao citation to among the postings including a planted item occurred inside one hr of shipment and is presented by the authors as one confirmed case of success, not a achievement rate.
Why This Matters
The topic all English-web tool cited inside a week existed lone in the researchers’ posts. Posts on the another two topics, an opinion inquiry and a made-up product, mostly didn’t get through to the identical tools.
None of the topics the researchers posted concerning was a genuine brand or product, so the tests don’t display whether the identical form holds for brand or merchandise searches, including ones alongside small safety elsewhere.
Google’s guide to its AI features says seeking inauthentic “mentions” “isn’t as helpful as it power seem,” pointing to center ranking systems that concentration on high-quality satisfied and another systems that obstacle spam. The document doesn’t say which Google merchandise it checked.
Looking Ahead
The researchers’ tests ran in March and April, before Google clarified on May 15 that its spam policies shield attempts to manipulate generative AI responses in Google Search. The authors say exact rates may drift as the tools change.
The researchers say they disclosed what they established to all 10 platforms, and that ChatGPT, Perplexity, and Google AI were among those that acknowledged receipt. I’ll update this narrative if a revised type of the document changes the findings.
Featured Image: miss.cabul/Shutterstock
