Google Takes Applications For A UGC Pipeline Separate From Crawling

Search Engine Journal by 10 min read 92x views
Google Takes Applications For A UGC Pipeline Separate From Crawling

Share Post

For most websites, crawling is motionless how you enter Google Search. Googlebot finds a URL and retrieves the page; then, independently, Google’s systems decide if they desire to add this leaf to their index.

On Oct. 8, Google added Search Central records describing how approved forums and social platforms can present user-generated satisfied immediately to Google.

The UGC Fresh Data Program allows approved platforms to present satisfied and communication signals through a “dedicated data ingestion pipeline” that accepts data submission from those platforms. Google states that the pipeline is distinct from the two traditional crawlers and the Indexing API. According to Google, involvement in the UGC Fresh Data Program does not justify your satisfied volition be included in Google Search.

Google has partnered alongside idiosyncratic social platforms before, including Twitter in 2009 and Reddit in 2024. This period the criteria are posted and there’s an application form, so you can inspect most of Google’s requirements against a platform’s own pages before applying. The form asks how many monthly energetic users the phase has and whenever it launched.

What Google Posted

The Search Central records changelog links to the leaf under the admission for Oct. 8: “Added records on the UGC Fresh Data Program.” Google wrote that the leaf is meant to explain the program’s intent and assistance applicants “understand the eligibility criteria and how to participate.”

The program page opens alongside this description:

The UGC Fresh Data Program is an undertaking designed to assistance platforms display fresh, elevated quality, and authentic user-generated satisfied (UGC) throughout assorted features on Search.

Google says sending satisfied through the pipeline “helps justify that caller satisfied is processed and quickly updated in Search.” The leaf gives no period example for “quickly.” It additionally says the program “doesn’t justify that satisfied volition appear in Search.”

The leaf limits the program to “trending UGC content.” It describes the pipeline as “independent of the Google Indexing API and traditional integrated crawling for broad web content.”

Oct. 8 is whenever the leaf went up. Neither the leaf nor the changelog admission says whenever the pipeline started operating.

Who Google Says Can Apply

Google lists six criteria, using the labels below.

  • Content focus. The phase must chiefly presenter “UGC, social postings, or forum discussions.” Content should sit on dedicated pages alongside stable URLs, “not on overview or nourish pages.”
  • Volume and audience. “A elevated quantity of UGC satisfied and a important person base.”
  • Content accessibility. Content must be on community pages that Googlebot and users can reach. Anything rearward a login or paywall doesn’t qualify, and all UGC must be tied to “a creator alongside a community profile.”
  • Technical readiness. The phase must be capable to execute “secure OAuth 2.0 API authentication” and build “strictly validated JSON-LD payloads.” It must already use, or be prepared to implement, valid on-page schema.org markup. Google’s example is SocialMediaPosting or DiscussionForumPosting alongside interactionStatistic sub-fields.
  • Content health and moderation. No forbidden content, affirmative energetic moderation that includes a way for users to study posts.
  • Content freshness. Content must be “as caller as possible, ideally sent inside minutes,” and the phase must be capable to present “regular updates of involvement counters (within 72 hours of creation).”

Submitting an use “doesn’t justify participation,” according to the page, and Google volition portion developer documents “once accepted into the program.”

The form has seven required fields: business name, assistance domain, 10 example URLs alongside UGC, monthly energetic users, initiate date, a Search Console owner’s email and a communication email. It calls a submission “merely an expression of interest” and says Google “has no duty to merge your satisfied in our features.”

The form goes additional than the leaf on anywhere satisfied could display up, saying submitted satisfied “is eligible to assist on Google Search and another surfaces, particularly for time-sensitive or caller content.” It adds, “we never justify serving as that is resolute entirely by ranking algorithms.”

How It Compares With Crawling And The Indexing API

Any location can ask Google to crawl its pages, according to Google’s recrawl documentation. Owners and complete users of a Search Console asset can petition crawls of idiosyncratic URLs in the URL Inspection tool, inside a quota, and sitemaps shield larger sets of URLs. Google says crawling “can obtain anyplace from a few days to a few weeks,” and that a petition “does not justify that inclusion in hunt results volition happen immediately or equal at all.”

The Indexing API is narrower. It covers pages alongside JobPosting, or BroadcastEvent embedded in a VideoObject, and lets their owners inform Google whenever a leaf is added, updated or removed. This “allows Google to agenda pages for a caller crawl.” There is a “default 200 quota” accessible for API onboarding and submission testing; however, the API “requires additional endorsement for use and asset provisioning.” In a September post I noted multiple job boards had gone months without a answer to their endorsement requests.

Recrawls, sitemaps, the Indexing API provision the URL for crawling. Platforms present what Google calls “content and communication signals” through the UGC program, but this does not substitute community URLs. The criteria motionless necessitate that the satisfied be accessible via Googlebot.

The Indexing API has a default quota for onboarding and testing, alongside additional endorsement required for additional usage. The UGC program requires Google to endorse applicants. Neither guarantees that submitted satisfied volition appear in Search. Of the three options being compared here, recrawl requests and sitemaps do not contain a petition for approval. The program’s “within minutes” wording covers sending, not indexing.

Google Has Partnered With Social Platforms Before

On Dec. 7, 2009, Google launched real-time hunt results that showed “live updates from group on famous sites akin Twitter and FriendFeed.” The article named Facebook, MySpace, FriendFeed, Jaiku and Identi.ca as new partners, alongside alongside Twitter, which Google stated it had announced “a few weeks ago.”

In July 2011, Google stated “that accord expired on July 2,” as Datamation reported at the time. In May 2015, Google said it was bringing Tweets to Search on mobile, and it added desktop that August.

In February 2024, Google stated that it “now has access” to Reddit’s Data API. According to Google, this Data API delivers “real-time, structured, distinctive content.” Google noted that accessing the Data API would provision “enhanced signals that volition assistance us improved comprehend Reddit satisfied and display, train on, and alternatively use it.” Furthermore, Google states that this alliance does not power its use of publically accessible (crawl-able) satisfied from the Internet for indexing, training or displaying specified satisfied in Google products.

Those partnerships mentioned complete are the ones Google announced alongside named entities. However, those before posts did not provision adequate specialized data so as to authorize difference alongside the new pipeline. While the 2024 clarification notes “access to Reddit’s Data API,” the Oct. 8 posting describes approved platforms providing data through a Google ingestion pipeline. There was no citation to the Reddit alliance in the program leaf or the form.

As noted previously, among the uses listed in the 2024 article is training. However, neither the program posting nor the posted use form referenced “training.”

There is a apparent difference between what has been published concerning the new program and the before partnerships. Google has made accessible to the broad community the two the eligibility criteria for involvement in the new program and an online application. As earlier discussed, Google provided acknowledgment of idiosyncratic social phase partners through previous announcements.

What Forum Markup Already Carries

Google’s assistance for the markup the program names predates it by nearly three years. In November 2023, Google added conversation forum and overview leaf organized data, which I covered at the time. Google stated the markup plant alongside features “designed to display first-person perspectives from social media platforms, forums, and another communities,” and that using it “does not justify appearance.”

In June 2024, the changelog confirmed that SocialMediaPosting markup is additionally supported, “to additional accurately indicate how the data ingestion for these features work.” In March 2026, Google added forum and Q&A properties “for providing additional clarity on comment thread construction to Google ingestion systems.” The update included labels for AI-generated and bot content, which I covered in March.

Google’s current discussion forum markup page lists five communication types it supports in interactionStatistic: likes or upvotes, dislikes or downvotes, views, comments, and reshares. The program leaf names the identical markup, alongside interactionStatistic, as its example of what applicants should have on their pages.

Neither the forum records nor the program leaf says how Google uses those counts in ranking.

Why This Matters

Google documents item to a distinct intake for trending posts from forums and social platforms, distinct from the crawling it uses for broad web content. Approved platforms dispatch the posts and their communication signals themselves, during recrawl requests, sitemaps and the Indexing API all hand Google a URL to crawl.

Any phase that includes lone nourish items; requires login credentials for article views; or has person generated posts that cannot be connected rear to a community overview does not meet requirements under the current details despite of the size of their platform. The ten example URLs provided are intended to portray the format the phase plans to dispatch to Google.

Whether or not the phase chooses to apply, the specialized SEO can concentration on the markup. Valid on-page organized data is among the requirements, and in 2023 Google provided Search Console reporting and Rich Results Test support for the two forum and overview markup. Those reports display errors, warnings, and valid items connected to marked-up forum and overview pages.

If a site’s chief origin of satisfied is not user-generated content, afterward it fails the archetypal qualification. However, these types of sites may motionless be competing against User Generated Content (Forum/Social) on the identical searches, but Google did not name any participants. If you discover your searches yielding additional forum results, currently, there is no way inside either the program leaf or form to decide whether the program is liable for those risen results.

What Google Hasn’t Said

As of Oct. 15, none of the program page, form, or changelog admission (Oct. 8) names a participating platform. None of these items established a numerical value for “a important person base” notwithstanding this being among the values requested via the Form to be reported as Monthly Active Users.

The program leaf references “various features on Search,” during the form says “Google Search and another surfaces.” They don’t acknowledge a feature. The 72-hour oppose opening isn’t explained either.

The three listed documents additionally do not signify how Google utilizes the communication signals/engagement counts provided by participants; nor do they define if participants volition obtain any reporting.

Looking Ahead

Applications are open through the form connected from the program page. Google says it volition get rear to applicants “within 6-8 weeks alongside a reply on the position of your application.” Selected platforms afterward get “detailed specialized integration records and onboarding instructions,” alongside no stated agenda for that step.

Google’s Search Central changelog records the Oct. 8 addition. It’s a location to observe for forthcoming changes to the documentation.

More resources:


Featured Image: Tetiana Yurchenko/Shutterstock. Source: Google.

Category SEO
Other Article Search Engine Journal
↑
Close Right Ads
Close Left Ads