Google’s Gary Illyes said on LinkedIn that he’d person to cheque whether a tract tin safely widen an unavailable_after date. His first read, which he called a gut feeling, was that pushing the day guardant is fine, but he near unfastened whether the tag tin beryllium trusted for pages that expire and past get renewed.
What Was Asked
SEO advisor Javier Lorente Murillo reached retired to Google’s Search squad pinch a mobility astir a classifieds tract that has a dependable halfway of astir 5,000 indexed URLs and adds astir 10,000 caller listings each month. Most of these listings enactment unrecorded for 24 to 72 hours.
He planned to usage unavailable_after connected short-lived pages to debar 404 errors and negociate crawl budget. Because users tin renew ads, his strategy would dynamically update the expiry date. He asked whether Googlebot would commencement to disregard a day that keeps changing pinch each renewal.
What Illyes Said
Illyes opened by saying he didn’t know:
“that’s a awesome question… I person nary idea, I person to check.”
He added:
“my gut emotion is that it’s good to push guardant the unavailable_after day BUT you request to support successful mind that we’ll request to crawl the page again to “see” the caller date. it doesn’t person implications connected thing but scale selection, wherever it acts arsenic a “you tin driblet this now” signal.”
In a later response to different remark connected the aforesaid post, he said Google would fetch the page alternatively of checking it cheaply:
“we seldom spot HEAD requests truthful we’d still download the content.”
What Google’s Documentation Says
Google’s robots meta tag documentation explains ‘unavailable_after’ arsenic a norm instructing Google not to show a page successful hunt results aft a specified day and time. The aforesaid page says Googlebot will crawl that URL considerably little often aft that date.
The page besides talks astir the recrawl constituent Illyes mentioned, highlighting that Robots meta tags and X-Robots-Tag headers are detected erstwhile a URL is crawled. This intends that a changed day will only look connected the adjacent fetch.
That portion of his reply echoes what Google already states. The declare that the norm touches thing beyond scale action is his own, and it doesn’t look successful the docs.
Why This Matters
Illyes didn’t opportunity what happens if the past day Google saw passes earlier it gets backmost to the page. That is the lawsuit he said he would person to check.
With the renewal lawsuit unanswered, there’s thing from Google to scheme around. What decides the result is really often Google crawls the page, and a tract proprietor can’t power that.
Looking Ahead
As of publication, Google’s documentation doesn’t screen an expiry day that changes connected each renewal.
The first telephone is what you’re asking the unavailable_after tag to do. Letting a page driblet retired of hunt results aft a group day is what Google documents it for. Keeping a page live while leaning connected the tag to negociate crawl is the lawsuit that’s still open.
Featured Image: ESB Essentials/Shutterstock
English (US) · 
Indonesian (ID) · 
