Raw quadrate brackets in a URL and their percent-encoded version, %5B and %5D, are distinct URLs, Google Search Advocate John Mueller said on Bluesky Monday. He wrote that he would have assumed they were equal and that he “probably gave group the incorrect answer on this in the past.”
Google volition crawl the two versions, he said; if it finds they merge identical data it volition eliminate replication and choice one version; he recommends you choose one format to utilize throughout all links, canonical tags, sitemaps, and your organized data. The rules for copy formats use likewise to spaces, affirmative signs, and “%20,” he said.
Why Encoded Brackets Are A Different URL
Mueller traced the difference to RFC 3986, the net norm for URI syntax, which puts brackets and affirmative signs in its reserved set of characters. Under Section 2.2, URIs that toggle a reserved character for its percent-encoded type “are not equivalent.”
He was responding to Vanessa Fox, an ex-Google employee who developed Google’s Webmaster Central; she had asked in a Bluesky reply (after asking the inquiry in the comments of the archetypal Non-Canonical Cabinet episode) if Google treated the encoded form of a URL as a distinct URL. She provided examples by using the two ?filter[]=123 and ?filter%5B%5D=123 (one in links; the another whenever identifying canonical versions). The JavaScript inspect he made afterward compared these two URLs and returned false.
He pointed out that the use of raw brackets in a query cord is additionally technically invalid according to RFC 3986; this is since RFC 3986 restricts the use of brackets to the hostname, for wrapping raw IPv6 addresses.
He stated that letter cases inside encoding do not create a difference, hence “%5B” and “%5b” are equivalent; they adhere to the rules set onward by RFC 3986 concerning hex characters.
What Google Does With Both Versions
Mueller described what Google does whenever a server returns the identical leaf for the raw and encoded forms:
That said, *maybe* your server makes them “soft-equivalent” by serving the identical content. We’d crawl the two versions, if we see the identical satisfied (maybe equal checksum level), we’d de-dupe them and choice one.
He additionally stated that a browser may encode brackets previous to sending an encoded type of what is typed by the person into the URL bar. However, if a bracketed URL is requested from JavaScript (or another code) afterward the brackets power be sent as is. He recommended encoding brackets whenever they need to be used inside a URL: “always encode it, never display [].”
What Google’s Documentation Already Says
Google’s URL construction documentation already calls for encoding characters akin brackets. It says Google Search supports URLs as defined by IETF STD 66, the norm that contains RFC 3986, and that “Characters defined by the norm as reserved must be percent encoded.”
I covered the archetypal Non-Canonical Cabinet episode final week. That event dealt alongside emoji and non-English characters in URLs, which browsers percent-encode in a URL path. Brackets are a distinct case, since the norm reserves them as delimiters.
Sites Already Using Raw Brackets
Vanessa Fox responded that she was thinking concerning a website which uses an invalid bracket notation throughout, does appear to be indexing fair fine and has decided to depart it solitary since it would be too much difficulty to change it into valid brackets.
Mueller stated he wouldn’t anticipate problems from using this construction but “wouldn’t set up a new location akin this.” Mueller additionally indicated that he would pay additional notice to all inner links (including metadata connected to rel=canonical and organized data) so they are uniform.
Additionally Mueller stated that according to problems they have encountered at Google alongside regards to how these characters functioned complete 10+ years ago, another crawl / indexer / fetch programs may not be “as exact in this logic” whenever handling them.
Why This Matters
Google volition choose one type of the URL, Mueller said, if there are multiple versions accessible alongside the identical content. If all your URLs use the identical format (i.e. either raw or encoded) afterward this choice is additional apt to go your way. The finest resolution is to encode the brackets in the URL as requested by Google’s own documentation. Mueller expects an existing location using raw brackets to motionless continue to work. Your project would hence appear to be to create certain that your inner links and metadata item to the identical type of the URL.
Looking Ahead
Mueller suggested checking the crawlers and fetchers that matter to you for how they agreement alongside bracketed URLs. Mueller didn’t name specifically what crawlers or fetchers could have a issue but he stated that several crawlers/fetchers power not prosecute Google’s exactness whenever dealing alongside bracketed URLs.
Google had not yet posted their second installment of the Google Video Series titled “Non-Canonical Cabinet” concerning robots.txt as of October 20th.
Featured Image: Popartic/Shutterstock
