Google Search Internal Issues With Square Brackets [] Punctuations In URLs

Search Engine Roundtable by 4 min read 80x views
Google Search Internal Issues With Square Brackets [] Punctuations In URLs

Share Post

Google Brackets

Google Search had several issues handling quadrate brackets symbols marks (i.e. [ ]) inside URLs. John Mueller from Google responded to erstwhile Googler, Vanessa Fox, saying that he is "seeing several of the issues internally that came up alongside these kinds of characters in the URL."

He afterward added in circular brackets saying, "(though they're 10+ years ago)." He additionally said, "it power be that another crawlers / indexers / fetchers aren't as exact in this logic, so it power be value double-checking those you attention about."

Vanessa Fox's first inquiry was on Bluesky, she wrote:

I posted this inquiry in the YT video comments: For URLs alongside characters that can be encoded (brackets, affirmative signs, spaces), does Google treat the encoded type as a distinct URL? (example.com/category?filter[]=123 vs. example.com/category?filter%5B%5D=123 used in links vs. canonical values).

John Mueller from Google had this lengthy reply:

Well, TIL. I would have assumed they're equivalent, but they're NOT. This was a fun side-track. "?filter[]=123" ≠ "?filter%5B%5D=123" since in RFC 3986, Section 2.2 the characters brackets ([, ]) and affirmative signs (+) are defined as part of the reserved set.

"URIs that differ in the replacement of a reserved character alongside its corresponding percent-encoded octet are not equivalent. Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, volition alter how the URI is interpreted by most applications. Thus, characters in the reserved set are protected from normalization..." — IETF STD 66 / RFC 3986, Section 2.2

JavaScript can equal inspect for this: new URL('https://example.com/category?filter[]=123').href === new URL('https://example.com/category?filter%5B%5D=123').href // Output: false

That said, *maybe* your server makes them "soft-equivalent" by serving the identical content. We'd crawl the two versions, if we see the identical satisfied (maybe equal checksum level), we'd de-dupe them and choice one.

So in short, choice one, and use it consistently (links, canonical, sitemaps, organized data). The identical is alongside " " (space), + (plus), and "%20", but alongside the reward of this being distinct depending on which URL component it's in (path vs query string).

I knew concerning the spaces part, but the brackets was unexpected, and I likely gave group the incorrect answer on this in the past. Woops. BTW for hex-encoding, the case doesn't matter, so "%5B" === "%5b".

As if that wasn't enough; brackets: not lone are they not equivalent, the URL alongside ?filter[]=123 is technically invalid (according to RFC 3986) since [ ] may lone appear in the hostname to cover raw IPv6 addresses 🤯. Browsers mostly hex-encode them whenever you act them, and servers are akin cats.

Vanessa Fox afterward followed up and wrote:

Ack - I saw that! But this particular location is using the invalid form of [] throughout and it seems to be indexing fine, and it would be a ton of activity to alter it so they are leaving it for now... fingers crossed?

John Mueller replied:

I'd anticipate it to continue to activity - the web is messy and not all of it is "technically correct" - but I wouldn't set up a new location akin this, and I'd pay additional notice to all the inner linking (including metadata akin rel=canonical, organized data, etc) to create it consistent.

Also, seeing several of the issues internally that came up alongside these kinds of characters in the URL (though they're 10+ years ago), it power be that another crawlers / indexers / fetchers aren't as exact in this logic, so it power be value double-checking those you attention about.

So Google likely can grip it now, but it's improved to be safe?

Forum conversation at Bluesky.

Update: John added, "No issues - the net standards for URLs are fair extremely specific."

Other Article Search Engine Roundtable
↑
Close Right Ads
Close Left Ads