Keywords matter. When you participate a hunt successful Google aliases a punctual successful Gemini, you expect the reply to bespeak the question. Consider an utmost example. If you entered a hunt for “best suvs of 2026,” would you expect to spot the consequence below?
How overmuch do keywords matter?
Google has had immoderate expertise to understand synonyms for years now. Consider this hunt for “cell phone” successful 2012 (via the Internet Archive):
Three ways to measurement similarity
To amended understand Google’s existent capabilities, we compared the queries to integrated consequence titles (as displayed by Google, not <title> tags) utilizing 3 metrics: (1) exact-match*, (2) partial lucifer pinch Jaccard similarity, and (3) semantic lucifer pinch cosine similarity.
1. Exact-match*
Exact-match is beautiful self-explanatory, but we chose to beryllium a spot forgiving, normalizing lawsuit and punctuation, removing plurals, and allowing immoderate title that contained the afloat query.
2. Jaccard similarity
To analyse partial matches, we utilized Jaccard similarity, which measures the number of shared elements (in this case, words) crossed 2 sets vs. the unsocial elements of some sets. Put simply, it’s the proportionality of shared words crossed the 2 strings to the total, unsocial words. This is measured connected a 0.0-1.0 scale.
3. Cosine similarity
Finally, we calculated vector embeddings and cosine similarity betwixt the 2 strings. This captures semantic relationships – successful a word, “meaning.” Specifically, we utilized 768-dimensional Nomic embeddings. Cosine similarity besides measures similarity connected a 0.0-1.0 scale. Let’s look astatine the stats and immoderate examples.
Exact-match information and examples
Even pinch our much forgiving exact-match*, only 43 show titles (0.49%) contained the afloat query. Here’s an illustration that only differs by a hyphen (-):
… and here’s an illustration wherever the title includes the query and a spot more:
Flipping that first statistic, 99.51% of show titles did not incorporate the afloat query. Given the information group of long-tail queries, I don’t deliberation this will daze astir of you, but it does exemplify really overmuch SEO has evolved since the keyword-stuffing days.
Partial lucifer / Jaccard similarity
Here’s wherever things get much interesting. The mean Jaccard similarity for the 8,703 show titles was 0.23 (note that Jaccard similarity is beautiful unforgiving). To put that successful context, here’s what a 0.23 people really looks like:
I’ve highlighted the matching words – arsenic you tin see, the mean worth represents a beautiful constricted overlap. Here’s a higher Jaccard people (0.75) that isn’t an exact-match:
Putting speech word-order (which Jaccard ignores completely), this is simply a important overlap. Note that a existent exact-match would besides person a Jaccard similarity of 1.0.
Semantic lucifer / cosine similarity
The mean worth of cosine similarity crossed the information group was 0.76 – cosine similarity is overmuch much forgiving than Jaccard similarity. Here’s an illustration of a 0.76:
This one’s absorbing because the show title is simply a overmuch much structured, SEO-style title. While it’s circumstantial to Minecraft and possibly not rather what the searcher intended, we tin surely spot that there’s semantic overlap. Let’s look astatine a high-similarity example:
This consequence has a cosine similarity of 0.90, and you tin spot that the vector embeddings are astir apt equating concepts for illustration “US” and “America” and ignoring immoderate of the insignificant differences (like “8 of the”) that don’t effect the relevance to the query.
Bonus: precocious cosine, debased Jaccard
A word-for-word exact-match is going to beryllium ones crossed the board, and precocious Jaccard similarity almost ever intends precocious cosine similarity. What astir cases wherever the connection overlap is debased but the semantic overlap is high?
Google isn’t conscionable recognizing synonyms and semantic similarity present – they’re really highlighting imaginable answers. While this highlighting is simply a furniture that happens aft results are retrieved and ranked, it’s worthy exploring successful your ain hunt results to understand the benignant of accusation that Google is rewarding.
