How I Found Internal Linking Opportunities With Vector Embeddings

I felt overwhelmed erstwhile I first publication Mike King’s article connected vector embeddings. The concepts seemed complex, and implementing them for SEO was intimidating. But pinch Screaming Frog’s caller features and Gus Pelogia’s fantabulous guide, I saw the imaginable to amended soul nexus building utilizing this method.

Based connected the 2 resources above, I decided to create a detailed, step-by-step guideline to make this process much approachable, moreover for group unfamiliar pinch Python aliases vector embeddings.

In this article, I’ll locomotion you done really I utilized vector embeddings to place soul linking opportunities astatine standard truthful you tin confidently use these techniques to your SEO strategy.

To transportation retired this process, I utilized the following:

• Screaming Frog
• OpenAI API Key
• Google Sheets aliases Excel

By the end, I had a broad spreadsheet that included:

• Every important URL from my tract listed successful file A (target URL)
• The URL of each page that links to the target URL (excluding navigation)
• URLs to the apical 5 astir intimately related pages based connected cosine similarity
• Opportunities wherever 1 aliases much of those 5 URLs are not linking to the target URL

Example Spreadsheet

This is the illustration I utilized successful the screenshots below, and it will look thing for illustration this:

For convenience, I person saved the Screaming Frog civilization configuration profile. Download it here. In Screaming Frog, unfastened a civilization configuration record by using: Go to File > Open and prime the .seospider record you want to load. Here is the explainer video.

After loading the configuration file, I opened the API tab, selected OpenAI, and pasted successful my API key.

Note: The API Access characteristic supra pulls embeddings into the API report. However, the book beneath requires the overmuch smaller record generated by the Custom JavaScript function.

Next, I opened Screaming Frog and followed these steps:

• Navigated to Configuration > Custom > Custom JavaScript.

• Clicked “Add from Library” and selected “(ChatGPT) Extract embeddings from page content.” This allowed Screaming Frog to extract the information needed for the soul nexus audit.

• I edited the civilization JavaScript codification to see my OpenAI API Key. Then, I pasted the API Key I generated successful Step 1 into the due conception of the code. NOTE: You will person to do this moreover erstwhile utilizing the civilization configuration floor plan linked above.

Export All Internal Links from Screaming Frog

• I saved the crawl erstwhile it was completed.
• I exported the “All Inlinks” information from Screaming Frog. This record contains each soul nexus connected the tract and tin beryllium rather large. For example, my file, all_inlinks.csv, was astir 52 MB and represented 1,428 URLs.

Export vector embeddings from Screaming Frog

• I exported the Custom Javascript results wherever the vector embeddings are located.
   

I opened this convenient Python script connected Google Collab, which was created by Britney Muller, and made a transcript to use.

• I pressed the “Play button”
• Scrolled down to “Choose Files"
• Uploaded my first CSV record (all_inlinks.csv) and fto it process.
• Uploaded my 2nd record (custom_javascript_all.csv) erstwhile prompted.
• Accepted the Save File messages erstwhile they were done processing. You should get an XLSX type and a CSV version.

Note: The original type of this guideline was astir 3 times arsenic long, comprised mostly of formatting and information cleanup. Britney’s book automates ALL of this, redeeming you respective hours of tedious work.

Troubleshooting errors

Sometimes, an rumor pops up that keeps the book from working. But don’t worry, clicking “Explain Error” will typically guideline you to the fix.

The mentation of the correction helped maine fig retired that I needed to unfastened the CSV record and look for irregularities successful the Embeddings column. It turned retired location was a blank cell.

Other examples of what mightiness origin errors during this shape are: 

• Extra columns
• The incorrect record name
• The incorrect file names

I opened a caller Google Sheet and imported the XLSX file. The book besides outputs a CSV, but I for illustration the XLSX type because it preserves conditional formatting.

If you’re moving from Excel aliases different spreadsheet programme connected your computer, simply unfastened the XLSX file.

Validate the data

I double-checked a fewer entries manually to guarantee everything was accurate. Now, I person a complete database that shows each target URL successful file A, the apical 5 related URLs, and whether those URLs are linking backmost to the target URL. 

My last spreadsheet looked for illustration this, pinch “Exists” aliases “Not Found” indicating whether each related URL was linking backmost to the target URL: