Preparing for the Page Experience update we needed to connect Screaming Frog to PageSpeed Insights via API. Should not be an issue at all. Turned out to be very much an issue.
If you are reading this post, I will assume you are facing the same issue. If not, that's your chance to find the answer before you get this issue. But seriously, if you do SEO, own a website or work on one, it's really THE time to prepare for the Page Experience update.
Contents:
What is Google Page Experience Update?
First things first, before going deeper into the issue and the solution, I want to make sure you understand what the Page Experience update is and why we need to pull PageSpeed Insights data at scale.
Google Page Experience update - is an update to Google's ranking algorithm designed to make sure users have great experience on pages that are displayed on top of Google Search Results.
As of August, less than it is estimated that less than 15% of websites were optimized to pass the Core Web Vitals test.
Does Page Speed Play a Role?
Yes, Page Speed is what it's all about.
What is Core Web Vitals Update? - it's an update that will add three more ranking factors to the existing UX signals for Google Search. Namely, Loading, Interactivity, and Visual Stability. In other words, how soon after clicking on Google Search Results a User will be able to use the page.
If you are running a small local website, you can check page load speed fast and free by using this tool.
How to Measure Page Speed on Your Website at Scale?
Now, if you run an eCommerce website or a fairly large B2B or B2C website, you can still get a general idea for the speed of your website from that same test I mentioned above.
However, if you need to check things at scale there are two ways to go:
Google Analytics - to get Page Load Speed data in Google Analytics - go to Behaviour - Site Speed - Overview to see page load speed for your website, as shown below, see more here.
Now if you are really looking to get the Site Speed in the works, chances are by now the recommendation would have come from your SEO or CRO team, that's where the Screaming Frog API integration will come in handy.
Before Connecting Screaming Frog via API
Before you can connect via API, you will need a paid subscription to the Screaming Frog, it runs at $200 per year.
The Problem Connecting to PageSpeed Insights via API Key
It should be easy, as Screaming Frog instructed, all you need to do is simply go to the Google Cloud Platform, create an API key and paste it into Screaming Frog.
And that's what I did, but it did not work.
When I pasted the API key in Screaming Frog, it returned an error, "Failed to Connect", "API Key Invalid".
The problem is, Screaming Frog is providing the wrong link to the wrong website.
Solving the "API Key Invalid" and "Failed to Connect" Error
Solving solving the issue was fairly easy. The correct page to get the API key for the PageSpeed Insight tool is located over on this page.
Click on the link above and scroll down to the "Get a Key" button.
Click on the button and create a project (or click "My new project", that's what I did).
And voila, your page speed insights API key is ready to go into the Screaming Frog for the next crawl.
Your Turn
Try this method and easily link your Screaming Frog to the PageSpeed Insights via API. Hope your website will be ready for the Page Experience update in time! Let me know if it worked for you in the comments below!
Sasha Matviienko is an Analytics & SEO Consultant in Analytics & Digital Marketing consultant, founder of growth360. Sasha has 10 years of Digital Marketing experience from Paid Search, SEO and Display to advanced Data Analytics and Conversion Optimization experience working with clients of all shapes and sizes from small local businesses to Fortune 100 brands.