How this works
Page speed and server response get confused constantly, so they are worth separating.
Page speed scores measure how quickly the finished page appears to a human: images, fonts, layout, scripts, all of it. What matters to a crawler is narrower: how long your server takes to start answering at all, before anything is drawn. It's called time to first byte.
A site can score well on page speed and still answer slowly, and the reverse. Optimising images won't help here.
Why it matters
A visitor who really wants your opening hours will sit through five seconds of loading. A crawler has millions of pages to get through and no particular attachment to yours.
Google documents this plainly: when a site responds slowly or returns errors, its crawl rate drops. Assistants' crawlers behave the same way, for the same practical reason.
Slow server response doesn't produce an error you'd notice. Nothing breaks. Pages just get visited less often, new pages take longer to be discovered, and updates take longer to be picked up. The site looks fine from the inside the whole time.
It also tends to get worse quietly. Plugins accumulate, the database grows, traffic rises, and the hosting plan stays the same as the day you signed up.
How to fix
Most of this is your hosting, not your content.
1. Turn on page caching. This is the single biggest lever for most small business sites. With caching, your server hands over a prepared copy of the page instead of rebuilding it from scratch for every request. Make sure it applies to visitors who aren't logged in, which includes every crawler.
2. Put a CDN in front of the site. A CDN serves your pages from a location near whoever asked, and absorbs a lot of repeat requests before they reach your server at all.
3. Look at what's running on every page load. Heavy plugins, live-updating widgets, external scripts that have to answer before the page can be built. On most sites a handful of extras account for most of the delay.
4. Consider the hosting plan itself. The cheapest shared hosting tiers share one machine between many sites. If caching and a CDN don't move the number, this is usually why.
5. Measure properly. Ask your host for your time to first byte, or use any free tool that reports it separately from page speed. You want it comfortably under a second; several seconds is where crawlers start backing off.
Where to change this on your platform
WordPress
this is where it matters most, because WordPress rebuilds pages on request by default. A caching plugin or caching built into your host is the fix.
Shopify, Wix and Squarespace
these are hosted platforms running their own infrastructure and caching. Server response is largely out of your hands, and usually fine. If a page is slow on one of these, the cause is normally heavy third-party apps or embedded widgets rather than the platform.
If you've done all this and it's still slow
Then the delay is in something your pages do on every load: a database query, an external API call, a licence check. That's a conversation with your developer or host rather than a setting, and it's worth having: this one gets worse over time, never better.
