Is Google going to crawl less?

Is Google going to crawl less?

One part of yesterday’s Google “Search Off the Record” podcast that seems to have grabbed many analysts’ attention is a hint that Google may reduce the amount of webpage crawling it does.

The immediate reaction to this might be that any such move would lessen the importance of search engine optimisation (SEO), but on the contrary, the indication seems to be that any scaling back of crawling would only be to refine the process and concentrate on regularly changing pages.

The podcast features Google’s John Mueller, Gary Ilyes and Martin Splitt, who begin to discuss the search company’s indexing practices at the 2:40 mark. Ilyes talks about improving Google’s sustainability and environmental practices, and how chopping some of the “low-hanging fruits” like unnecessary crawling could help with this.

Ilyes mentions how Google will return to a page periodically to crawl it, but notes that this process could be made more efficient. He gives the example that a website’s “About” page is unlikely to change regularly, so constantly “refresh crawling” that page may not be a good use of resources. On the other hand, some home pages (Ilyes gives the example of big American news sites like Wall Street Journal and CNN) are being updated by the minute, so it would be a question of Google identifying those pages that require regular crawling.

Writing for Search Engine Journal, Matt Southern notes that there is not necessarily any connection between crawling frequency and search engine rankings, and that it will be important to learn which pages need updating regularly.

With Ilyes mentioning the home pages of major news sites, one small step businesses could take to keep their home pages fresh is to have their recent blog posts appear on them. They could also add testimonials to their home page as and when they get them, along with any recent videos they have filmed.

While changing content for the sake of change should not be advised, a website should always be an evolving tool, and provided it’s of high quality, regular, fresh content is a signal to Google of a vibrant site. Speak to Engage Web to learn what we can do to help your site’s rankings and boost traffic and enquiries.

Content Team Leader at Engage Web
John works for Engage Web as a Content Team Leader and regularly contributes to the website and programmes of his beloved Chester F.C.
John Murray

Get in touch

    Please confirm we can contact you


    Book a consultation with Engage Web