Websites using Cloudflare could see their traffic affected following changes set to take place on 15 September.
At the moment, Cloudflare gives website owners the option to block automated AI (artificial intelligence) bots from crawling their site.
From 15 September, this will be replaced by a new set of controls. These will split automated bot traffic into three different categories: Search, Agent and Training.
Search crawl bots are used to discover and index website content so that it can appear in search engine results such as Google and Bing.
Agent bots access websites in real time on behalf of users. For example, an AI assistant such as ChatGPT may visit a website to retrieve information when answering a user’s question.
Training bots, on the other hand, collect website content to train or improve AI models.
With this update, website owners will be able to choose whether to allow or block each type of crawler separately.
The potential SEO (search engine optimisation) issue that website owners face with this change comes from bots that serve more than one purpose.
For example, Cloudflare says that crawlers including Googlebot and BingBot may be classed as both Search and Training crawlers.
This means that, under these new rules, if a website owner chooses to block Training crawlers, this could also result in these multi-purpose bots being blocked.
Put simply, a website owner may decide that they want to prevent their content from being used for AI training, but in doing so, could unintentionally block a Search crawl bot they rely on to help their website appear in search results.
If search engines are unable to crawl pages properly, this could affect how quickly new or updated content is discovered. Over time, this could have an impact on visibility in search results and lead to a decline in organic website traffic.
This change does not mean that websites using Cloudflare will automatically see their rankings drop from 15 September. Businesses using Cloudflare should review their bot settings once these changes are introduced to make sure they are not accidentally blocking crawlers that help their website appear in search results.
Want to make sure your website stays visible in search results? Get in touch with our team at Engage Web today.
- Could Cloudflare’s September update affect SEO? - September 9, 2026
- Google begins changing search result tracking parameters - September 3, 2026
- What are context hints for ChatGPT Ads? - September 1, 2026
