Which AI Bots Should You Allow on Your Website?
Deciding which AI bots to allow used to be one checkbox. It is now twenty-two named user agents across a dozen operators, several of which do the opposite of what their name suggests, and two of which never touch your server at all.
16th Jul 2026