v1.8: adds VelenPublicWebCrawler

Thank you to Brian at Skewray Research for calling this one out.

Full Changelog: v1.7...v1.8

v1.7: adds Meta-ExternalAgent

Many thanks to @nisbet-hubbard for raising this as well.

Full Changelog: v1.6...v1.7

v1.6: adds OAI-SearchBot to robots.txt

chore: add OAI-SearchBot

v1.5: drops AdsBot-Google; adds GoogleOther-Image/Video

Full Changelog: v1.4...v1.5

v1.4: add ImagesiftBot, remove peer39

What's Changed

• Adds ImagesiftBot (thanks @corbindavenport!)
• Removes peer39 bots as they are not used to train/related to AI
• Add reference by @glyn in #18

Full Changelog: v1.3...v1.4

chore: updates to bots table, readme

What's Changed

• More bot information and improved README by @corbindavenport in #17

New Contributors

• @corbindavenport made their first contribution in #17

Full Changelog: v1.2...v1.3

chore: tidy up bots

What's Changed

• Narrows the scope of bots blocked to strictly include those relating to AI
  • This sees some bots removed that should likely still be blocked, but for different reasons.
• Remove potential false positives by @nisbet-hubbard in #16

v1.1: updates bots list to include Applebot-Extended in favor of Applebot

• Updates bot listings to include Applebot-Extended in favor of Applebot at the former is for AI training, the latter is used for search.

v1: First proper release

Let's consider this the first proper release, with additional updates to be formally marked with a release.

Full Changelog: https://github.com/ai-robots-txt/ai.robots.txt/commits/v1