Reddit has announced that it’s changing its Robots Exclusion Protocol, also known as its robots.txt file. This might seem like a boring update but in the battle between AI companies who need content for their Large Language Models and the companies who actually own that content., this is another line in the sand. By changing its robots.txt file, and also by continuing to rate limit and block unknown bots and crawlers, Reddit seems to be working to AI companies from using their content without proper credit or compensation.
Subscribe for more on YouTube: https://tcrn.ch/youtube
Follow TechCrunch on Instagram: http://tcrn.ch/instagram
TikTok: https://tcrn.ch/tiktok
X: https://tcrn.ch/x
Facebook: https://tcrn.ch/facebook
Read more: https://techcrunch.com/
