A Traefik Middleware Plugin that helps you manage and enforce your robots.txt - holysoles/bot-wrangler-traefik-plugin
Detailed discussions on excessive crawling targeting code.forgejo.org: - [February 2025](https://codeberg.org/forgejo/discussions/issues/297) - [April 2025](https://codeberg.org/forgejo/discussions/issues/331) --- Codeberg and the Forgejo infrastructure (which are entirely separate) were both ...
There's a war going on on the Internet. AI companies with billions to burn are hard at work destroying the websites of libraries, archives, ...
tl;dr: Here’s a how-to for adding some “AI”-poison to your static site that’s hosted on Codeberg Pages (or GitHub Pages). I’d appreciate some feedback on if this is useful/how it could be improved. If you’re running any type of website in 2025, you’ll likely be suffering from the impact of...
Attackers explain how an anti-spam defense became an AI weapon.
AI-backed personal assistants are turning us children again and killing our youngsters own childhood, we need to act now
A quick guide on combining an anti-AI tarpit with automatic blocking
People who make useful content and services intended for interactive human use available for free have written at length about the ongoing issues with AI crawlers scouring the web in an insatiable search for new training data. Some people are coming up with interesting technical solutions to the problems posed by AI crawlers, but I have ultimately opted for a much simpler solution: a paywall. Kullish is a project that I operated free of charge for all users from 2020 until January 2025. Kullish is a
More 409 than 402, personally.
A new avenue for identifying greedy, badly-behaved bots
There are many challenges involved with running a web site like LWN. Some of them, such as fin [...]