id summary reporter owner description type status priority milestone component version severity resolution keywords cc mlocked 11958 PATCH: Robots.txt for mythweb skd5aner Rob Smith "Over the years, several people accidentally leave mythweb exposed to the internet. Even though strongly discouraged for a multitude of reasons, one of the biggest issues that occurs in this situation is that various search engines can cause havoc by crawling the site and hitting links that modify the system. There are dozens of reports of bots like googlebot causing modifications to all recording rules, etc. At the very least, including a robots.txt file will keep the majority of valid bots from causing havoc when crawling the site. It will not prevent bots that ignore robots.txt from causing issues. This is not a magic bullet, but it's certainly a simple way to add a small layer of protection that could otherwise cause someone to have a real bad day. The attached patch simply creates a robots.txt file at the root of the mythweb directory, and disallows all user-agents from crawling the site. This patch can apply to master and also to 0.27 and 0.26." Patch - Feature closed minor unknown Plugin - MythWeb 0.27-fixes medium Won't Fix 0