work2code.com
Mistakes to avoid while creating robots.txt – Work2Code
Most of the programmers and testers don't really know about how robots.txt file can pose security risks and how robots.txt is one of the source for OSINT for maclicious hackers