There are lots of reasons why you may want to do this - to protect your admin area, hide some nasty scripts or to save on bandwidth (yeah, i’ve heard it done…). Of course, all people have to do is look at your robots.txt and find out a load of directories that you don’t want found… (Read more to see full post)

(more…)