chewie.co.uk
MSNbot 2.0b is ignoring robots.txt and No Index meta tags
Back in December 2008 Microsoft Live Search announced that it would be releasing its new spider/crawler into the wild to crawl all those lovely websites out there, now 4 months on it seems that the…