http://csnet.ca/1/cr//robots.txt