My website seems to work really well, but I notice that certain people when they login generate errors which appear in my website host's log. These errors are quite regular and the two most common ones are:
[Wed Aug 25 16:46:26 2010] [error] [client {client URL}] File does not exist: /home/myusername/public_html/404.shtml
[Wed Aug 25 16:46:26 2010] [error] [client {client URL}] File does not exist: /home/myusername/public_html/robots.txt
I've deleted the URL which caused the error in the above so as not to incriminate anyone and substituted {client URL} in each line. I've also changed the name of the directory which causes it to "myusername". I'm not sure if it compromises security if I left the correct directory in the above or not so I have replaced it as a precaution. If I login to my website, I don't create the error. Does anyone know the reason for this and what does it signify? Generally speaking, these two errors appear together, in fact I think they always do.
My website URL is
http://sidlawweather.co.uk.
If anyone can suggest the reason for this, I'll be very grateful.
Your first clue is the lack of a robots.txt file. The Carterlake template doesn't provide for one. Robots.txt is a standards file that spiders (search engine bots) use to get a map of your website. Spiders first request the robots.txt file...if it can't find one, it starts traversing the website willy-nilly usually.
The 404.shtml file is likely another, or, your htaccess file has a rule pointing to 404.shtml as your default 404 (not found) page and it doesn't exist. Nothing really to be alarmed with. IMO.
