The prices on ebay are going UP

--- "Dwight K. Elvey" <dwightk.elvey(a)amd.com> wrote: It is a convention amongst robot/spider writers to pick up and respect the contents of a "robots.txt" file. This file contains optional instructions for respecful software to not index or cache listed web pages and/or directories. I have one on my Antarctic web site to keep them from indexing things like my web stats directories (who needs to cache months-old webified access reports, after all). I learned about robots.txt when I started seeing a lot of non-existent page "errors" in the Wusage logs. Did a bit of digging and discovered that it's a good idea to have even an empty robots.txt file on a web page that is likely to see a crawler go by. I think as much as 30% of the traffic on my site is automated. -ethan __________________________________________________ Do You Yahoo!? Yahoo! - Official partner of 2002 FIFA World Cup http://fifaworldcup.yahoo.com