<snip>
 I've started putting high bandwidth items exclusively under Gopher and
 making
 the Gopher proxy extremely strict. This has the nice side effect of
 increasing,
 slowly, the installed base of Gopher clients. There are a few Gopher bots
 out
 there but they're usually single people, easily tracked down, and typically
 apologetic when they realize it went out of control.
 The AI bots do none of that, so I have no compunction about blackholing
 them in
 huge netblocks. If they were willing to work with sites, they wouldn't
 constantly toss out new IPs and user agents like germophobes and wet wipes.
 
that's an interesting approach for sure.
although gopher is probably not a long-term solution, I suppose it does not
matter as sooner or later the WWW will be regional webs, protected by
corps, governments and netizens themselves.  Parts of it will be AI swamps
where seldom one ventures and returns unaffected