On Tue, 12 Oct 2004, David V. Corbin wrote: Absoluetely -- it's even possible for rsync to propagate data loss! (--delete) But the very top of the pile of reasons for data loss is human error, and a simple "recent copy" solves 99% of that. There is no single system that solves all problems. Since complexity often causes humans to make more mistakes, and inherently decreases MTBF, and costs more, simplicity is usually most optimum (sic), barring other EXPLICIT constraints. * At work (university computer lab) I have students with home dirs (many files, various sizes) and video projects (few files, HUGE). Major worry: student foolishness, second worry, other human data loss, tertiary, the usual (hardware failure, etc). Long-term archiving not needed. Heterogenous network (pc, mac, linux). My solution here is two servers, weewee and fluffy, both terabyte RAID5 (serial ATA, 3ware, SuSE 9.0) 2.8GHz, 4GB RAM. Weewee runs SAMBA, mounts homes and video home. UPS with software support. It also runs an rsync/cp -a script that generates rotating hourly snapshots of everything (except video); students can recover files hour by hour. Fluffy takes a daily snapshot of weewee, and rotates that once per day, and maintains the last 10? days of data. There is no off-line backup. Various desktop machines backup into weewee also, get rotated, etc. Weewee and fluffy have a private gigabit ethernet with crossover cable; copying video data can take 10 - 300 minutes via cron. Who cares? Cheap ($3K/server) nearly zero maintenance. Load average rarely exceeds .1. * At home everything exists on my laptop, including wps.com. Major worries: support fancy data-dispersion scripts, human data loss, last, hardware. Rsync "mirroring" between three (soon four) net-wide machines. Paper copies. Occasional CDRs mailed to friends. Archiving back to 1994 (sporadic).

On Tue, 12 Oct 2004, David V. Corbin wrote: Oh and I've made that very mistake many times, which is why I've become personally very paranoid about it! (The propagated backup error is what caused me to lose all my work in 1994.) It's really hard to do backups right. Or, easy, but usually succumbs to "security guard syndrome", you get number & lazy. ...assumes you actually have one! But the context here was a mail archive, so pyramidal data isn't needed for that part, since it's just a grow. I use rsync and cp -a to create a rotating hard-link full-snapshot backups on the university servers; 400GB of homedir junk grows only about 5%. It prevents rm -r * type data loss; if there's a bad sector under the underlying inode's data it's total loss -- but that's what RAID5 and dual-server is for. With my oh-so-clever script system I use now, rsync --delete is very dangerous (I do it only manually), and I've just last week had to drag my entire music collection back down to the home machine (duh). I'm right-now working on making it a lot smarter to punt the whole process if there's a discrepancy. They all require vigilance!