19 Jan
2011
19 Jan
'11
3:29 a.m.
On 1/19/2011 2:35 AM, Pontus Pihlgren wrote:
We (a fellow admin and I) have been talking about
mirroring bitsavers
(and perhaps other sources) "any day now". Perhaps setting up a tracker
and create torrents would be a more sensible use of bandwidth!
I see a few issues with it though.
1. On what level should I make the torrent? One per file, one per
subdirectory, one per manufacturer?
- I'm thinking both subdir and manufacturer. It is then easy to
select what files you want from a torrent.
How big is the stuff you are talking about? I would imagine
documentation, which hasn't been converted properly, would probably take
up a bunch of room. If we are talking about binaries and such, then
space probably isn't a big concern.
This really comes back to size for manageability --- you could .rar each
manufacturer, and then distribute a torrent of multiple .rar's.
Don't make the mistake of having multiple torrents for the whole
project. I think that's a mistake. Have one big torrent, with the
individual files being selected/selected within the clients.
When sharing and redistributing this stuff, I much like to download one
huge mama file that contains EVERYTHING. Bandwidth and disk space is
pretty cheap these days. Plus, if this ever gets redistributed into the
wild, you don't want to end up splitting the content. Right? So maybe
they have IBM and DEC (or whatever) but no CPM or whatever. Then people
have to try to find the right torrents --- your collection is no longer
a full collection.
2. How does one handle updates?
- I have no clue ... a whole new torrent for one new/changed/deleted
file in a directory? Is there such a thing as incremental torrents?
Have you seen the TOSECs?
http://en.wikipedia.org/wiki/TOSEC
Not so much for naming per se, but how they distribute them? They
basically have a new torrent released for each update -- with each
update happening perhaps every 6 months or year.
You could just do quarterly updates or something, and release a new
torrent that contains the latest files. It's not too hard just to look
for the latest .torrent out there.
Mame ROMs distribution works the same way. Added a couple more roms to
the collection? bump version from .133 to .136 and release a new torrent.
Oh, and one last thing on this. You can download one torrent "on top"
of an existing directory of a completed torrent. The individual files
get hashed.... meaning
.133 contains ibm.rar dec.rar and cpm.rar
.136 is going to modify cpm.rar but not ibm.rar and dec.rar
As long as the directory structure between torrents is consistent, and
the download directory is the same --- ibm.rar and dec.rar will be
listed as finished with cpm.rar showing inconsistent. You would need to
delete the inconsistent(ie older) rar and then restart the download (so
the hash re-check occurs)
Obviously some of this is torrent-client specific, but I think you could
make this work.
3. Would bitsavers and other agree to be a seed as well?
- A critical mass of users is ofcourse necessary.
You likely need less than you think. I've seen torrents being kept
alive for years with just a handful of seeders. As long as you maintain
a couple "official" seeders, you should be fine.
Once again comes back to size.
Hmm.. perhaps I should just rsync it and use the existing ftp server.
Cheers,
Pontus
Yup.
Keith