New subject: Tape dumping, Darn! I forgot about partitions!

3 May 2002

...
  From: "Stan Sieler"
&lt;sieler(a)allegro.com&gt;
 To: cdl(a)proxima.ucsd.edu (Carl Lowenstein), classiccmp(a)classiccmp.org,
         classiccmp(a)classiccmp.org
 Date: Fri, 3 May 2002 12:10:42 -0700
 Subject: Re: Tape dumping programs for Unix/Linux...
 Priority: normal
 In-Reply-To: &lt;200205030710.AA16219(a)proxima.ucsd.edu.UCSD.EDU&gt;

 Re:

  Look for a program named "copytape". 
It copies tapes including the
 block structure, file marks, etc.  Unlike dd(1) which only really
 works if all blocks are the same size and there is only one file
 on the tape.  It can copy tape-drive to tape-drive or tape-drive to
 disk file, and puts markers in the disk file to indicate the
 structure of the data on the tape.

 The versions I found do a fair job of copying a tape...but not
 a complete job (particularly for DDS and DLT tapes).

 Why?

 It doesn't know about "setmark".

 On a DDS (DAT) drive, you can have a data record (of variable length,
 not just multiples of 128 or 256 bytes, BTW), an EOF (end-of-file mark),
 a setmark, or an end-of-tape indicator. 
OK.  It _was_ written long before anything as smart as a DAT drive
had been devised.

Why do you stress "data record not just multiples of 128 or 256 bytes"?
Oh, this is qualified by "on a DDS drive".  Certainly 7-track and 9-track
1/2" drives could have fairly arbitrary data records.  I have seen
tapes with records as short as 14 bytes, and as long as 65535 bytes.
Also pre-DLT tapes (DEC TK50) with 1MB records, but only to be read
by a MicroVax not a PDP11.

...
  From "man
mt" on HP-UX (since at least 1992, so it fits the 10 year rule, too   :)
    DDS format devices also support setmarks which are hierarchically
    superior to filemarks.  A setmark is used to delineate a group (set)
    of files.  Reading a setmark is also returned as a zero-length read.
    The two can be distinguished by unique bits in the mt_gstat field.

 (In fact, #$%^ HP took out the above text from the 10.20 "man mt" doc, leaving
 just a bare reminder that setmarks exist)

 Who uses setmarks?  Backup software that wants to be able to locate
 files quickly.

 E.g., consider a modified tar/gtar that writes an EOF after every
 file, and a setmark after every 1000 files, and also writes s table
 of contents (list of files to be backed up) at the start of the tape.
 If the user says "restore file FOO", the software can read the table of
 contents and determine that FOO is file, say, 2349.  It can then issue
 two "forward setmark" commands, and then 349 "forward EOF" (or
"forward
 filemark" or "forward space file") commands to quickly get to the file.

 DDS drives can find a setmark very quickly, faster than they can find
 an EOF.

 So....any software trying to *accurately* duplicate a tape should
 record the fact a setmark was seen, as well as data record and EOFs
 and errors :) 
As I recall, DDS tapes can also have two partitions, independently
writeable.  So the tape index can be written onto the first partition
_after_ the data is written on the second partition.  I don't know any
common software that takes advantage of this.

    carl
-- 
        carl lowenstein   marine physical lab   u.c. san diego
                                          clowenstein(a)ucsd.edu