Marking the last byte with the 8th bit saved using a leading string length byte or a
trailing zero string delimiter.
It dates back to the days when saveing one byte per keyword in the lookup table was a big
memory savings :)
Bob
On Sun, 27 Aug 2006 23:01:56 -0400, Roy J. Tellason wrote:
On Saturday 26 August 2006 09:23 pm, Dave Dunfield
wrote:
> I think you are right about the '=' as
well - funny that they would
> "tokenize" in this manner - I've written several BASICs over the years
-
> some which operated on pure plain text (no tokenization at all), but most
> replaced all keywords with a single easily recognizable token - I don't see
> what advantage this type of tokenization could have done as it saves no
> space, and you still have to parse the whole word.
Is there any chance that there are more than one
keyword that share the same
first few characters?
This reminds me of wordstar "document"
format, too...
--
Member of the toughest, meanest, deadliest, most unrelenting -- and
ablest -- form of life in this section of space, a critter that can
be killed but can't be tamed. --Robert A. Heinlein, "The Puppet Masters"
-
Information is more dangerous than cannon to a society ruled by lies. --James
M Dakin