31 Dec
2006
31 Dec
'06
9:50 p.m.
Happy New Year!
I'm back at it again - I plan on doing better this time. (I've fixed
the specific double delete bug that bit me last time, improved tracing
quite a bit, and added a lot of consistency checking code.)
Just like last time, all you need to do is telnet to 24.159.203.149,
port 2301. Once again, that's my Linux box, but today it is forwarding
port 2301 to my PCjr. The commands are slightly enhanced from the last
test, but it isn't anything highly interactive.
Up to 9 people can connect simultaneously. We didn't get more than 2 or
3 simultaneous connections last time, so I'm probably overly optimistic.
Some of you noted from last time that there is no character echoing or
line editing. Telnet users on Unix machines will probably have local
line editing, which makes things a wee bit nicer. The standard Windows
telnet client doesn't do that by default, so you will type but not see
your characters. That's a project for a different day.
Here is a picture of the machine you are connecting to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
And lastly, if you remember, type in your OS and machine type as a
command. It will complain about it being an invalid command, but I'll
see it in the log and I'll have a better idea of what I'm testing against.
Thanks in advance,
Mike
31 Dec
31 Dec
10:03 p.m.
Some of you noted from last time that there is no
character echoing
or line editing. Telnet users on Unix machines will probably have
local line editing, which makes things a wee bit nicer.
Yes, I found I did.
The standard Windows telnet client doesn't do that
by default, so you
will type but not see your characters. That's a project for a
different day.
What, fixing Windows telnet to actually conform to the spec? Yes,
that's quite a project. :-)
And lastly, if you remember, type in your OS and
machine type as a
command. It will complain about it being an invalid command, but
I'll see it in the log and I'll have a better idea of what I'm
testing against.
Actually, it complained about it being two or three "Bogus command"s
(I'm not sure what the relevant difference between the two-complaint
ones and the three-complaint ones is.)
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
10:22 p.m.
der Mouse wrote:
The TCP/IP part has been interesting and fun. String parsing and doing
a user interface is not as fun, hence the limited set of commands and
lack of edit capability. At least for testing the TCP/IP code, it'll do
for now ..
I saw your session in real-time on the tcpdump. I suspect it didn't
disconnect cleanly, so I'll be going back through the logs to find out
what happened. (My side timed out sending a packet to your side.)
Thanks again,
Mike
Actually, it
complained about it being two or three "Bogus command"s
(I'm not sure
what the relevant difference between the two-complaint
ones and the three-complaint ones is.)
1 Jan
1 Jan
1:22 p.m.
I saw your session in real-time on the tcpdump. I
suspect it didn't
disconnect cleanly, so I'll be going back through the logs to find
out what happened. (My side timed out sending a packet to your
side.)
I initiated at least three, and I think four, sessions (from two
different addresses). One I deliberately disconnected by escaping and
telling telnet to quit; the rest, I think I disconnected with "quit".
I don't see why any of them would time out, unless you don't handle RST
segments correctly.
I'll be happy to work with you to debug this, if you want.
der Mouse
12:17 a.m.
On 31 Dec 2006 at 23:50, Michael B. Brutman wrote:
I'm back at it again - I plan on doing better this
time. (I've fixed the
specific double delete bug that bit me last time, improved tracing quite a
bit, and added a lot of consistency checking code.)
I telnet-ed from my Windoze to a Linux box and ran Telnet from there
to your server to get echoing. (What's the record for nested
telnets?)
It looked pretty good, but then I fired off some garbage characters
at it and it quit responding.
Cheers,
Chuck
12:27 a.m.
Michael B. Brutman wrote:
Here is a picture of the machine you are connecting to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
BTW, I'd like to re-re-direct people to that photograph. That has to be
the only PCjr in existence with real functioning ISA cards.
--
Jim Leonard (trixter at oldskool.org) http://www.oldskool.org/
Help our electronic games project: http://www.mobygames.com/
Or check out some trippy MindCandy at http://www.mindcandydvd.com/
A child borne of the home computer wars: http://trixter.wordpress.com/
12:37 a.m.
It was thus said that the Great Michael B. Brutman once stated:
Happy New Year!
Happy New Year!
I'm back at it again - I plan on doing better this
time. (I've fixed
the specific double delete bug that bit me last time, improved tracing
quite a bit, and added a lot of consistency checking code.)
Just like last time, all you need to do is telnet to 24.159.203.149,
port 2301. Once again, that's my Linux box, but today it is forwarding
port 2301 to my PCjr. The commands are slightly enhanced from the last
test, but it isn't anything highly interactive.
Up to 9 people can connect simultaneously. We didn't get more than 2 or
3 simultaneous connections last time, so I'm probably overly optimistic.
Well, I tested the nine simultaneous connections and yes, it seems to
work, although pressing ^C causes whatever program is listening on the other
end of the TCP connection to stop responding to commands.
Here is a picture of the machine you are connecting
to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
Very neat. How did you do that to a PCjr? Some sort of hacked up ISA bus
or something?
And lastly, if you remember, type in your OS and
machine type as a
command. It will complain about it being an invalid command, but I'll
see it in the log and I'll have a better idea of what I'm testing against.
sockets
9 sockets are active, 0 more are available
Ip Address and Port MSS State Started
66.252.226.50:61108 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
192.168.2.1:3018 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.226.50:61097 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
65.11.56.196:42377 1452 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.226.50:61111 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.227.139:52611 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
66.165.166.190:49217 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
64.124.179.116:36491 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
204.29.162.199:38999 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
These (with the exception of the 192.168.2.1 address) are me. All Linux
systems (2.0 to 2.6) with one Mac mini thrown in.
-spc (Oh, and your system clock is a bit off 8-)
6:23 a.m.
Hi guys,
It fell over again during a session with Spc. This failure isn't so
obvious .. it seems to function normally, and then the consistency
checking finds the heap corruption.
Once again, thanks for the help. Time to do a post-mortem.
Mike
6:26 a.m.
Sean Conner wrote:
ISA bus to PCjr bus adapter :-)
It wasn't too long after this that it died, so I'm going to test 9
simultaneous connections here.
Mike
Well, I tested the nine simultaneous connections and
yes, it seems to
work, although pressing ^C causes whatever program is listening on the other
end of the TCP connection to stop responding to commands.
I think that's a telnet feature - it probably sent an 'interrupt'
command, and the other side is not a real telnet server so it doesn't
respond in the proper way.
Here is a picture of the machine you are
connecting to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
Very neat. How did you do that to a PCjr? Some sort of hacked up ISA bus
or something? And lastly, if
you remember, type in your OS and machine type as a
command. It will complain about it being an invalid command, but I'll
see it in the log and I'll have a better idea of what I'm testing against.
sockets
9 sockets are active, 0 more are available
Ip Address and Port MSS State Started
66.252.226.50:61108 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
192.168.2.1:3018 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.226.50:61097 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
65.11.56.196:42377 1452 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.226.50:61111 1432 ESTABLISHED Mon Jan 01 02:13:50 2007
66.252.227.139:52611 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
66.165.166.190:49217 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
64.124.179.116:36491 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
204.29.162.199:38999 1460 ESTABLISHED Mon Jan 01 02:13:50 2007
These (with the exception of the 192.168.2.1 address) are me. All Linux
systems (2.0 to 2.6) with one Mac mini thrown in.
-spc (Oh, and your system clock is a bit off 8-)
7:16 a.m.
Found it! Testing is open again - hammer away at will!
(Hopefully this time I'm just going to sit back with a nice glow and
monitor it working perfectly.)
The postmortem:
Consistency checking is a good thing. The runtime of my compier has a
'heapcheck' function that walks the heap and verifies that everything is
in order. I have some code controlled by a #def that is doing this
checking (along with checking my own data structures) fairly often. It
impacts performance, but at least I can get closer the problem.)
There were almost no symptoms for this heap corruption - the machine was
running fine, sessions were still going, and DOS didn't behave badly
after I shut the server down. Nor could I see an obviously bad flow in
the traces. So I started diagnostics on the machine, thinking it was a
fluke.
While I was walking the dog this morning, it occurred to me what
happened. Simple buffer overun .. I allocated only 512 bytes for
output per session, but tried sending quite a bit more. Spc exposed my
crap coding with the 'sockets' command that showed nine sessions .. The
last time I tested with 9 simultaneous connections I didn't have the
consistency checking, so I was probably corrupting heap and not
realizing it.
Telnet users:
My understanding of telnet is that it is a protocol, not just raw
characters. So when you hit Ctrl-C under a typical Telnet client in
Unix, you are sending the equivalent of a 'process' interrupt.
I see lots of funny characters on my side, but they'll get consumed and
life goes on. I suspect that telnet is not so tolerant though, and is
expecting a proper response back from the telnet server. I don't have a
real telnet server here - it's not negotiating options or talking the
official protocol.
Short story .. don't bother with control chars. My code doesn't care,
and I think the normal telnet clients get tied up because my code isn't
responding right.
Anybody notice the BIOS date and machine ID? That's generated at
runtime, not a static string. :-)
Thanks again, and party on!
Mike
5:18 p.m.
Found it! Testing is open again - hammer away at
will!
Cool! I left you with a deliberately half-open socket, for your
testing pleasure. (I tried to leave you with another one, but it went
away - and the rest of the connections didn't, so I don't think it was
you resetting things.) It's the one from 216.46.1.51:64971. (I
created it by connecting and then hard-resetting the machine.)
Telnet users:
My understanding of telnet is that it is a protocol,
not just raw
characters.
You are correct. See RFC 854. :-)
So when you hit Ctrl-C under a typical Telnet client
in Unix, you are
sending the equivalent of a 'process' interrupt.
Most likely an Interrupt Process (0xff 0xf4) sequence, quite possibly
an Abort Output (0xff 0xf5) and/or a Synch (0xff 0xf2 with the TCP
urgent pointer pointing just after the 0xf2) as well.
I see lots of funny characters on my side, but
they'll get consumed
and life goes on. I suspect that telnet is not so tolerant though,
and is expecting a proper response back from the telnet server.
I'm inclined to doubt it. There is no defined response that a telnet
client can reasonbly expect to IP, AO, or Synch.
I'll snoop traffic and try a ^C to see what's really going on - at the
wire level, at least.
Anybody notice the BIOS date and machine ID?
That's generated at
runtime, not a static string. :-)
I noticed it, but had no way to tell whether it were static or not. :)
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
5:42 p.m.
So when you
hit Ctrl-C under a typical Telnet client in Unix, you
are sending the equivalent of a 'process' interrupt.
Most likely an
Interrupt Process (0xff 0xf4) sequence, quite possibly
an Abort Output (0xff 0xf5) and/or a Synch (0xff 0xf2 with the TCP
urgent pointer pointing just after the 0xf2) as well. [...]
I'll snoop traffic and try a ^C to see what's really going on - at
the wire level, at least. 
9:39 a.m.
Michael B. Brutman wrote:
Happy New Year!
I'm back at it again - I plan on doing better this time. (I've fixed
the specific double delete bug that bit me last time, improved tracing
quite a bit, and added a lot of consistency checking code.)
Works for me
Just like last time, all you need to do is telnet to
24.159.203.149,
port 2301. Once again, that's my Linux box, but today it is forwarding
port 2301 to my PCjr. The commands are slightly enhanced from the last
test, but it isn't anything highly interactive.
Up to 9 people can connect simultaneously. We didn't get more than 2 or
3 simultaneous connections last time, so I'm probably overly optimistic.
I can leave a telnet session connected indefinitely, if it won't time out...
Some of you noted from last time that there is no
character echoing or
line editing. Telnet users on Unix machines will probably have local
line editing, which makes things a wee bit nicer. The standard Windows
telnet client doesn't do that by default, so you will type but not see
your characters. That's a project for a different day.
Here is a picture of the machine you are connecting to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
And lastly, if you remember, type in your OS and machine type as a
command. It will complain about it being an invalid command, but I'll
see it in the log and I'll have a better idea of what I'm testing against.
Done
Gordon
7:31 p.m.
New subject: TCP/IP Testing help, Round 2 - Done!
This round of testing is over. Here are the highlights:
- No heap corruption since I fixed the buffer overflow bug this morning.
(Doh!)
- I have a lot of a lot of open connections in ESTABLISHED state which I
suspect are no longer valid, but because there is no data transfer there
is no error. I already knew I'd have to implement a session timeout or
keep-alives, and this confirms it.
- I have one connection in FIN_WAIT_2. I suspect that will never close
on it's own because the host has long since forgotten about it, so I'll
need to implement a cleanup timer for those. (I read that Berkeley
derived stacks do this, even though it is a violation of the spec.)
- I suspect I need to work on my RST segment processing - I think I'm
being too strict about what I consider to be an allowable RST. It's all
in the RFCs, I just must not be grokking.
I'm not going to be testing anything again anytime soon .. it's back to
real work, and I have a lot to do before I can subject anybody to my
code again. Besides the issues above I need to work on the following:
- Besides tracking window sizes, actually honor them. (My code is ill
behaved, and can send more than the window advertises.)
- Telnet options negotiation
- Some sort of 'select' call so that the app doesn't have to keep
polling each socket directly. This is DOS so underneath the covers it
will poll anyway, but having the app do it makes the code nearly
unmanageable.
- Go through the logs (both on the server machine and the tcpdump logs
on the Linux firewall) and see what else I goofed up.
Once again, thanks to those of you who helped me test. I've never
undertaken something this large at home - this has been a year in the
making. TCP/IP on an 8088 class machine isn't particularly new and
exciting, but having my own implementation allows me to determine the
balance of features and performance, and to fix the dang bugs that will
pop up. :-) This code (without disk access) can transfer an receive on
a socket at over 50KB/sec on a PCjr with a NEC V20. That ain't shabby
.. :-)
Mike
8:50 p.m.
New subject: TCP/IP Testing help, Round 2 - Done!
TCP/IP on an 8088 class machine isn't particularly
new and exciting,
Oh, I don't know; *I* think this is Pretty D*mn Cool!
Well, okay, TCP on an 8088 may not be exciting. TCP using your own
code on an 8088, that rocks!
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
2 Jan
2 Jan
6:47 p.m.
New subject: TCP/IP Testing help, Round 2 - Done!
der Mouse wrote:
And thanks again for the testing - I watched some of your sessions in
real time by looking at the tcpdump output.
I started this project about a year ago by trying to get the packet
driver for a Xircom PE3 10BT (parallel port Ethernet adapter) to send a
packet. That worked pretty well - I had to learn how to mix x86 ASM in
my C code, but it worked. Receiving packets was a different story - the
dang Xircom packet driver was written in a non-friendly way, so
interfacing to it was not as easy as it should have been. After two
days of disassembling code, hacking, and experimenting I finally got
data in and out.
After that I needed ARP. Then I went to UDP. TCP scared me, so I
procrastinated quite a bit. I still have a bit of cleanup and testing
to do, but I'm finally over the hump again and it's back to being fun.
About six months ago I found a good ISA bus adapter for the Jr and I
tried out that Western Digital card. Wow .. compared to the Xircom on
the parallel port, it's a rocket. The Xircoms are a good solution for
limited machines, but I definitely need to reproduce the adapter card.
At some point when it is a little more polished and tested I'll put a
web page up and release it to the universe. There are a handful of
TCP/IP stacks out there already, but I think mine is going to be the
fastest by a pretty wide margin.
Part of the joy of coding is seeing others make use of the code. I'd
like to see some new/refreshed TCP/IP apps for older machines. I was
intending to do a telnet BBS, but that's going to take a lot more effort
and the audience for that is fairly limited.
Mike
TCP/IP on an
8088 class machine isn't particularly new and exciting,
Oh, I don't know; *I* think this is Pretty D*mn Cool!
Well, okay, TCP on an 8088 may not be exciting. TCP using your own
code on an 8088, that rocks!
3 Jan
3 Jan
5:08 a.m.
New subject: TCP/IP Testing help, Round 2 - Done!
At some point when it is a little more polished and
tested I'll put a web
page up and release it to the universe. There are a handful of TCP/IP
stacks out there already, but I think mine is going to be the fastest by a
pretty wide margin.
Mike, the RTL8019 is a very friendly chip for 8 bits, even being used
into a 16-bit card. Almost all embeeded systems uses the 8019 as the
ethernet interface. Take a look at the packet whacker in www.edtp.com
If you want to skip programming and have a independent tcp stack, you
can also take a good look in the wizchip series of chips and modules. They
have 10/100 modules with internal tcp stack and even a wireless one! I used
them in my last project. Very friendly to use.
Congratulations for a great job! So bad I've never ever touched one
PCJr. But my first colour CGA monitor was taken from one :D
Greetz from Brazil
Alexandre Souza
http://www.tabajara-labs.com.br
10:05 a.m.
New subject: TCP/IP Testing help, Round 2 - Done!
- Telnet
options negotiation
be careful when doing this. I've written a few different
telnet
servers and it's very easy to end up in a loop. 
11:20 a.m.
New subject: TCP/IP Testing help, Round 2 - Done!
On 1/1/07, Michael B. Brutman <mbbrutman-cctalk at brutman.com> wrote:
- I suspect I need to work on my RST segment
processing - I think I'm
being too strict about what I consider to be an allowable RST. It's all
in the RFCs, I just must not be grokking.
What do you mean by "allowable RST?" RST can be thought of as
hard-close, or "Sorry peer, I don't have any context about the
connection you sent a packet for," or "this port is not accepting
connections."
It is always allowed. It is not the ideal way to close a connection,
but as a peer receiving it you must always handle it, and you may send
it if you have a good reason.
Some consideration may be given to detecting spoofed RSTs attempting
to interfere with valid connections. But I suspect you are not
terribly interested in complicating your code with such corner case at
this point (there are a lot of things like this if you really want to
get into it).
-- John.
6:34 p.m.
New subject: TCP/IP Testing help, Round 2 - Done!
John R. wrote:
What do you mean by "allowable RST?" RST can
be thought of as
hard-close, or "Sorry peer, I don't have any context about the
connection you sent a packet for," or "this port is not accepting
connections."
It is always allowed. It is not the ideal way to close a connection,
but as a peer receiving it you must always handle it, and you may send
it if you have a good reason.
Some consideration may be given to detecting spoofed RSTs attempting
to interfere with valid connections. But I suspect you are not
terribly interested in complicating your code with such corner case at
this point (there are a lot of things like this if you really want to
get into it).
There are lots of rules in RFC 793 for RST segment generation and handling.
You don't want to just whack any connection that gets an RST. The RST
has to have correct sequence and ack numbers in the header. Correct can
be extended to mean 'in the allowable window'. Otherwise, somebody can
have some real fun with your server by just spraying random RST segments
at you.
I code my RST handling code such that the RST has to be within the
current receive window, which is how I read 793. I think I have misread
or misinterpreted, and I should include RST packets that are in my
window of previously sent (but not yet acknowledged) packets.
Corner cases are an interesting design problem. I could make the code
quite a bit smaller if I just tweaked the spec a bit. Things like not
supporting two sides to do active opens at each other can save code, and
it is not a commonly used feature. I might go back and start sprinkling
my code with #defines to delete features like that if they are not needed.
9:31 p.m.
New subject: TCP/IP Testing help, Round 2 - Done!
> [...RST handling...]
You don't want to just whack any connection that
gets an RST. [...]
Otherwise, somebody can have some real fun with your server by just
spraying random RST segments at you.
True. (At least if ths stack is for use on the open Internet. In some
rather closed environments, it may be perfectly acceptable to take down
a connection on any RST with the right address/port numbers.)
I code my RST handling code such that the RST has to
be within the
current receive window, which is how I read 793. I think I have
misread or misinterpreted, and I should include RST packets that are
in my window of previously sent (but not yet acknowledged) packets.
Not quite, I think. Rather, you should take an RST only when its
sequence number is right at the edge of the window, provided you send a
pure-ACK packet in response to an RST with a bad sequence number.
These make it significantly harder for third parties to shoot down your
TCP connections. If all they need to do is hit your window, the chance
of a random sequence number working is window-size/2^32, which can get
comparatively large, especially when window scaling is in use (though
admittedly I imagine a PCjr stack does not do window scaling :). But
if you accept an RST only when its sequence number is at your window's
edge, the chance is 1/2^32 regardless of window size. In order to take
down legitimately half-open connections, you need to communicate that
sequence number to the legitimate peer, which is what the pure-ACK in
response to wrong-sequence RSTs is for - to elicit a properly-sequenced
RST. (If you want to be really spiffy, rate-limit the pure-ACK
packets, so you can't be used to turn an RST flood into an ACK flood.)
I think this algorithm is written up in an RFC somewhere, but I forget
the number.
/~\ The ASCII der Mouse
\ / Ribbon Campaign
X Against HTML mouse at rodents.montreal.qc.ca
/ \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
10:45 a.m.
Here is a picture of the machine you are connecting
to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
Nice!
6:39 p.m.
Alexandre Souza wrote:
Glad you like it!
A few years ago I did a pin-to-pin mapping from the Jr's bus to ISA.
Except for DMA and some IRQs, it's doable.
The particular adapter that I'm using is a commercial product from the
mid 80s. I know of somebody who designed their own version of it and is
running an XT hard disk controller with a modified BIOS on their Jr.
Cards that do not use DMA and makes limited use of IRQs are candidates
for running on the machine. Conflicts with built-in hardware have to be
avoided as well, as the onboard diskette and serial ports can't be
disabled without a wire cutters.
I've run that Western Digital 8003, a 3Com 3C503, an NE1000, a Trantor
T130 SCSI card, and a Future Domain 85x variant in the adapter. One day
I'll figure out how to rework the BIOS on the Future Domain cards so
that I can get the machine to boot from the hard disk.
Mike
Here is a
picture of the machine you are connecting to, complete with
the Western Digital 8003 Ethernet card grafted onto the side:
http://brutman.com/PCjr_WD_small.jpg
Nice!
6344
days inactive
6346
days old
23 comments
9 participants
participants (9)
-
alexandre-listas＠e-secure.com.br -
brad＠heeltoe.com -
cclist＠sydex.com -
gordon＠gjcp.net -
jhoger＠gmail.com -
mbbrutman-cctalk＠brutman.com -
mouse＠Rodents.Montreal.QC.CA -
spc＠conman.org -
trixter＠oldskool.org