29 Sep
2010
29 Sep
'10
6:24 p.m.
I have been working on my TCP/IP stack for DOS, adding IP fragmentation
support. There are not too many more features that I want to add to
make it 'complete' before I open source the code and IP fragment support
was a big one.
I am having a terrible time testing it though. It seems that IP
fragments out in the wild are pretty rare. I tried connecting to a slew
of remote FTP sites hoping to find one that was behind a really bad
network, and thus would have fragments coming from it. No joy.
It seems that there are a lot of tricks out there to prevent fragments
from being created, especially when using TCP. The
only way I can test
the code is to send myself oversized UDP packets. If it works
for UDP
then it should work for TCP too, but I'd really like to test the TCP
path explicitly. Combine the tricks with modern broadband and getting
fragments is really difficult.
Even on the home network I am having a hard time getting fragments. I
put a Linux box between the DOS PC and a Windows machine, and set one of
the Ethernet MTUs to 576. Well, that didn't force fragments because the
Windows box is too clever. I could start turning everything off in the
registry, but I really don't want to get that involved.
Off the top of my head I think I am going to have to get another Linux
box and dumb that down, if it is possible. Dumbing Linux down to turn
off the features and then restoring it to a good state is probably
safer/easier than doing it with Windows.
Does anybody have a good technique for setting up a simple network that
will result in IP fragments of TCP?
On a related note, is this even worth it? I don't know of anything that
needs to send fragments except for NFS over UDP. There might be other
applications that send big packets over UDP but those would be the only
class of applications that absolutely require fragment support. With
TCP it is nice, but a user should be able to get around any problem by
setting the local MTU to 576.
Mike
29 Sep
29 Sep
6:37 p.m.
At 09:24 PM 9/29/2010, Michael B. Brutman wrote:
Does anybody have a good technique for setting up a
simple network
that will result in IP fragments of TCP?
Set up a network going through a SLIP link with a small MTU. That's the
easiest way to test such things that I've ever found. We used to use
this kind of configuration to verify VPN implementations across limited
MTUs.
-Rick
6:55 p.m.
Rick Murphy wrote:
At 09:24 PM 9/29/2010, Michael B. Brutman wrote:
You might also block all icmp in between so pmtud doesn't work.
Peace... Sridhar
Does anybody have a good technique for setting up
a simple network
that will result in IP fragments of TCP?
Set up a network going through a SLIP link with a small MTU. That's the
easiest way to test such things that I've ever found. We used to use
this kind of configuration to verify VPN implementations across limited
MTUs. 
8:38 p.m.
On 9/29/10 9:55 PM, Sridhar Ayengar wrote:
You might also block all icmp in between so pmtud doesn't work.
That oughtta be easy enough to test. Every clueless network admin in
the country blocks ICMP.
-Dave
--
Dave McGuire
Port Charlotte, FL
Does anybody have a good technique for setting up a
simple network
that will result in IP fragments of TCP?
Set up a network going through a SLIP link with a small MTU. That's the
easiest way to test such things that I've ever found. We used to use
this kind of configuration to verify VPN implementations across limited
MTUs. 
9:42 p.m.
Michael B. Brutman wrote:
Does anybody have a good technique for setting up a
simple network
that will result in IP fragments of TCP?
Interpose a router with two interfaces, one configured with a smaller
MTU. I expect that's what you're using the Linux box to do. It's either
that or fiddle with the sending IP stack so that IP knows the network
has a small MTU but TCP doesn't find out.
If the sending host is setting the DF bit expecting to get ICMP
messages back for path MTU discovery, hmm, that will make this tricky:
you could get your router to drop those ICMP messages before they go
out, but then your router will give the appearance of silently
dropping those DF'd datagrams, and the sending host's path MTU
discovery may discover the path MTU slowly through timeouts and
backoff, or it may not discover it at all, with the result that the
send just fails.
What you really want to do in that case is have your router clear the
DF bit, adjust the IP header checksum, and then go on to fragment the
datagram before sending it out the smaller-MTU interface. This isn't
RFC-compliant behavior, but you're wanting to test stuff, right?
And since you're wanting to test stuff, the next things will be to
check that you're assembling the datagram correctly when you've got
all the fragments, and to check that you give up when you don't get
all the fragments after some time, and to check that you don't leak
memory or packet buffers either when you assemble the datagram or when
you give up on the datagram.
I wouldn't use Linux for this, but that's because I've done things
like this before, using FreeBSD, ipfw, dummynet, netgraph, and
a small C program to do the DF-clearing stuff. They're the tools
I'm comfortable with for this sort of thing. You may have a learning
experience either way, and you've already got a penguin handy, you
probably know how to work it better than I do.
On a related note, is this even worth it? I don't
know of anything
that needs to send fragments except for NFS over UDP. There might be
other applications that send big packets over UDP but those would be
the only class of applications that absolutely require fragment
support. With TCP it is nice, but a user should be able to get around
any problem by setting the local MTU to 576.
It's not that things send fragments, it's that a network link in the
middle has a smaller MTU than the networks on the ends and that
routers having more knowledge of this than the hosts on the ends
fragment large packets as they pass through.
The user first has to recognize the problem as one that can be got
around in this way. That's a learned response, and I'm not sure how
people learn it these days. And yeah, you may be doing this for a
PCjr and not supporting a web browser or NFS, but FTP can trip over
this pretty easily, and Telnet can too if the phase of the moon is
just right.
What I'm really saying here is, if you should decide that fragment
reassembly isn't worth doing, think instead about having your TCP
stack figure out that it's going to be sending to a non-local host,
and adjust its MSS downward for those connections. And giving the
user a way to turn that behavior off if he's sure it's safe and he
finds the switch.
-Frank McConnell
30 Sep
30 Sep
12:12 p.m.
Frank McConnell wrote:
Correct, but actually in the case of large UDP packets the source
machine sends the fragments. TCP is too dang smart and tries all of the
path MTU discovery tricks, but UDP doesn't have that luxury. Which is
great because it does give me an easy way to test - but only with UDP.
I put the gateway in the middle of the machines to try to force TCP
fragments. I may have to get more perverse and use a SLIP connection to
the DOS machine, which has a much smaller MTU. But I suspect that the
TCP path length discovery will get in the way again. (I'm really going
to have to try to turn all of that off.)
Michael B. Brutman wrote:
That is exactly what the Linux box in the middle is supposed to be
doing. One interface is setup for an MTU of 1500, and the other is
setup for an MTU of 576.
Does anybody have a good technique for setting up
a simple network
that will result in IP fragments of TCP?
Interpose a router with two interfaces, one configured with a smaller
MTU. I expect that's what you're using the Linux box to do. It's either
that or fiddle with the sending IP stack so that IP knows the network
has a small MTU but TCP doesn't find out. If the sending host is setting the DF bit expecting to
get ICMP
messages back for path MTU discovery, hmm, that will make this tricky:
you could get your router to drop those ICMP messages before they go
out, but then your router will give the appearance of silently
dropping those DF'd datagrams, and the sending host's path MTU
discovery may discover the path MTU slowly through timeouts and
backoff, or it may not discover it at all, with the result that the
send just fails.
All of the modern TCP/IP implementations seem to use a combination of
these techniques. The Windows XP machine I'm using for the FTP server
definitely is, and I really don't want to mess with the registry.
Overall this is a good thing for end users, but a pain for those of us
trying to implement.
What you really want to do in that case is have your
router clear the
DF bit, adjust the IP header checksum, and then go on to fragment the
datagram before sending it out the smaller-MTU interface. This isn't
RFC-compliant behavior, but you're wanting to test stuff, right?
And since you're wanting to test stuff, the next things will be to
check that you're assembling the datagram correctly when you've got
all the fragments, and to check that you give up when you don't get
all the fragments after some time, and to check that you don't leak
memory or packet buffers either when you assemble the datagram or when
you give up on the datagram.
I wouldn't use Linux for this, but that's because I've done things
like this before, using FreeBSD, ipfw, dummynet, netgraph, and
a small C program to do the DF-clearing stuff. They're the tools
I'm comfortable with for this sort of thing. You may have a learning
experience either way, and you've already got a penguin handy, you
probably know how to work it better than I do.
I'm hesitant to start writing my own router test code because it is
error prone too and I'll wind up making my code match the test. But
after giving it some more thought, I would get much better error
injection capability than I have now if I started mucking with the
packets myself.
On a related
note, is this even worth it? I don't know of anything
that needs to send fragments except for NFS over UDP. There might be
other applications that send big packets over UDP but those would be
the only class of applications that absolutely require fragment
support. With TCP it is nice, but a user should be able to get around
any problem by setting the local MTU to 576.
It's not that things send fragments, it's that a network link in the
middle has a smaller MTU than the networks on the ends and that
routers having more knowledge of this than the hosts on the ends
fragment large packets as they pass through. The user first has to recognize the problem as one
that can be got
around in this way. That's a learned response, and I'm not sure how
people learn it these days. And yeah, you may be doing this for a
PCjr and not supporting a web browser or NFS, but FTP can trip over
this pretty easily, and Telnet can too if the phase of the moon is
just right.
Right. And that's why I want to add fragmentation support. I get an
occasional report of a program not working, and if it just looks like a
failed connection or a packet loss problem there isn't much I can do. I
set the DF bit on, but ICMP gets filtered too often so I can't rely on
it to tell me that fragmentation is needed. For those users I tell them
to lower the MTU, but I don't know how many people are giving up without
talking to me first. (And I have all of this in the README, but nobody
reads those.)
What I'm really saying here is, if you should
decide that fragment
reassembly isn't worth doing, think instead about having your TCP
stack figure out that it's going to be sending to a non-local host,
and adjust its MSS downward for those connections. And giving the
user a way to turn that behavior off if he's sure it's safe and he
finds the switch.
-Frank McConnell
That's not a bad idea! Local subnet = large MTU is probably safe,
anything else fall back to 576.
The additional expense of the fragmentation reassembly code is 3KB of
code, and anywhere from 4kb to 32kb of data depending on the
configuration. I was pre-allocating the memory for the packet
reassembly; seeing how rare it really is I might go to malloc and deal
with the performance penalty. Once you start having to reassemble
packets your performance has gone to hell anyway.
Thanks - this has been helpful.
Regards,
Mike
1:33 p.m.
It was thus said that the Great Michael B. Brutman once stated:
Correct, but actually in the case of large UDP packets the source
machine sends the fragments. TCP is too dang smart and tries all of the
path MTU discovery tricks, but UDP doesn't have that luxury. Which is
great because it does give me an easy way to test - but only with UDP.
I put the gateway in the middle of the machines to try to force TCP
fragments. I may have to get more perverse and use a SLIP connection to
the DOS machine, which has a much smaller MTU. But I suspect that the
TCP path length discovery will get in the way again. (I'm really going
to have to try to turn all of that off.)
While I'm familiar wih IP, I haven't tried implementing it, but, what if
you were to set an MTU size of around 50 bytes? The IP header (sans
options) is 20 bytes, and the TCP header is another 20. I would think
setting the MTU to just above 40 might cause some fragmentation (I thought
of suggesting an MTU of just above 20, but then you're testing for IP
fragmentation, and not TCP fragmentation.
I tend to do stupid stuff like that, but then again, I *was* hired to
write testing code at my current job ...
-spc (In thoery, IP should work with an MTU of 30 bytes, right?)
1:51 p.m.
Sean Conner wrote:
While I'm familiar wih IP, I haven't tried
implementing it, but, what if
you were to set an MTU size of around 50 bytes? The IP header (sans
options) is 20 bytes, and the TCP header is another 20. I would think
setting the MTU to just above 40 might cause some fragmentation (I thought
of suggesting an MTU of just above 20, but then you're testing for IP
fragmentation, and not TCP fragmentation.
I tend to do stupid stuff like that, but then again, I *was* hired to
write testing code at my current job ...
-spc (In thoery, IP should work with an MTU of 30 bytes, right?)
The IP header is 20 bytes without any IP header options, and the TCP
header is another 20 bytes without any TCP header options.
My implementation has room for 10 fragments per packet (configurable
with a #define), which is great for a 'normal' MTU size. If I tested
with a ridiculously small MTU it would wind up throwing a lot of things
away, but many implementations do the same thing.
My fragment problem is mostly on the source side - the source machines
are too clever about trying to probe and eliminate fragments.
Linux has also been 'interesting' as a gateway. Some kernel versions
allow TCP fragments through even though they have bad checksums. I
throw away any packet with a bad checksum. Mixing NAT (Network Address
Translation) and fragments was problematic in my setup too - I think
that Linux was completely screwing up the payload checksums for TCP.
Mike
1:57 p.m.
Sean Conner [spc at conman.org] wrote:
While I'm familiar wih IP, I haven't tried
implementing it,
but, what if you were to set an MTU size of around 50 bytes?
There's a requirement that you be able to support 68 byte packets
(iirc).
But his problem is almost certainly that some form of MTU discovery is
in
use and it's thwarting his testing. It should be possible to overcome
that
either by disabling it (which probably does mean hacking the registry,
using a VM might make this less risky) or by refusing to respond to
whatever style of packets the source uses to probe (or even faking
a "godd" response).
Antonio
4 Oct
4 Oct
7:18 p.m.
Well, it turned out to be far easier to generate fragments than it was
to debug them. I had the perfect TCP/IP stack that didn't try any fancy
tricks to probe the MTU size all along - my own!
An instance of DOSBox with routing properly setup to go through the
Linux box did the trick. I ran netcat on both sides, and soon was awash
in fragments.
Then came the part I hate about DOS programming - memory corruption and
crashes. I really hate it when I can't find the bug that causes the
problem; I'm really paranoid about that kind of thing because once a
little entropy creeps into the system, it's hard to remove. Eventually
I figured out my bugs and it's been happily reassembling packets under
all sorts of stressful conditions for the last few hours.
Mike
5192
days inactive
5197
days old
9 comments
7 participants
participants (7)
-
arcarlini＠iee.org -
fmc＠reanimators.org -
mbbrutman-cctalk＠brutman.com -
mcguire＠neurotica.com -
ploopster＠gmail.com -
rick＠rickmurphy.net -
spc＠conman.org