[e2e] TCP Framing

[John Day]

>Hi.  At the IETF just completed, I sat through an exposition of
>the following Internet Draft:
>
>   	"Title		: ULP Framing for TCP
>	Author(s)	: J. Williams et al.
>	Filename	: draft-williams-tcpulpframe-01.txt
>	Pages		: 12
>	Date		: 22-Mar-01
>
>     This document proposes a framing protocol for TCP which is designed
>     to be fully compliant with applicable TCP RFC's and fully
>     interoperable with existing TCP implementations. The framing
>     mechanism is designed to work as a 'shim' between TCP and higher-
>     level protocols, preserving the reliable, in-order delivery of TCP
>     while adding the preservation of higher-level protocol record
>     boundaries if the record is less than or equal to the path MTU. The
>     shim is designed to enable hardware acceleration of data movement
>     operations (e.g. direct placement of receive TCP segments into
>     higher-level protocol buffers) for the protocols that use it, even
>     if TCP segments are delivered out-of-order."

Bob, let me take a stab at this.  I have thoroughly read your note 
but have not read the draft yet.  I wil do that next, but it is late 
in the day here and I don't want to wait until I can get to the draft 
which might not be until tomorrow sometime.

I have agonized over this problem for almost 30 years.  I have for 
some time believed that TCP approach was the proper approach.  (This 
stemming from our early discussions back in the early 70's when 
Multics did streams and that IBM system you were in charge of did 
records.  I think I remember you arguing hard for record capabilities 
in some of those early meetings!  ;-)  Sorry couldn't resist. 
Streams were elegant.  I remember how much I detested those damn half 
duplex terminals and the push for records in Telnet.  Telnet was the 
most elegant thing we created in that early batch of protocols.) 
Streams were more general than records. (In those days, records 
usually meant fixed length as you no doubt remember.)  Records were a 
pain in the neck (or something else).  One of the mistakes I always 
thought the OSI model made (it wasn't just TP4) was the definition 
that "the identity of SDUs were maintained end-to-end."  SDU (Service 
Data Unit) was the lump of stuff handed across the layer boundary. 
SDUs could be fragmented or contatenated however the protocol wanted 
into any number of PDUs for sending to the other end.  The only 
requirement was the the guy on the other end was handed the same SDU 
that it had been given.

Now it is the case that some applications do work with fixed or 
variable records and can't do anything until they have the whole 
thing.  In fact most of our applications today do this except Telnet 
and it probably should be.  (Telnet processing would be a whole lot 
more efficient if you could find the next IAC without looking at 
every byte  But that is a nit and much less a problem now than with 
the hardware then.)  Right, it was quite a revelation to me when I 
realized that the one protocol that I really thought was pure stream 
would be more efficient not with records but with SDUs.)

Now clearly it can be rightly argued that if an application wants to 
see records, then it should do it.  Of course, the application 
designer could just as easily argue that "there are many more 
applications that need to do this than just the one application.  And 
isn't one of the main design principles that if lots of things need 
the same function that rather than do it n times (potentially 
differently), it should be done once consistently?  TCP already has 
the machinery to do reassembly, why not have it do it, why should I 
have to replicate this byte diddling stuff in my application. 
Afterall, it is perfectly legal for TCP to deliver my data one byte 
at a time.  If I am lucky, one call will give me what I need but more 
likely it will take more system calls to TCP before I get something I 
can use.  Why does it need to be that much work."  Well, yea. . . .

So I started looking for some other reason to decide which side of 
the line it should be done on.  Frankly, I haven't found any.  It 
appears that the work is the same whether TCP does it or the 
application does it.  Although, it may be replicated code, but that 
is not a big deciding factor. So no real architectural argument there.

The only thing I have really found goes against us and that sort of 
comes from thinking about these things as objects.  At first I 
thought that the only difference between the OSI SDU and what we 
thought of as records was that records were fixed length and SDUs 
could be variable in length.  But at some point, I realized that from 
the application's point of view it was something different:  "Here I 
want you to send this stuff.  I don't care what you do to it.  Send 
it all at once, send it with something else, break it up into big or 
little pieces, I don't care what kind of mess you make of it.  Just 
clean up after yourself when you are done."  And I thought hmmmm, 
that isn't an unreasonable demand by the layer above and it maintains 
the invariance of the interaction which is always a strong design 
property for interfaces.

So these days, I am still on the fence, but leaning toward a solution 
that maintains invariance, i.e.  what I put in at one end will come 
out the other.  This is really the orthogonality that all interfaces 
should exhibit.  groan.

>
>I would like to suggest two things about this, one simple and one
>subtle.  The simple one is this: to say that the ULP framing is fully
>compliant with the applicable TCP RFCs is simply false.  For some of
>us, at least, such a lack of truth in technical advertising is a red
>flag.
>
>The reason why it is false, and its consequences, form the subtle bit.
>It is true that the proposed shim does not change the definition of the
>TCP protocol on the wire.  However, it does change a more fundamental
>principle of TCP, which is the deliberate decoupling of what happens on
>the wire from what the user sends.  (See the following sentences from

Yes, but in some sense we have decoupled the sender but coupled the 
receiver more tightly.

>RFC 793, for example:
>
>     The TCP is able to transfer a continuous stream of octets in each
>     direction between its users by packaging some number of octets into
>     segments for transmission through the internet system.  In general,
>     the TCPs decide when to block and forward data at their own
>     convenience.
>
>The last sentence may be phrased in a slightly academic manner;
>the reader is assumed to understand the the "convenience" of the
>transport layer is to provide optimal performance.  In an earlier
>paragraph the spec says:
>
>     TCP is designed to work in a very general environment of
>     interconnected networks.
>
>Now for the subtle bit.  Generality and optimization are typically
>contradictory.  The Internet protocol suite was designed deliberately
>and carefully for generality, at the possible expense of optimization.
>It was also designed for simplicity at the expense of optimization.
>We recognized that later engineering efforts would rob some of the
>simplicity in order to reach greater optimization, and indeed,
>this has happened and is probably not a bad thing.  On the other
>hand, we should be very wary of over-engineering optimal solutions
>that cut down the generality.

Above I was trying to point out that this is generality on sending 
but not for receiving.  It does not impact either generality or 
optimality "to clean up after yourself." Be careful about the 
generality thing:  It was generality as we understood it in the 
mid-70's.  (I have found some aspects of TCP that I thought were done 
for good performance and design reasons and have recently realized 
that they were really based on the nature of the traffic at the time 
and that those conditions no longer hold.  ooops.)  We weren't and 
aren't infallible.

>
>The ULP protocol is a classic example of this issue.  The TCP spec
>deliberately decoupled the packaging of data across the API (the ADPU,
>if you will) from the segmentation that TCP does on the wire.  This was
>not an accident; it was designed to allow TCPs to be able to adapt to
>new and different environments, without requiring that applications
>change.  The ULP proposal changes this to tight coupling, since it only
>works if the send call units are mapped directly into segments.  So
>adopting ULP MAY (and note that one cannot ever be sure that it
>will/won't) reduce the freedom of TCP to adapt to future changes.
>And contrary to what the ULP proponents claim, it is a very fundamental
>change in TCP.  We should think VERY carefully before making such
>a change, and we should be honest about what we are doing.

You may be right here.  I will read the document later and see what I 
think.  IF ULP is really a protocol on top of TCP, I don't see how it 
can constrain TCP.  I don't even see that every application that uses 
TCP would have to use ULP.  But I also would agree with you about 
whether this is really a necessary addition at this point.

>
>Moral: Over-engineering may be bad for the Internet, eventually.
>
>Finally, an historical irony: the ULP hack is acknowledged to be a
>stopgap until SCTP has advanced to save us.  Essentially, SCTP is OSI
>TP4 with features.  TCP's idea of decoupling API from wire protocol
>units was, at the time of its development a new idea that was at
>variance with the evolving OSI suite.  Now, things seem to be
>running full circle.  Are we really sure we want to do this?

I have browsed through SCTP and it looks like a bell-heads dream. 
Not sleek and simplicity at all.  Just lots and lots of mechanism and 
weight.  I would also agree with your appraisal of the OSI situation 
at the time.  I think that when the people put the SDU thing in the 
OSI stuff they were trying to force a fixed record view of the world. 
The problem was that it gotten written in general terms and while you 
could read that into it (and I did for years).  Later as I indicated, 
I began to realize that it actually said something much more 
interesting.  So now when I talk about this, I say there are three 
cases:  Stream, record and for lack of a better term Idempotent for 
the literal interpretation of the SDU because it keeps things 
invariant.

Well, there it is something for you to think about too.  ;-))) 
Whaddya think?  Now to read the damn thing!  Gotta go to dinner, 
people are hollering at me.

Take care,
John