[e2e] Policing TCP flows

[Vadim Antonov]

<delurking>

Dennis,

> The necessary length of a queue is supposed to be on the order of the
> delay-bandwidth product of the path, which means that in a "normal" case
> an absolutely full queue shouldn't do more than double the RTT.

This is not exactly the case, because in the real-life Internet RTTs vary
a lot.  This means that it is not path RTT, it is characteristic RTT of
the network which has to be used to choose buffer size.  So, for
short-haul paths buffers are too large, thus increasing latencies
unnecessarily, and for long-haul paths they are not sufficient to
accomodate delays in the TCP congestion control's loop causing stop-and-go
behaviour.

Additionally, characteristic RTT itself depends on buffer sizes; ACKs get
stuck in the similarly-sized queues in the return path.  This should
inflate the required buffer sizes somehow - more if traffic is symmetric,
less if it is heavily asymmetric.  (Did anyone try to attach higher
queueing priority to pure ACKs?)

> Furthermore,
> RED and other congestion feedback schemes (or their policer equivalents) are
> supposed to moderate the queue occupancy such that the congested queue
> length remains some fraction of full under stable load conditions, so
> a queuing discipline which is better than fill-and-drop should usually
> increase the RTT only by some fraction (substantially less than 2) under
> congestion if it is behaving properly.

RED helps, but only by a small (not an order of magnitude) factor.  
Prevalence of small flows makes RED less effective, too.  Also, I strongly
suspect that on egress side of the networks the combination of low duty
cycles on tail circuits (with heavy-tailed distribution laws) combined
with decreasing link capacity on links radiating from the backbone tends
to create multiple congestion points along the path even in purely
tree-like networks.  (Actually, in this case the number of congestion
points on egress path is limited by the number of link capacity grades.  
I.e. OC-12 -> OC-3 -> T-3 -> T-1).  You can see that phenomenon when a
large road radiates into a number of smaller roads with greater total
capacity, which in turn radiate further.

Policing can be made more TCP-friendly by emulating lower-bandwith
links with appropriate queues instead of just dropping extra packets, or
allowing bursts.  Some flavour of fair queueing with rate limits on
queues should do the job nicely.

End-to-end is rather imperfect thing, Internet is more so, yet there is
nothing better invented so far :)

--vadim

</delurking>