[e2e] opening multiple TCP connections getting popular
rbriscoe at jungle.bt.co.uk
Thu Aug 30 10:59:17 PDT 2007
At 16:57 30/08/2007, Joe Touch wrote:
>Lachlan Andrew wrote:
> > Greetings Joe,
> > On 30/08/2007, Joe Touch <touch at isi.edu> wrote:
> >> Lachlan Andrew wrote:
> >>> I thought that Bob's suggestion was simply that the transport/network
> >>> layer provide a mechanism for charging based on the congestion caused.
> >>> That sort of mechanism *is* "solvable at the transport layer".
> >> If we charge based on congestion caused, that's a network layer issue,
> >> not transport.
> > True, the charging is network layer, but letting the application know
> > how much it is being charged so that it can change its rate is
> > transport layer, isn't it?
>It's transport layer if you're charging the app based on what the app
>sends to the transport. Unfortunately, the network can be just as
>congested by a retransmission as a first try, so this is network
>pricing. Although the transport and application may _react_ to it, it is
>generated (and thus the problem exists) at the network layer.
Here's where the transport layer comes in: Policing. Even tho it's "in the
network", policing is a transport layer function.
We have proposed a bulk per-user policer based on measuring downstream
congestion volume (basically a big token bucket of downstream congestion
tokens for all flows at once). Despite being bulk across flows, this
policer incentivises each flow's end-point transport to take care to manage
congestion between themselves within the envelope of the bulk policer. But
just because the policer is bulk, doesn't mean it's not transport.
The subtle point here is that I haven't actually proposed charging for
congestion. Strictly I've proposed how to reveal a generally useful metric
at the network layer - downstream congestion volume. Certainly that /could/
be charged for, but that wasn't my motivation... at least not for end-users
(because there's loads of evidence users don't like unpredictable pricing
Instead, we wanted to reveal a metric in network layer headers that would
allow inline transport mechanism (policing) to be applied to it, not just
out of band (charging, penalties etc).
(Hence the need for a metric about the expectation of downstream congestion
so it could be policed at the ingress before the packet entered the
However, between networks I was motivated by congestion charging or any
similar use of the metric (e.g. incorporation in inter-domain SLAs with
penalties for exceeding thresholds etc). Strictly, that's application-layer
(in the management-plane if that's how you see the world) on top of a
> >> First, however, what does that mean?
> >> There are many issues there; it's not cut-and-dry.
> > Agreed. The question is whether the IETF should be working to address
> > those issues, or ignoring them.
>I've stated before - the IETF isn't the IRTF; some of this is research
>(i.e., what are the dimensions of the solution). Some of it is also a
>combination of policy and pricing - neither of which are IETF purvue either.
Of course I'm not expecting the IETF to do the policy, but I am expecting
it to provide the protocol that can have policy applied to it. That's very
much IETF land.
>The IETF is a good place to standardize a technical solution once we
>know what we're solving. At this point, "we need to do something" is
>insufficient rationale to proceed
"We need to do something" is a rather dismissive characterisation of a
fully spec'd protocol proposal:
I can't win either way. When I put re-ECN to the IETF, everyone didn't
understand why it was needed. Now I've explained why it's needed, people
complain the IETF doesn't get involved in economics and policy. I'm not
asking the IETF to get involved in economics and policy, I'm asking for a
bit in a header. The wider industry does the policy. But the IETF needs to
state the requirements it understands it is trying to meet.
>(and, as others have noted, it's not
>clear we _need_ to do anything).
Tell that to the CEO of any network operator. I think you're saying there's
not an engineering problem. But that's because resource allocation problems
are economic problems not engineering problems... However, this one has an
engineering solution. Of course, the network engineering continues to
_work_ with any particular sharing of resources, but that doesn't mean it's
'working' in an economic sense.
Consider this: folks at the IETF can expend weeks on whether a certain
sentence about the minutiae of a certain protocol field will be a MUST or a
SHOULD. But apparently we don't need to spend any time at all on fixing the
whole of resource allocation and accountability.
> >> A solution inside the transport layer solves only the most
> >> trivial variant, and leaves so many loose ends elsewhere it doesn't put
> >> a dent in the overall problem.
> > Can't the same be said for enforcing flow-rate fairness?
>Excellent point. You'll note that we do nothing to enforce that. The
>goal is that the network congestion feedback is proportional the number
>of streams - there's no rule that prevents opening multiple streams (as
>was noted) or merging them either. The whole thing works mostly because
>the degenerate case - a stream per packet - is prohibitive.
> > If that does
> > more than put a dent in the overall problem, then congestion charging
> > can make a bigger dent. If flow-rate fairness *doesn't* put more than
> > a dent in the overall problem, should we forget about "TCP
> > friendliness"?
>We don't enforce that either. We encourage that as a general principle,
>and assume (as above) that so long as the number of TCP streams is
>roughly proportional to the number of users, things will work fine.
Server farms would get a couple of flows? I think you might mean "roughly
proportional to the amount different users are paying for their access".
But then, why constrain ourselves so that usage has to be approx
proportional to access link capacity? It's much more economically efficient
to separate the two:
- access capacity is only about peak bit rate
- different users have extremely different demands for how much they use
>don't work if the number of flows grows with the load; if we get to a
>point where that actually happens, then maybe we'll need to do somthing.
Indeed, we have got to that point.
And what about if the duration of the numerous flows from single users are
also hundreds of times longer than those from others (who are paying the
Indeed, we have got to that point too.
See for instance any graph on distribution of usage between different
residential users. E.g. Figs 12 & 13 in Kenjiro Cho's paper on p2p usage in
Japan in SGICOMM'06.
But anyway, why is it that the IETF can happily expend time on volumes of
engineering trivia, but an economic efficiency problem is given a bar
hundreds of times higher before the IETF should stir itself into action? If
there was a 54% inefficiency in a protocol, the IETF would be in there
trying to fix it. But if it's 54% economic inefficiency it's not important?
>Joe Touch Sr. Network Engineer, USAF TSAT Space Segment
> Postel Center Director & Research Assoc. Prof., USC/ISI
Bob Briscoe, <bob.briscoe at bt.com> Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK. +44 1473 645196
More information about the end2end-interest