[e2e] Switched Ethernet is Not an End-to-End System; was Protocols breaking the end-to-end argument

[Detlef Bosau]

Hi Richard,

to be honest: I thought a bit about answering to this post. However, I 
well remember my first posts to this list some years ago and I well 
remember some posts of people, who where both, new to this list and new 
to distributed systems. So, people can learn from posts to this list, 
myself explicitly included.

So, I think it to be worthwhile to add some comments.

 
Richard Bennett wrote:
> Detlef, I'm asking you to think abstractly about certain problems in 
> the design of distributed systems. I regard many of

That's exactly what I'm doing. I simply does not matter whether a packed 
is conveyed via a shared Ethernet or via a switched Ethernet. From an 
abstract point of view, the packet is conveyed from a sender to a 
receiver. And this process takes some time and the "server", i.e. the 
conveying network, may process quite some packets in parallel. (I 
recently talked about TCP multipath issues to some colleagues, and I got 
the insight, perhaps many others got this long ago, that it simply does 
not matter, whether one packet is conveyed via a Cheapernet segment 
while a second one is conveyed via a fibre link and a third one via 
WiFi. There are three packets on the fly.

Now: The endpoints are not interested in the details of transport here - 
and in addition: there's no need for them to give it even a thought! 
(O.k., you will object that the service times on different media may be 
different. Granted. Leave it out for the moment.)

Do you see, what I'm doing here? I make an _abstraction_. I regard the 
net as some kind of transportation pipeline with some average (or 
expected) service time and a certain capacity. And from this point of 
view, I well may ignore the details.


> the interpretations and applications of end-to-end notions to network 
> management and regulation as flawed,

O.k., you're welcome to mention some concrete examples. But of course, 
when I mention a concrete flaw, I must not only say: "This is wrong, and 
I'm old enough and big enough and ugly enough to be correct here!" but I 
have to give a compelling reason, why something is wrong.

> regardless of the intent of any of the authors of any particular paper 
> decades ago, or even to their subsequent refinement by the original 
> authors or others.

In fact, I'm not interested in the intention but in the results.
>
> One the problems that arises in distributed systems is access to 
> shared resources, and the end-to-end solution to this problem is 
> deeply flawed. 

Why and where?


> In order to share resources according to a policy, there needs to be a 
> system element enforcing the policy. 

Exactly. (Certainly, you know Keshav's PhD dissertation.)

However, and I'm convinced Keshav will agree here: There are systems 
_without_ such a system. And there are systems, where the amount of 
resources is unknown. Think not only of any kinds of ad hoc systems and 
spontaneous collaboration.
Think of an arbitrary "Linux installation party" - where some PC are 
interconnected via a coax-"backbone".
Do you measure the coax backbone's length in advance to calculate its 
storage capacity for packets?
Certainly not.

You simply use the system, and - surprise! - it works!


> In network operations, this function is indispensable and must be part 
> of the network infrastructure. There are many

I've just given you and example, where this indispensable function can 
well be spared.

> reasons for this, of course, but the chief ones are a) the lack of 
> reliability and trust in end systems;

The less components a system will need, the more reliable a system will 
be. (This basic insight is often referet to  / abbreviated as KISS. Keep 
It Small and Simple.)
> and b) the lack of information about system demand on the part of the 
> end system, which in principle only knows that it wants  and not what 
> anyone else wants to do on the shared resource.
>
And particularly, an Ethernet switch in between has no knowledge about 
the end system's needs.


> The idea that access to shared resources is best accomplished by 
> uncoordinated end systems leads to a lot of grief.
>
One part of the grief is that you can read my answer ;-)

> What does this have to do with Jacobson's Algorithm? A lot, it turns 
> out, as we see in the rise of systems that take the job of congestion 
> mediation a couple of steps further, such as ECN and Re-ECN.

Which rise of system are you talking about? I definitely see a rise of 
papers which propose design rules for ECN, showing that we did not find 
compelling ones throughout the last decade.

However, without VJCC, this seminal discussion of ECN would not even be 
possible.


> If you recognize that information about the state of shared resources 
> is essential to their rational management, no problem.
>

I well recognize this. However, I recognize as well, that VJCC does its 
job. And the algorithm is shamelessly simple.

> Of course, all of this is simply to say that optimal solutions to 
> problems of resource management can only be made close to the resource 
> in question, 

Which is known from the works by Srinivasan Keshav and Frank Kelly and 
many others.

However, it's a simple fact in practical engineering, that we do not 
obtain optimal solutions for all problems, particularly the 
aforementioned works run into some difficulties when applied to wireless 
systems, but we obtain _practical_ solutions, we can well live with which.

Actually, we are that comfortable with these practical solutions, that 
the only purpose of some of the optimal ones is to protect the bald 
heads of PhD students from the cold by providing them warming hats.

And, of course, these practical solutions provide you with a punch-ball 
as well ;-)
> but it doesn't address a later version of the end-to-end arguments 
> principle, that sub-optimal solutions (from an efficiency of 
> fair-sharing perspective) to such problems may be preferred if they 
> lead to greater system generality,

Wow!

You're willing to accept sub-optimal solutions when they lead to greater 
system generality!

Welcome to the club!

I think, about all of us do.

> opportunities for innovation, free speech, reliability, or some other 
> reason.
>
> I hear this sort of argument being made very frequently these days, 
> and it does have some resonance. Efficiency is

At least, it shows some effect eventually...

> after all a short-term goal, while generality is a long-term goal and 
> innovation is a positive externality. But to appreciate that, one 
> needs to think a bit abstractly.
>

...which may mean that we should rather not bother with irrelevant 
problems of minor importance.

(@Joe: Hopefully, I was not too misbehaved here, but I really could not 
resist...)

Detlef

-- 
Detlef Bosau            Galileistraße 30        70565 Stuttgart
phone: +49 711 5208031  mobile: +49 172 6819937 skype: detlef.bosau     
ICQ: 566129673          detlef.bosau at web.de     http://www.detlef-bosau.de