[e2e] Simulator for wireless network

[Detlef Bosau]

U.Shanker wrote:
>>>
>>> Don't simulate, use real live setups.
>>
>> Unfortunately, that´s not always possible.
> Agreed
>>
>> In addition, there are always two ways to produce wrong results.
>>
>> The first is: Simulate. As you never know, what you´re simulating, 
>> your results will be wrong for sure.
> Interesting. But probably I just want effect of one or other 
> parameters on the whole system. In that case, simulation is the best.

O.k., I see I should add simleys if appropriate.

What I wanted to say is that there is no such thing as an ideal way and 
that either simulation or a real test bed is the only way to go in any case.


>> The second is: Conduct real experiments. As you never know, what 
>> you´re really doing, your results will be wrong for sure.
>>
>> So, the best way is, and somewhere I heard this were an old jiddish 
>> saying: If you have only two impossible choices, choose the third.
> What about taking some real network data and using simulations based 
> on that. We use this using Opnet modeler etc. I think this is one of 
> the best approach.

I never worked with the Opnet modeler. However, I would not believe that 
this is perfectly bug free. As the NS 2 is not perfectly bug free.
As a driver for some Lucent WaveLAN card provided for M$ Wista or 
Vindows or how it´s actually called could be expected to be perfectly 
bug free.

I even don´t know whether the source code for the Opnet modeler is 
available. The NS2 code is, and so I can see what I´m doing.

And BTW: I´m absolutely not convinced of actually availalbe RLP / mobile 
network simulators for the NS2, perhaps I don´t know all of them.
Of course, this is due to the questions I´m actually trying to find an 
answer to. So, I have to write my own simulator classes. Of course, it´s 
always a concern that no one will believe my results produced with this 
classes, this matches Craig´s experiences. The only thing I can do is to 
make the classes available to the public and to request comments on this 
code.


> I think, here you made very good point. I tried ns2 and spend most of 
> the time between understand the whole NS2 system and debugging between 
> C++ and otcl code.

And this is a necessary experience. Of course, there are other 
simulators. I had a look at the Omnet simulator and the BSD based 
classes for the Omnet provided by the TH Karlsruhe. Of course, this is 
interesting. However, it appears not be quite widespread and editing the 
BSD kernel code for own agents is at least somewhat cumbersome. So, I 
still use the NS2, because it´s simply easier for me.

However, in each case you have to understand what you´re doing. I often 
was asked: "What are you simulating there?" And that is a valid 
question. You must always know which parts of a network, which 
mechanisms and algorithms you are simulating, otherwise you won´t be 
able to understand the differences between a simulator and reality.

So, a simulator never can be used as a black box but you must know it´s 
internal architecture and limitations.

I know, often PhD students want to achieve their PhD in a short period 
of time and think it is sufficient to plan two years for understanding 
simulation itself, the NS2 in particular and do some research with it.

It is a matter of fact, that some papers were generated that way.

However, these papers _are_ that way.

Admittedly, I´m anything but a talented programmer. So, it took me at 
least about two years to reasonably understand what I´m doing with the 
NS2, particularly as there was no one I could learn from whom. The so 
called "NS2 expertise" of many people, even PhD students, it not even 
superficial - it simply doesn´t exist.

And it´s not only sufficient to understand how a simulator works. Some 
weeks ago I had a first glance at a PhD thesis on emulation which 
numbered several phenomena which must be simulated in the emulator. 
Wonderful. I remembered an older post by David Reed then, who told us 
that the typical number of phenomena
- transport
- routing
- serialziation
- perhaps somewhat MAC
is quite funny to look at - but it doesn´t reflect the situation in a 
real network. It´s at best some naive understanding on what is happening 
there.

So, that´s the reason why I emphasize the importance of proper 
questioning. You _MUST_ reduce complexity and focus yourself to a 
certain aspect of networking you want to study, otherwise you will get 
lost in all the details.

And there is a very important and interesting experience, I´m actually 
making myself.

At the moment, I´m trying to understand some aspects of RLP and 
proportional fair scheduling. Now, when I try to write an RLP simulator 
myself, I recognize all the gaps in my own understanding of RLP and I 
see all the degrees of freedom in such an implementation and the design 
alternatives you have. I would never see that when I would rely upon a 
"ready tool".

> Finally I moved to the OPNET (free for the universities !). It really 
> solved most of the my problems and I was just able to concentrate on 
                                                       
^^^^^^^^^^^^^^^^^^^^^ I´m not with an university, so I have to use tools 
which are freely available.
However, from what I´ve seen so far about OPNET, the source code is not 
completely available - and that´s not acceptable for me.
For some reasons, I needed to write an event dispatcher for a certain 
agent because I needed some coroutine model in my simulator.
Wonderful. Events in the NS2 are not polymorphic per se. It´s trivial to 
make them polymorphic. Simply add a sophisticed method:
...
virtual void make_it_polymorphic(){};
...

and anything is fine. However: How will I do that in a simulator, I have 
some object code of which and can plumb the one or the other class into 
the whole thing myself?

In the NS2, I simply added the line above - and now I can write

MyWonderfulMostIntelligentAgent::handler(Event *){
    if (dynamic_cast <selfmsg *> (e) {selfmsghandler(e);  return};
    if (dynamic_cast <linknotification *> (e) {linknotification(e); return};
    ....
    Agent::handle(e);
    return;
}

(I think, in the Omnet it´s done that way.)





> simulation only. But if somebody can master a tool like NS2, 
Who if not yourself shall master your simulator?

You will have to master your simulation scripts, your scenarios, your 
system model,  your implmentations, perhaps some day your thesis.

So, it´s a good point to start with: Master your simulator ;-)

> Finally, as Detlaf said,  it will really depend what exactly you want 
> to do. Want to see effects of some standard parameters within the 
> complex WLAN/UMTS/WIMAX environment, then OPNET could be very fast in 
> providing results. 
At least it provides columns with numbers.

My room is full of papers like that and I did not yet manage to carry 
them all to the waste basket.

I´m somwehat tired to read all these papers where people tuned some 
parameters which they don´t understand and take those funny numbers 
produced by the simulator as "results".

Let me take a concrete example. For my RLP simulator (I´m sorry, but I´m 
somewhat restricted to this topic these days ;-))
I have to decide whether the ARQ is done stop´n wait or sliding window. 
Now, there are some useful guidelines to that in Gorry Fairhursts RFC. 
And in Rami Mukhtars PhD thesis, it´s stop´n wait, IIRC. (Please don´t 
kill me, if I´m wrong ;-))
So, it´s about a few weeks now, where I simply consider whether ARQ 
should be done stop´n wait or sliding window.

Now, I´m implementing my simulator myself. So I must _make_ this 
decision. And if I ever get a paper published on this matter, I must 
_defend_ this decision.

I have no such excuse like "the OPNET did it that way" or "the Bell Labs 
did it that way". I made my decsion myself (in fact, I made one!) and I 
think I can defend it. And basically, there would be the alternative to 
study both ways and compare the results.

So, what I´m using from the NS2 is the event driven simulator framework 
and the routing mechanisms. The "intelligence" for a simulation is put 
in the agents. And these will be my own ones. Of course, the TCP agents 
are taken from the NS 2 package. But I think, these are sufficiently 
accepted and well proven.


> If want to do changed in the standard TCP implementations together 
> with WLAN etc, then may be NS2 will be faster. Looking for the complex 
> environment and some changes in the TCP, then you have to take the 
> time and probably use the OPNET.
>
> I was really very fast with NS2 and TCP and WLAN simulation. But when 
> it comes to the UMTS and WiMAX etc together with TCP/WLAN then it 
> appeared that, I have to move back from ns2 2.29 to 2.27, as UMTS 
> implementation was based on the older NS2 version and so on. 
Why did you rely upon old, perhaps unmanaged code? Those "contributions" 
which are written by some PhD student, and when he got his degree he 
forgets about his code and hopefully no one will ever detect the errors?

I know, what I´m writing here sounds extremely harsh and bitter.

But it reflects my own life experience and my own programming experience.

When I have to _rely_ on some simulation code, I have to know what I´m 
doing. And there are some extremely helpful tools to achieve this.
Your brain, your hands, the vi and the gcc.


> If you plan to spend next few years with this kind of simulations work 
> then probably its worth start using NS2/NS3.
>

I agree.

And if there were no "sentimental reasons" and I hade the time to start 
with something completley new, it would be probably worthwile to join 
the NS3 project. However, the main issue is to make perfectly clear what 
you´re asking for. And when the question is clear, as I said: a good 
question is nearly half the answer, you can choose the way to answer it, 
be it analysis or simulation and in case of simulation you will be able 
to write the appropriate code. (Existing code will hardly be sufficient 
because if there would be existing code for your question, your question 
might not be an open one.)

As I said. This whole thing sounds somewhat harsh and bitter. But I only 
share my own experience here. I don´t want to discourage anyone. 
However, simulation is no simple task which can be done quickly and 
easily. Doing good simulations require good standing and years
of concentrated and extremely focussed work.

Detlef

-- 
Detlef Bosau
Galileistrasse 30
70565 Stuttgart
Mail: detlef.bosau at web.de
Web: http://www.detlef-bosau.de
Mobile: +49 172 681 9937