> The _serious_ issue behind this anecdote is: Who has ever validated
> NS2 code?

Hang on a minute here.  I largely agree with Lloyd and Jon.  But, let's
backup for a moment.  Two notes:

  * The core ns2 developers have, in fact, built validation tests into
    the ns2 package.  For instance, I know that Sally Floyd has very
    carefully vetted the TCP code in ns2 and has designed validation
    tests for TCP.  Probably others have done such things with other
    portions of the code.  Does that make the code bug-free?  Of course
    not.  I don't think I have ever engaged in an ns2 exploration of
    some idea without finding some small bug in the ns2 code.  Does that
    mean that all of ns2 is validated?  No.  But, there *is* *some*
    careful validation being done in there.  Could it be better and more
    comprehensive?  I am sure.  But, let's give some credit in this
    space.

  * The ns web page says:

      Read this first:

      While we have considerable confidence in ns, ns is not a polished
      and finished product, but the result of an on-going effort of
      research and development. In particular, bugs in the software are
      still being discovered and corrected. Users of ns are responsible
      for verifying for themselves that their simulations are not
      invalidated by bugs. We are working to help the user with this by
      significantly expanding and automating the validation tests and
      demos.

      Similarly, users are responsible for verifying for themselves that
      their simulations are not invalidated because the model
      implemented in the simulator is not the model that they were
      expecting. The ongoing Ns Manual should help in this process.

    It's not claimed to be perfect.  It's a work-in-progress.  It's a
    power tool: it can be useful and it can cause big problems.

These are not meant to be offered as an excuse, or to say that the ns2
folk should not strive to do better.  Or, that another simulation effort
altogether could or should not be undertaken.  Not at all.  Clearly,
there is room for improvement in this area - as with anything.  However,
the observation that lousy researchers produce lousy research seems
independent of ns2 to me (who often finds himself reading equally lousy
measurement papers from people who have taken great pains to figure out
how to type "traceroute", "ping", "pathload" or whatever).

allman



-- 
Mark Allman -- ICIR/ICSI -- http://www.icir.org/mallman/