[e2e] was double blind, now reproduceable results
braden at ISI.EDU
Wed May 19 16:56:29 PDT 2004
*> I said nothing about non-determinism. I was referring to forcing science
*> into a situation where the only things you can study are those that are
*> perfectly reproducible (i.e. abstracted completely from their
*> environment). I was referring to phenomena that depend on large-scale
*> environmental characteristics.
So, let me try to refine the question. For a published result to be
good science, it should show some known dependence upon appropriate
measures of the relevant "large-scale environmental characterstics".
For example, a result might depend strongly upon the average link
bandwidth or the average node degree or some other, much more
complicated, statistical measure of properties of the "large-scale
environment". Call the environment measures X, which might be a single
number or a tuple.
Then you can ideally publish a result y as y = f(X), and you can say to
the reader, "You will be able to reproduce my results by constructing a
environment with measure X and then rerunning my experiment."
And of course people can explore different parts of the X space.
This is often hard to do, and I cannot deny that in many cases we are
mightily challenged to come up with an appropriate measure X. However,
I would suggest that this is the ideal towards which we should strive.
We should not just say, "It's too hard to do."
More information about the end2end-interest