From simon at limmat.switch.ch Thu Mar 1 07:09:42 2001 From: simon at limmat.switch.ch (Simon Leinen) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010228164710.G51394@ted.isi.edu> References: <13695.982276825@dstc.edu.au> <20010228164710.G51394@ted.isi.edu> Message-ID: >>>>> "tf" == Ted Faber writes: >> [...] >> TCP 532400385076 95.81 % >> UDP 21201575665 3.82 % > Bytes or packets? Does the other unit support your conclusion, too? Those were bytes. Here's a summary from the same transatlantic links (Eastbound direction only) which counts flows and packets too (new version of script appended): protocol.......flows..............packets...............bytes......... GRE 7071 ( 0.01 %) 268698 ( 0.02 %) 213346212 ( 0.04 %) ICMP 3473563 ( 6.09 %) 10420689 ( 0.94 %) 1083751656 ( 0.20 %) IGMP 4 ( 0.00 %) 8 ( 0.00 %) 7264 ( 0.00 %) IP 11604 ( 0.02 %) 3724884 ( 0.34 %) 763601220 ( 0.14 %) IPINIP 4716 ( 0.01 %) 14148 ( 0.00 %) 2589084 ( 0.00 %) TCP 35155287 (61.62 %) 942530269 (85.00 %) 532400385076 (95.81 %) UDP 18399711 (32.25 %) 151843881 (13.69 %) 21201575665 ( 3.82 %) So in terms of number of flows and packets, this particular transatlantic link does indeed have a higher share of UDP than what the 1998 "beast" paper observed. Wether this is a general trend, or just due to different usage patterns between our link and the links observed by CAIDA, I don't know. Actually I'd be glad if people could run the script on other routers which aggregate large numbers of users (especially non-academic users) and tell me whether the results are wildly different or wildly similar. For a comparison, here are the totals from an access router at a random university: protocol.......flows..............packets...............bytes......... GRE 383 ( 0.00 %) 17235 ( 0.00 %) 3602115 ( 0.00 %) ICMP 101931237 ( 1.75 %) 305793711 ( 0.45 %) 37918420164 ( 0.11 %) IGMP 34662 ( 0.00 %) 901212 ( 0.00 %) 58578780 ( 0.00 %) IP 1406788 ( 0.02 %) 15474668 ( 0.02 %) 3528224304 ( 0.01 %) IPINIP 1297 ( 0.00 %) 1297 ( 0.00 %) 583650 ( 0.00 %) TCP 4361852662 (74.91 %) 63919315234 (93.39 %) 32455859980970 (96.82 %) UDP 1357265629 (23.31 %) 4201556174 ( 6.14 %) 1025284993546 ( 3.06 %) This matches the CAIDA values much better. Maybe the transatlantic figures are biased because we run authoritative name servers for some ccTLDs - those can be expected to generate lots and lots of single-packet UDP port 53 flows. Note also that the "flow" concept used in the CAIDA work isn't exactly the same as Cisco NetFlow's, although the numbers may still be comparable. In case someone wants to know, we use the default flow timeout values (30 minutes maximum lifetime or 1 minute(??) maximum inactivity). -- Simon. #!/usr/local/bin/perl -w my (%tb,%tp,%tf); my ($tb,$tp,$tf) = (0,0,0); while (<>) { my ($proto,$f,$fps,$ppf,$bbp) = split; next unless defined $bbp && $proto =~ /^[A-Z]/ && $f =~ /^[0-9]+$/; $proto =~ s/-.*//; my $p = $f * $ppf; my $b = $p * $bbp; $tf{$proto} += $f, $tf += $f, $tb{$proto} += $b, $tb += $b, $tp{$proto} += $p, $tp += $p unless $proto eq 'Total:'; } printf "protocol.......flows..............packets...............bytes.........\n"; foreach (sort keys %tb) { printf "%-7s %8.0f (%5.2f %%) %9.0f (%5.2f %%) %13.0f (%5.2f %%)\n", $_, $tf{$_}, 100.0 * $tf{$_} / $tf, $tp{$_}, 100.0 * $tp{$_} / $tp, $tb{$_}, 100.0 * $tb{$_} / $tb; } 1; From simon at limmat.switch.ch Thu Mar 1 07:32:23 2001 From: simon at limmat.switch.ch (Simon Leinen) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <4591.983412132@dstc.edu.au> References: <4591.983412132@dstc.edu.au> Message-ID: >>>>> "gm" == George Michaelson writes: > Wow. Completely proved me wrong. Well... other networks may be different. For example, we used to charge the universities by volume (for transatlantic traffic we still do), so that certainly has some influence on our usage. >> 98 Also predates an explosion in IP-in-IP and other encapsulated >> flows (VPNs, IPSEC, PPPoE) so I'd be willing to hazard there are >> more fragmented flows than shown there. > And I look to be wrong on IP-in-IP as well. That may also be different for other ISPs. Our university users don't do much VPN (yet?). > It also predates the explosion of applications such as Napster and > Gnutella (which both run over TCP), whose traffic volume dwarfs > that of all UDP traffic (at least on our network). > The application mix that makes TCP predominate.. I didn't expect > that. I had assumed like FSP these things used UDP layering. You underestimate people's ability to learn from past mistakes (-: although TCP is used for concerns other than TCP-friendliness). > The UDP is going to be NTP and DNS? Re-running my script with a tiny change: protocol.............flows..............packets...............bytes......... GRE 7071 ( 0.01 %) 268698 ( 0.02 %) 213346212 ( 0.04 %) ICMP 3473563 ( 6.09 %) 10420689 ( 0.94 %) 1083751656 ( 0.20 %) IGMP 4 ( 0.00 %) 8 ( 0.00 %) 7264 ( 0.00 %) IP-other 11604 ( 0.02 %) 3724884 ( 0.34 %) 763601220 ( 0.14 %) IPINIP 4716 ( 0.01 %) 14148 ( 0.00 %) 2589084 ( 0.00 %) TCP-BGP 154665 ( 0.27 %) 154665 ( 0.01 %) 10053225 ( 0.00 %) TCP-FTP 1478600 ( 2.59 %) 7393000 ( 0.67 %) 2336188000 ( 0.42 %) TCP-FTPD 161850 ( 0.28 %) 69433650 ( 6.26 %) 49783927050 ( 8.96 %) TCP-Frag 285 ( 0.00 %) 3420 ( 0.00 %) 413820 ( 0.00 %) TCP-NNTP 70222 ( 0.12 %) 113338308 (10.22 %) 21307601904 ( 3.83 %) TCP-SMTP 968681 ( 1.70 %) 17436258 ( 1.57 %) 7689389778 ( 1.38 %) TCP-Telnet 75043 ( 0.13 %) 1876075 ( 0.17 %) 324560975 ( 0.06 %) TCP-WWW 24258155 (42.52 %) 315356015 (28.44 %) 235255587190 (42.34 %) TCP-X 6509 ( 0.01 %) 2512474 ( 0.23 %) 293959458 ( 0.05 %) TCP-other 7981277 (13.99 %) 415026404 (37.43 %) 215398703676 (38.76 %) UDP-DNS 8185178 (14.35 %) 16370356 ( 1.48 %) 2111775924 ( 0.38 %) UDP-Frag 444 ( 0.00 %) 1053168 ( 0.09 %) 767759472 ( 0.14 %) UDP-NTP 3676906 ( 6.44 %) 3676906 ( 0.33 %) 279444856 ( 0.05 %) UDP-TFTP 11 ( 0.00 %) 11 ( 0.00 %) 693 ( 0.00 %) UDP-other 6537172 (11.46 %) 130743440 (11.79 %) 18042594720 ( 3.25 %) So NTP is marginal in terms of traffic, DNS too (although not in terms of number of flows). The bulk of UDP *bytes* does in fact come from "UDP-other" - I can think of audio/video streaming and gaming, although the latter may be insignificant on a transatlantic link. > Are the ssh tunnels looking like TCP and so IPSEC/ip-in-ip doesn't > figure because grassroots, people use applications tunnels instead? Maybe. We definitely have customers who use IPSEC for VPN applications (probably showing up in the "IP-other" category), but I don't know whether they do this transatlantically, and our users may do this less than users of commercial networks(?). -- Simon. From floyd at aciri.org Fri Mar 2 09:12:11 2001 From: floyd at aciri.org (Sally Floyd) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <200103021712.f22HCB567380@elk.aciri.org> >So the 95% figure for TCP still looks reasonable in 2001, at least for >that particular link Many thanks for posting your results. I have added a pointer to them on a web page on "Measurement Studies of End-to-End Congestion Control in the Internet", at "http://www.aciri.org/floyd/ccmeasure.html", where we are trying to track information from measurement studies about how end-to-end congestion control is actually doing in the Internet. I would be particularly interested if anyone's measurements ever indicated a surge of non-congestion-controlled traffic in the Internet... - Sally -------------------------------- http://www.aciri.org/floyd/ -------------------------------- From kjc at csl.sony.co.jp Fri Mar 2 10:25:35 2001 From: kjc at csl.sony.co.jp (Kenjiro Cho) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <200103021712.f22HCB567380@elk.aciri.org> References: <200103021712.f22HCB567380@elk.aciri.org> Message-ID: <20010303032535C.kjc@csl.sony.co.jp> Sally Floyd wrote: > >So the 95% figure for TCP still looks reasonable in 2001, at least for > >that particular link > > Many thanks for posting your results. I have added a pointer to them > on a web page on "Measurement Studies of End-to-End Congestion Control > in the Internet", at "http://www.aciri.org/floyd/ccmeasure.html", > where we are trying to track information from measurement studies about > how end-to-end congestion control is actually doing in the Internet. We are maintaining trans-Pacific packet traces along with their summary info taken from the WIDE project backbone at http://tracer.csl.sony.co.jp/mawi/ (note that addresses are scrambled in tcpdump binary outputs.) > I would be particularly interested if anyone's measurements ever > indicated a surge of non-congestion-controlled traffic in the > Internet... Our data also confirms that TCP is still more than 90% of the traffic under normal situations. But, unfortunately, unusual traffic patterns do happen these days. For example, http://tracer.csl.sony.co.jp/mawi/samplepoint-A/2000/200006171359.html http://tracer.csl.sony.co.jp/mawi/samplepoint-A/2000/200006181359.html -Kenjiro From dpreed at reed.com Fri Mar 2 11:02:32 2001 From: dpreed at reed.com (David P. Reed) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <200103021712.f22HCB567380@elk.aciri.org> Message-ID: <5.0.2.1.2.20010302135526.03130a60@mail.reed.com> At 09:12 AM 3/2/01 -0800, Sally Floyd wrote: >I would be particularly interested if anyone's measurements ever >indicated a surge of non-congestion-controlled traffic in the >Internet... Good idea, but I'd caution people to observe that non-TCP traffic is still capable of congestion control. For example, one can do streaming media over UDP with congestion control - the same signals (lost packets, RED, and ECN) can be used to reflect congestion to the endpoints and implement a closed-loop adaptive solution (for video, lowering frame rate, and prioritizing audio, for example). So the actual detection and measurement of "non-congestion-controlled" traffic flows is an end-to-end issue. It isn't strictly observable at router, certainly not by just looking at protocol numbers. - David -------------------------------------------- WWW Page: http://www.reed.com/dpr.html From simon at limmat.switch.ch Fri Mar 2 12:48:03 2001 From: simon at limmat.switch.ch (Simon Leinen) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <5.0.2.1.2.20010302135526.03130a60@mail.reed.com> References: <5.0.2.1.2.20010302135526.03130a60@mail.reed.com> Message-ID: >>>>> "dpr" == David P Reed writes: > At 09:12 AM 3/2/01 -0800, Sally Floyd wrote: >> I would be particularly interested if anyone's measurements ever >> indicated a surge of non-congestion-controlled traffic in the >> Internet... > Good idea, but I'd caution people to observe that non-TCP traffic is > still capable of congestion control. For example, one can do > streaming media over UDP with congestion control - the same signals > (lost packets, RED, and ECN) can be used to reflect congestion to > the endpoints and implement a closed-loop adaptive solution (for > video, lowering frame rate, and prioritizing audio, for example). ...or giving up out of frustration, or getting kicked out of a game. The thing that comes closest to incapable of congestion control is probably DNS (except zone transfers). But in terms of bytes, DNS makes up only ~0.3% of all traffic around here (even though we have a couple of ccTLD servers on our network). Unfortunately I cannot look at the "UDP-other" traffic (~90% of UDP traffic or 2.7% of all bytes) very well. I'd venture a guess that most of this is RealMedia/QuickTime/Windows Media Player. Those should use fairly well-defined congestion control mechanisms. Is there any work on characterizing these kinds of transport protocols with respect to their levels of "TCP-friendliness"? > So the actual detection and measurement of "non-congestion-controlled" > traffic flows is an end-to-end issue. It isn't strictly observable at > router, certainly not by just looking at protocol numbers. Absolutely, -- Simon. From floyd at aciri.org Fri Mar 2 21:01:11 2001 From: floyd at aciri.org (Sally Floyd) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <200103030501.f2351B573554@elk.aciri.org> >I'd venture a guess that >most of this is RealMedia/QuickTime/Windows Media Player. The packet traces from the MAWI Working Group Traffic Archive at "http://tracer.csl.sony.co.jp/mawi/" break down the udp traffic into dns, rip, realaud, halflif, everque, quake, and other. E.g., "http://tracer.csl.sony.co.jp/mawi/samplepoint-B/2001/200102251400.html". For the days that I looked, the UDP traffic on this transoceanic link was dominated by DNS, actually. But maybe transoceanic links have different traffic mixes than other ones. >Those >should use fairly well-defined congestion control mechanisms. Is >there any work on characterizing these kinds of transport protocols >with respect to their levels of "TCP-friendliness"? We have just started to look at this. In addition to thinking some about the potential fit of equation-based congestion control (e.g., TFRC) for these kinds of traffic. It turns out that the deployment of ECN in the Internet would add a new interest to some of these issues. - Sally -------------------------------- http://www.aciri.org/floyd/ -------------------------------- From ehall at ehsco.com Fri Mar 2 22:07:58 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <200103030501.f2351B573554@elk.aciri.org> Message-ID: <3AA08A3E.541D233@ehsco.com> > For the days that I looked, the UDP traffic on this transoceanic > link was dominated by DNS, actually. But maybe transoceanic links > have different traffic mixes than other ones. People don't play action-oriented multi-player games over long-haul networks. Shoot-em-up games are very sensitive to latency and packet loss. Playing a shoot-em-up with >200ms RTT will get you killed fast by players with <20ms (client-side events have to wait for server-side messages to arrive so the "closer" player gets a distinct advantage in terms of shorter inter-command gap). After a while, you learn to play on servers that are close. Anyway, trans-oceanic links are radically different in that regard. They will always have lower gaming levels. FWIW, network games are fascinating examples of interactive applications. They are the new TELNET, except that they also have range issues that TELNET didn't often encounter (similar to the annoying remote echo problem but on a larger scale). Also, not all of these games use UDP. Many of them are using TCP for a variety of familiar reasons. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From akyol at pluris.com Sun Mar 4 00:22:24 2001 From: akyol at pluris.com (Bora Akyol) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <200103030501.f2351B573554@elk.aciri.org> Message-ID: I would expect to see lots of content caching/distribution going on in the transoceanic links such that multimedia traffic probably always gets served from the nearest content server. If that is the case, how is the content getting replicated to these different continents? Do the traffic statistics over the transoceanic links capture this replication or are these being beamed over satellite? Thanks Bora On Fri, 2 Mar 2001, Sally Floyd wrote: > >I'd venture a guess that > >most of this is RealMedia/QuickTime/Windows Media Player. > > The packet traces from the MAWI Working Group Traffic Archive at > "http://tracer.csl.sony.co.jp/mawi/" break down the udp traffic > into dns, rip, realaud, halflif, everque, quake, and other. E.g., > "http://tracer.csl.sony.co.jp/mawi/samplepoint-B/2001/200102251400.html". > For the days that I looked, the UDP traffic on this transoceanic > link was dominated by DNS, actually. But maybe transoceanic links > have different traffic mixes than other ones. > > >Those > >should use fairly well-defined congestion control mechanisms. Is > >there any work on characterizing these kinds of transport protocols > >with respect to their levels of "TCP-friendliness"? > > We have just started to look at this. In addition to thinking some > about the potential fit of equation-based congestion control (e.g., > TFRC) for these kinds of traffic. It turns out that the deployment > of ECN in the Internet would add a new interest to some of these issues. > > - Sally > -------------------------------- > http://www.aciri.org/floyd/ > -------------------------------- > From smd at ebone.net Sun Mar 4 02:55:34 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010304105534.B8DE58A3@sean.ebone.net> | I would expect to see lots of content caching/distribution going on in the | transoceanic links such that multimedia traffic probably always gets | served from the nearest content server. I wouldn't. | If that is the case, how is the content getting replicated to these | different continents? Do the traffic statistics over the transoceanic | links capture this replication or are these being beamed over satellite? Some people (Yahoo, CNN, etc.) locate "european-flavour" servers in Europe, "cantonese-style" servers in Hong Kong, and so forth. Some people use Akamai and their competitors, which seem to be locating stuff in various places around the world. Most content just comes from wherever it happens to be hosted, and often enough that's somewhere in California. Works great. Sean. From jstevenson at orblynx.com Sun Mar 4 06:49:55 2001 From: jstevenson at orblynx.com (John Stevenson) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010304105534.B8DE58A3@sean.ebone.net> Message-ID: <3AA25613.5678C303@orblynx.com> << Most content just comes from wherever it happens to be hosted, and often enough that's somewhere in California. Works great. >> Not so if the client is in Indonesia, or eastern Europe, or Egypt, etc., a long or an indirect link over any big cable. And recently not quite so even in California (when the resulting brownouts kick in). John Stevenson Sean Doran wrote: > | I would expect to see lots of content caching/distribution going on in the > | transoceanic links such that multimedia traffic probably always gets > | served from the nearest content server. > > I wouldn't. > > | If that is the case, how is the content getting replicated to these > | different continents? Do the traffic statistics over the transoceanic > | links capture this replication or are these being beamed over satellite? > > Some people (Yahoo, CNN, etc.) locate "european-flavour" servers in Europe, > "cantonese-style" servers in Hong Kong, and so forth. Some people use > Akamai and their competitors, which seem to be locating stuff in various > places around the world. Most content just comes from wherever it happens > to be hosted, and often enough that's somewhere in California. Works great. > > Sean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20010304/69472035/attachment.html From cerpa at ISI.EDU Sun Mar 4 21:13:02 2001 From: cerpa at ISI.EDU (Alberto Cerpa) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010304105534.B8DE58A3@sean.ebone.net> Message-ID: On Sun, 4 Mar 2001, Sean Doran wrote: > | If that is the case, how is the content getting replicated to these > | different continents? Do the traffic statistics over the transoceanic > | links capture this replication or are these being beamed over satellite? > > Some people (Yahoo, CNN, etc.) locate "european-flavour" servers in Europe, > "cantonese-style" servers in Hong Kong, and so forth. Some people use > Akamai and their competitors, which seem to be locating stuff in various > places around the world. Most content just comes from wherever it happens > to be hosted, and often enough that's somewhere in California. Works great. > Do you have some measurements to back this up? I would be really interested to get any pointers to some data available confirming this. Best regards, -Al > Sean. > From T.Henderson at cs.ucl.ac.uk Mon Mar 5 07:29:58 2001 From: T.Henderson at cs.ucl.ac.uk (Tristan Henderson) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Message from "Eric A. Hall" of "Fri, 02 Mar 2001 22:07:58 PST." <3AA08A3E.541D233@ehsco.com> Message-ID: <20010305153003.9C49337D3F@kylie.cs.ucl.ac.uk> In message <3AA08A3E.541D233@ehsco.com>, "Eric A. Hall" said: > >People don't play action-oriented multi-player games over long-haul >networks. Shoot-em-up games are very sensitive to latency and packet loss. >Playing a shoot-em-up with >200ms RTT will get you killed fast by players >with <20ms (client-side events have to wait for server-side messages to >arrive so the "closer" player gets a distinct advantage in terms of >shorter inter-command gap). After a while, you learn to play on servers >that are close. > Do you have any data/stats to support these figures? I'm doing some analysis of shoot-em-up games and haven't been able to find anything authoritative about the maximum delays for networked games. I've seen figures of ~200ms being declared as the "maximum" delay before; e.g. a games designer says that they design for 200-300ms delays at http://www.gamasutra.com/features/19970905/ ng_01.htm. OTOH, there are plenty of usenet postings from people playing with RTTs of 300-1000ms, e.g. http://groups.google.com/groups?hl=en&lr=&safe=off&ic=1&th=1daccce21a879875 http://groups.google.com/groups?hl=en&lr=&safe=off&ic=1&th=2cd5a305b3152d89 http://groups.google.com/groups?hl=en&lr=&safe=off&ic=1&th=41803c558ae2df07 (apologies if these google links don't work; I haven't quite got used to their usenet archive yet) It would be useful to know the absolute highest delays that gamers can tolerate. > >FWIW, network games are fascinating examples of interactive applications. I agree. I'm particularly interested in the multiuser aspects - for example, as you state, there are dynamics which may force users with similar network characteristics to congregate together. Alas, games seem to have been neglected by the networking research community, but hopefully that is changing. Cheers, Tristan From smd at ebone.net Mon Mar 5 08:10:48 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: (Alberto Cerpa's message of "Sun, 4 Mar 2001 21:13:02 -0800 (PST)") References: Message-ID: <52d7bwgprr.fsf@sean.ebone.net> Alberto Cerpa writes: > Do you have some measurements to back this up? I would be really > interested to get any pointers to some data available confirming this. What type of measurements would you be looking for? My assertion of "works great" is based on some measurements of RTT (absolute and variability) and loss various networks here and there use to construct their SLAs. In the little network in which I play, all approach the minimum most of the time, even in outlying places like Bratislava, Budapest, Bucharest, Prague and so on, that are in the "Eastern Europe" someone suggested was badly connected to the world. Yes, TCP's ACK clocking means things farther away from each other are at a disadvantage, but realistically, it *is* only ~190ms from Romania to Southern California (which seems like a reasonable selection of worst case, topologically speaking). I am open to the argument that these are atypically good numbers, and that performance to Eastern Europe (for example) tends to be much worse. In fact, I would love to get such an argument into the hands of our sales people. :-) :-) Sean. - -- robuc101-ta#trace Protocol [ip]: Target IP address: cs.ucsd.edu Source address: 213.174.64.13 Numeric display [n]: Timeout in seconds [3]: Probe count [3]: 10 Minimum Time to Live [1]: Maximum Time to Live [30]: Port Number [33434]: Loose, Strict, Record, Timestamp, Verbose[none]: Type escape sequence to abort. Tracing the route to cs.ucsd.edu (132.239.51.18) 1 atvie103-ta-s1-0.ebone.net (213.174.70.81) 20 msec 20 msec 12 msec 16 msec 12 msec 24 msec 16 msec 12 msec 16 msec 12 msec 2 atvie101-tc-r6-0.ebone.net (195.158.245.49) 16 msec 20 msec 12 msec 16 msec 20 msec 20 msec 12 msec 16 msec 12 msec 16 msec 3 czpra103-tc-p2-0.ebone.net (195.158.242.45) 20 msec 28 msec 28 msec 28 msec 20 msec 24 msec 24 msec 24 msec 20 msec 20 msec 4 debln302-tc-p2-0.ebone.net (213.174.70.45) 32 msec 32 msec 28 msec 36 msec 32 msec 32 msec 36 msec 40 msec 32 msec 28 msec 5 debln301-tc-p1-0.ebone.net (213.174.70.37) 32 msec 40 msec 28 msec 36 msec 36 msec 36 msec 32 msec 32 msec 32 msec 32 msec 6 dedus206-tc-p6-0.ebone.net (213.174.70.41) 36 msec 44 msec 36 msec 40 msec 40 msec 36 msec 40 msec 40 msec 36 msec 36 msec 7 dedus205-tc-p7-0.ebone.net (213.174.70.125) 40 msec 40 msec 40 msec 36 msec 36 msec 40 msec 40 msec 44 msec 48 msec 40 msec 8 nlams303-tc-p2-0.ebone.net (213.174.70.134) 40 msec 40 msec 40 msec 40 msec 44 msec 40 msec 40 msec 40 msec 40 msec 44 msec 9 bebru203-tc-p1-0.ebone.net (213.174.71.1) 44 msec 44 msec 44 msec 44 msec 44 msec 44 msec 44 msec 44 msec 40 msec 44 msec 10 bebru204-tc-p2-0.ebone.net (195.158.225.82) 40 msec 44 msec 44 msec 44 msec 40 msec 48 msec 40 msec 48 msec 44 msec 48 msec 11 gblon505-tc-p1-0.ebone.net (195.158.232.41) 48 msec 48 msec 48 msec 48 msec 52 msec 52 msec 48 msec 48 msec 48 msec 48 msec 12 usnyk105-tc-p1-1.ebone.net (195.158.229.25) 116 msec 120 msec 116 msec 116 msec 120 msec 120 msec 116 msec 120 msec 120 msec 116 msec 13 sl-bb11-nyc-5-3.sprintlink.net (144.232.9.229) [AS 1239] 116 msec 120 msec 116 msec 116 msec 120 msec 120 msec 116 msec 120 msec 124 msec 116 msec 14 144.232.9.202 [AS 1239] 140 msec 116 msec 120 msec 116 msec 120 msec 116 msec 116 msec 120 msec 116 msec 116 msec 15 pos3-0-622M.nyc-bb8.cerf.net (134.24.33.158) [AS 1740] 120 msec 120 msec 116 msec 120 msec 116 msec 120 msec 116 msec 116 msec 116 msec 120 msec 16 so6-3-0-622M.chi-bb5.cerf.net (134.24.32.213) [AS 1740] 136 msec 136 msec 136 msec 136 msec 136 msec 140 msec 136 msec 140 msec 132 msec 148 msec 17 pos2-0-622M.chi-bb3.cerf.net (134.24.33.197) [AS 1740] 136 msec 140 msec 136 msec 140 msec 136 msec 140 msec 140 msec 136 msec 140 msec 136 msec 18 pos0-0-622M.sfo-bb4.cerf.net (134.24.46.58) [AS 1740] 192 msec 192 msec 188 msec 192 msec 192 msec 188 msec 188 msec 188 msec 188 msec 192 msec 19 pos7-0-622M.sfo-bb3.cerf.net (134.24.32.78) [AS 1740] 196 msec 196 msec 196 msec 196 msec 196 msec 192 msec 196 msec 196 msec 200 msec 196 msec 20 pos3-0-622M.lax-bb4.cerf.net (134.24.29.234) [AS 1740] 192 msec 188 msec 192 msec 192 msec 188 msec 192 msec 188 msec 192 msec 192 msec 192 msec 21 so1-0-0-622M.lax-bb7.cerf.net (134.24.33.170) [AS 1740] 188 msec 192 msec 188 msec 192 msec 192 msec 192 msec 188 msec 192 msec 192 msec 188 msec 22 so-6-0-0.san-bb4.cerf.net (134.24.29.13) [AS 1740] 196 msec 192 msec 196 msec 196 msec 192 msec 196 msec 196 msec 196 msec 196 msec 196 msec 23 pos1-0-0-155M.san-bb1.cerf.net (134.24.29.190) [AS 1740] 196 msec 196 msec 196 msec 196 msec 204 msec 196 msec 196 msec 192 msec 444 msec * 24 sdsc-gw.san-bb1.cerf.net (134.24.12.26) [AS 1740] 204 msec 204 msec 200 msec 200 msec 204 msec 208 msec 200 msec 204 msec 208 msec 200 msec 25 bigmama.ucsd.edu (192.12.207.5) [AS 195] 220 msec 220 msec 228 msec 220 msec 292 msec 236 msec 244 msec 248 msec 232 msec 224 msec 26 cse-rs.ucsd.edu (132.239.254.45) [AS 7377] 224 msec 224 msec 244 msec 224 msec 224 msec 228 msec 228 msec 228 msec 224 msec 224 msec 27 cs.ucsd.edu (132.239.51.18) [AS 7377] 216 msec * 212 msec * 212 msec * 212 msec * 208 msec * From smd at ebone.net Mon Mar 5 08:25:41 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:32 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010305162541.5E74F8A3@sean.ebone.net> Mmmm, socio-psychology meets networking. Always fun, never understood fully. :) | It would be useful to know the absolute highest delays that gamers can | tolerate. Surely this will be somewhat application-dependent? However, there's probably some literature here and there about human reflexes and how fast one needs a result back from a "twitch" in order to feel reasonably interactive. Probably very little of that will focus on network impact. | >FWIW, network games are fascinating examples of interactive applications. They're also fun. I've never been big into shoot-em-up games, since building the Internet is faster and harder, but some friends had me over to play Unreal Tournament with their clan the other week, and my eyes were opened a bit. UT in any event was more sensitive to loss and "drop outs" than to stable delay -- for me, anyway, choppy updates and missed action was more important and harder to compensate for than aiming ahead along the direction the target is seen to be moving. | I agree. I'm particularly interested in the multiuser aspects - for example, | as you state, there are dynamics which may force users with similar network | characteristics to congregate together. It turns out that LAN parties are pretty common: people drive across Europe to gather together around a hub or small switch, matching up as teams in a series of competitions within a broader league. The social aspect, it turns out, is as important as the locality. By analogy, although a good SLA can be gotten from a high-quality Chinese Restaurant's delivery service, 15 people eating the same stuff and communicating via a telephone bridge or IRC or whatever is not as much fun as the same 15 people together in the restuarant, even if the food is no better prepared or presented, and arrives at the table no more quickly. In the Internet space, we all know that there is a significant value in the social aspect of IETF meetings, despite the formalization of the mailing lists as being the places where real work happens. | Alas, games seem to have been neglected by the networking | research community, but hopefully that is changing. Heh - well, they're sure popular among operators, at least those on the operations front, as far as I can tell. Perhaps that is a reflection of a dichotomy between people who are reactive & practical versus people who like to plan in advance and understand the theory behind things. Sean. From T.Henderson at cs.ucl.ac.uk Mon Mar 5 09:54:38 2001 From: T.Henderson at cs.ucl.ac.uk (Tristan Henderson) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Message from smd@ebone.net (Sean Doran) of "Mon, 05 Mar 2001 17:25:41 +0100." <20010305162541.5E74F8A3@sean.ebone.net> Message-ID: <20010305175443.BF70537D3F@kylie.cs.ucl.ac.uk> In message <20010305162541.5E74F8A3@sean.ebone.net>, Sean Doran said: >Mmmm, socio-psychology meets networking. Always fun, never understood fully. >:) > >| It would be useful to know the absolute highest delays that gamers can >| tolerate. > >Surely this will be somewhat application-dependent? > Yes, you'd expect (within networked games) that delay requirements would look like shoot-em-up < RPG < chess. It should be possible, however, to come up with some general figures, a G.114 equivalent for shoot-em-ups. I'd just like something more concrete than figures pulled out of a hat, so if anyone knows of any (reasonably) scientific studies please point me at them. >However, there's probably some literature here and there about >human reflexes and how fast one needs a result back from a "twitch" >in order to feel reasonably interactive. Probably very little >of that will focus on network impact. > Precisely. There is stuff in the VR and physiology worlds about reflexes, but it's not clear that this applies to the Internet, where people seem to put up with a lot more than they'll admit to in a lab experiment. >| >FWIW, network games are fascinating examples of interactive applications. > >They're also fun. I've never been big into shoot-em-up games, >since building the Internet is faster and harder, but some friends >had me over to play Unreal Tournament with their clan the other week, >and my eyes were opened a bit. UT in any event was more sensitive >to loss and "drop outs" than to stable delay -- for me, anyway, choppy >updates and missed action was more important and harder to compensate >for than aiming ahead along the direction the target is seen to be moving. > Interesting. I've been concentrating on Half-Life mainly (it seems to be the most widely-played game according to tracking sites such as http://www.theclq.com/games.asp) but I might have to give UT a go as well. >| I agree. I'm particularly interested in the multiuser aspects - for example, > >| as you state, there are dynamics which may force users with similar network >| characteristics to congregate together. > >It turns out that LAN parties are pretty common: people drive across >Europe to gather together around a hub or small switch, matching up >as teams in a series of competitions within a broader league. > But this isn't always an option for geographically dispersed groups, so a lot of games server operators allow clans to book servers for private games. That's why I'd quite like to determine the QoS requirements for applications such as these; games players are already spending lots of money on their habit, so they should be quite receptive to paying for QoS. >| Alas, games seem to have been neglected by the networking >| research community, but hopefully that is changing. > >Heh - well, they're sure popular among operators, at least those >on the operations front, as far as I can tell. Perhaps that is >a reflection of a dichotomy between people who are reactive & practical >versus people who like to plan in advance and understand the theory >behind things. > No comment :) Cheers, Tristan From ehall at ehsco.com Mon Mar 5 10:02:38 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305153003.9C49337D3F@kylie.cs.ucl.ac.uk> Message-ID: <3AA3D4BE.212F190E@ehsco.com> > Do you have any data/stats to support these figures? Word-of-mouth, casual research. IE, asking the guy that killed me what his ping is. Also, lots of player forums (newsgroups, message boards, etc). I would agree that 200ms RTT seems to be about the max for combat. > design for 200-300ms delays at http://www.gamasutra.com/features/19970905/ > ng_01.htm. Interesting read. Thanks. > OTOH, there are plenty of usenet postings from people playing with RTTs > of 300-1000ms, e.g. Well, not all of them are telling the truth. I'm not sure I'd believe the boastings of nine-year olds in public forums. But there is a lot of skill involved. There are people with 5ms RTT that can't win no matter what, and there are people with 300ms RTT that win all of the time. Another issue here is that not all of the games are shooters. UO in particular has a lot of social elements, and it doesn't require any combat at all. A lot of the high-ping players naturally gravitate more towards the role-playing or social elements instead of combat, particularly after getting their clock cleaned consistently by low-pingers. I'm not saying low RTTs are not important, I am saying that there are games which embrace high-RTT players by offering non-combat activities, and this will likely become more important over time. > Alas, games seem to have been neglected by the networking research > community, but hopefully that is changing. It has gone both ways. Developers of new Internet-specific apps are not coming here, either. But I agree that there is a growing separation between the current Internet and the research community in general. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From ehall at ehsco.com Mon Mar 5 10:14:55 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305162541.5E74F8A3@sean.ebone.net> Message-ID: <3AA3D79E.B3BBCD7A@ehsco.com> > However, there's probably some literature here and there about > human reflexes and how fast one needs a result back from a "twitch" > in order to feel reasonably interactive. Probably very little > of that will focus on network impact. I'm sure that has something to do with it but I don't think it's the principle issue. I mean, it might be the primary factor when everybody is on the same LAN, but when you're talking about cross-country RTTs it's not the primary issue. Command queueing is the problem. Longer RTTs mean larger gaps between commands. This works both ways, in that movement and actions sent from the client take longer to reach the server, but data coming from the server is also rapidly becoming outdated by the time it reaches the client. This puts high RTTs at a distinct disadvantage to low RTTs, regardless of the player's twitch reflex capabalities. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From jms at central.cis.upenn.edu Mon Mar 5 10:35:29 2001 From: jms at central.cis.upenn.edu (Jonathan M. Smith) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Your message of "Mon, 05 Mar 2001 17:47:25 GMT." Message-ID: <200103051835.f25IZTj27538@central.cis.upenn.edu> There's a classic book which has a bunch of experiments on timing - very high quality stuff. The authors are Card, Newell and Simon, and it's called the "Psychology of Computer Human Interaction". I looked at it when I was trying to understand how much queueing delays and jitter of other types "mattered". -JMS From T.Henderson at cs.ucl.ac.uk Mon Mar 5 11:39:22 2001 From: T.Henderson at cs.ucl.ac.uk (Tristan Henderson) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Message from Lloyd Wood of "Mon, 05 Mar 2001 19:17:32 GMT." Message-ID: <20010305193927.CFD4B37D3F@kylie.cs.ucl.ac.uk> In message , Ll oyd Wood said: >On Mon, 5 Mar 2001, Eric A. Hall wrote: > >> But there is a lot of skill involved. There are people with 5ms RTT that >> can't win no matter what, and there are people with 300ms RTT that win all >> of the time. > >hacking your copy of the game for e.g. shooting accuracy has nothing >to do with it. (apropos: there's a rant on security of multiplayer >games under >http://tuxedo.org/~esr/writings/quake-cheats.html >) Deliberately compensating for lag in the game client in some >similar way would be interesting. Apparently the more delay-tolerant RPGs, e.g. Age of Empires and Warcraft, already do some compensation - they deliberately delay all interactions so that all players have similar delay. Not sure about shoot-em-ups though. Cheers, Tristan From touch at ISI.EDU Mon Mar 5 11:47:13 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305153003.9C49337D3F@kylie.cs.ucl.ac.uk> Message-ID: <3AA3ED41.E50A300F@isi.edu> Tristan Henderson wrote: > > In message <3AA08A3E.541D233@ehsco.com>, "Eric A. Hall" said: > > > >People don't play action-oriented multi-player games over long-haul > >networks. Shoot-em-up games are very sensitive to latency and packet loss. > >Playing a shoot-em-up with >200ms RTT will get you killed fast by players > >with <20ms (client-side events have to wait for server-side messages to > >arrive so the "closer" player gets a distinct advantage in terms of > >shorter inter-command gap). After a while, you learn to play on servers > >that are close. As Sean indicated, these are application dependent. More precisely, they depend on the level of predicatability in the feedback system, and how high in the human the processing occurs. The most basic human feedback loops (single flashing light, hit a switch) are in the 100 ms range. That means the network portion must be in the 20ms range to be 'noise' on the overall system delay. However, it gets longer for things like "multiple lights, hit the switch only if one of the lights is red". The response delay gets larger the more complicated the task. E.g., ask someone for a review of War and Peace, and you're liable to be willing to wait a few days. :-) It's all about expectations. Figures in the 100-200ms range are for maximum auditory delay for telephone echos, and date back to the early Bell Labs days. > OTOH, there are plenty of usenet postings from people playing with RTTs of > 300-1000ms, e.g. Many old video games had artificial delays incorporated (e.g., sluggishness in the controls of space invaders, etc). Part of the 'game' is getting acclimated to those delays. > It would be useful to know the absolute highest delays that gamers can > tolerate. People play chess by mail. It's more about expectations than about the inherent delay of the system. ---------------------- Regarding latency papers, there is Stuart Cheshire's from 1996, as well as more recent notes from David Reed. My dissertation was on this stuff, and examined the fundamental limits of latency in communication (pub'd 1992, links on my home page). Joe http://www.isi.edu/touch From ehall at ehsco.com Mon Mar 5 11:48:42 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305193927.CFD4B37D3F@kylie.cs.ucl.ac.uk> Message-ID: <3AA3ED9A.F68F1723@ehsco.com> > Apparently the more delay-tolerant RPGs, e.g. Age of Empires and > Warcraft, already do some compensation - they deliberately delay all > interactions so that all players have similar delay. Not sure about > shoot-em-ups though. Some of the shooters are using turn-based play in order to put more of an emphasis on tactics and less on connection. There are still cheat problems of course, but when *everything* goes through a scheduler on the server it really changes the nature of the game. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From ehall at ehsco.com Mon Mar 5 12:06:27 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: Message-ID: <3AA3F1C2.F69E2B94@ehsco.com> > Deliberately compensating for lag in the game client in some > similar way would be interesting. This was [re]introduced as a fairly big problem just a few weeks back, with client-based clock emulators being used to eliminate programmed delays. EG, for those actions which required client-side delay (casting a spell, healing, shooting a paced weapon, etc.), emulating the clock (and running the emulator at very high rates) meant that client-side delays were essentially removed from the game. It has been around is some form or another for a while. UO has long suffered from a "fast walk" hack that allowed players to move at their own pace instead of a rate set by the server. This was eventually fixed with rotating random keys and encrypted commands. The clock hack essentially made these fixes irrelevant, and allowed for many more cheats. Closing the loop, what this is driving is a return to closed communities. Diablo saw it badly (nobody with any sense played on the public servers when the clients had a built-in "God mode"), others are recognizing the problem and are now designing for it. I think it is trending away from massive multiplayer towards multiple-islands. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From demir at usc.edu Mon Mar 5 12:12:58 2001 From: demir at usc.edu (demir) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <3AA3ED41.E50A300F@isi.edu> Message-ID: > As Sean indicated, these are application dependent. > More precisely, they depend on the level of > predicatability in the feedback system, and how > high in the human the processing occurs. I am, completely, agree with above lines. I interpret this as "engineering human perception/action" where "communication" is also part of this. As "perception/action" will differ in our real life, so should differ in applications. As Joe stated, it is all about "expectations", I think, too. Alper K. Demir From ehall at ehsco.com Mon Mar 5 12:30:08 2001 From: ehall at ehsco.com (Eric A. Hall) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305153003.9C49337D3F@kylie.cs.ucl.ac.uk> <3AA3ED41.E50A300F@isi.edu> Message-ID: <3AA3F74F.C2724ACE@ehsco.com> > > >arrive so the "closer" player gets a distinct advantage in terms of > > >shorter inter-command gap). > As Sean indicated, these are application dependent. > The most basic human feedback loops (single flashing light, > hit a switch) are in the 100 ms range. That means the > network portion must be in the 20ms range to be 'noise' > on the overall system delay. However, it gets longer Not all functions fall in that category. Strafing is holding down a key while turning, for example, not click-click-click. Running/motion is holding down a key. Etc. Whenever a task involves interactive exchange of packets which are not driven by user interaction, then the player with the lower latency gets a distinct advantage. There are also tasks which are user-automated. For example, a user may have practiced a particular sequence of events, and may have developed a timing patter such that they can execute events without waiting for feedback from the system. Rather than "hit switch when light flashes" it becomes "hit switch every 5ms because that's how often the light flashes" which is fundamentally different, and this model also rewards players who have low RTTs vs high RTTs. The best Player-vs-Player fighters are trained monkeys with well-honed reactionary pathways which allow them to react to macros that fail. -- Eric A. Hall http://www.ehsco.com/ Internet Core Protocols http://www.oreilly.com/catalog/coreprot/ From touch at ISI.EDU Mon Mar 5 13:00:44 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <20010305153003.9C49337D3F@kylie.cs.ucl.ac.uk> <3AA3ED41.E50A300F@isi.edu> <3AA3F74F.C2724ACE@ehsco.com> Message-ID: <3AA3FE7C.E4C63F99@isi.edu> "Eric A. Hall" wrote: > > > > >arrive so the "closer" player gets a distinct advantage in terms of > > > >shorter inter-command gap). > > > As Sean indicated, these are application dependent. > > > The most basic human feedback loops (single flashing light, > > hit a switch) are in the 100 ms range. That means the > > network portion must be in the 20ms range to be 'noise' > > on the overall system delay. However, it gets longer > > Not all functions fall in that category. Strafing is holding down a key > while turning, for example, not click-click-click. Running/motion is > holding down a key. Etc. Whenever a task involves interactive exchange of > packets which are not driven by user interaction, then the player with the > lower latency gets a distinct advantage. Strafing needs high packet rate, but is latency independent. A better implementation would just send "start strafe" and "end strafe" signals anyway. > There are also tasks which are user-automated. For example, a user may > have practiced a particular sequence of events, and may have developed a > timing patter such that they can execute events without waiting for > feedback from the system. Rather than "hit switch when light flashes" it > becomes "hit switch every 5ms because that's how often the light flashes" > which is fundamentally different, and this model also rewards players who > have low RTTs vs high RTTs. Any such timing pattern should be uploadable. If you're forcing the user to input the sequence manually, it's just like the forced delays of the old Space Invaders days. > The best Player-vs-Player fighters are trained monkeys with well-honed > reactionary pathways which allow them to react to macros that fail. Right - all you really need to adjust is the non-predicatable part. Joe From J.Crowcroft at cs.ucl.ac.uk Mon Mar 5 14:17:11 2001 From: J.Crowcroft at cs.ucl.ac.uk (Jon Crowcroft) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Your message of "Mon, 05 Mar 2001 13:00:44 PST." <3AA3FE7C.E4C63F99@isi.edu> Message-ID: <11047.983830631@cs.ucl.ac.uk> what tristan asked for was _evidence_ you've all turned anecdotal or prescriptive since the evidne about relative TCP and UDP traffic fact is theres few facts about any of this, jusdt lots of opinion. go look at the original bell labs papers on interative audio RTTs : that was just opinion too - when we get to games (pace, Cheriton) same applies in spades j. From touch at ISI.EDU Mon Mar 5 14:51:03 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <11047.983830631@cs.ucl.ac.uk> Message-ID: <3AA41857.9F074B30@isi.edu> Jon Crowcroft wrote: > > what tristan asked for was _evidence_ There's abundant evidence in the human factors community. It's just statistical, like everything involving 'bags of mostly water,' and it's highly domain-specific, because the level of comprehensive and predictive complexity is hard to provide quantitative measures for. Joe From smd at ebone.net Mon Mar 5 15:01:02 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:33 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010305230102.1BAF88A3@sean.ebone.net> | fact is theres few facts about any of this, jusdt lots of opinion. Well, yeah, but Jon, given that the Internet is heterogeneous, anisotropic, expanding, and mutating, it is really hard to be anything but anecdotal, since even the most comprehensive data set (one that defeats the observer problem (i.e., in the absence of isotropism, how do we know what things look like "over there"?)) will quickly grow stale. | go look at the original bell labs papers on interative audio RTTs : | that was just opinion too - when we get to games (pace, Cheriton) same | applies in spades Are you arguing on the question of whether opnion can be "good enough", or on the question of whether something much more strong than opinion or localized (in space and time) measurements can be obtained with an affordable amount of effort? Sean. [in a long-ago CIDRD wg meeting when they were contentious] smd: well, that's just my opinion voice in crowd (tli? postel?): and we're ALL entitled to Sean's opinion From demir at usc.edu Mon Mar 5 15:31:38 2001 From: demir at usc.edu (demir) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010305230102.1BAF88A3@sean.ebone.net> Message-ID: I agree with below lines. However, these are all "chaotic", to me. I think, the main challenge is how could we "engineer" these anisotropic, expanding, and mutating world as engineered as possible so that, may be, "the Turing test" is achieved in the current state. As Joe Touch indicated, "the levels of comprehensive and comlexity is hard to provide quantitative measures". I think, searching for "evidence" requires to solve the "relativity" problem as a human factor. I assume these are all "phylosophical" issues that one might think unimportant. I think an "enhanced architecture" should consider all these and other related factors. Again, thse are all about "expectations". Alper K. Demir > | fact is theres few facts about any of this, jusdt lots of opinion. > > Well, yeah, but Jon, given that the Internet is heterogeneous, > anisotropic, expanding, and mutating, it is really hard to be > anything but anecdotal, since even the most comprehensive data > set (one that defeats the observer problem (i.e., in the absence > of isotropism, how do we know what things look like "over there"?)) > will quickly grow stale. > > | go look at the original bell labs papers on interative audio RTTs : > | that was just opinion too - when we get to games (pace, Cheriton) same > | applies in spades > > Are you arguing on the question of whether opnion can be "good enough", > or on the question of whether something much more strong than opinion > or localized (in space and time) measurements can be obtained with > an affordable amount of effort? > > Sean. > > [in a long-ago CIDRD wg meeting when they were contentious] > smd: well, that's just my opinion > voice in crowd (tli? postel?): and we're ALL entitled to Sean's opinion > From foo at eek.org Mon Mar 5 17:42:52 2001 From: foo at eek.org (foo) Date: Thu Mar 25 11:59:34 2004 Subject: [e2e] TEAR. Message-ID: <20010305194252.L33489@eek.org> Does anyone have any experience with or thoughts about TEAR (TCP Emulation at Receivers) developed by Injong Rhee at NCSU? http://www.csc.ncsu.edu/faculty/rhee/export/tear_page/ -Brian From J.Crowcroft at cs.ucl.ac.uk Tue Mar 6 00:00:35 2001 From: J.Crowcroft at cs.ucl.ac.uk (Jon Crowcroft) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Your message of "Tue, 06 Mar 2001 00:01:02 +0100." <20010305230102.1BAF88A3@sean.ebone.net> Message-ID: <3377.983865635@cs.ucl.ac.uk> In message <20010305230102.1BAF88A3@sean.ebone.net>, Sean Doran typed: >>| fact is theres few facts about any of this, jusdt lots of opinion. >>Well, yeah, but Jon, given that the Internet is heterogeneous, >>anisotropic, expanding, and mutating, it is really hard to be >>anything but anecdotal, since even the most comprehensive data >>set (one that defeats the observer problem (i.e., in the absence >>of isotropism, how do we know what things look like "over there"?)) >>will quickly grow stale. a few of us actually try to do some measurements in the real world - before we do to many, we thoguht we would see if some other people had some - there is a LOT on web, a LOT on voice over IP now, and a lot of it is done over a fairly well characterised set of IP paths globally, despite what you say about the heterogeneity - sorry, but the fac t is that when it comes to games, there isnt, as far as we can tell, but we thought we;d ask. >>| go look at the original bell labs papers on interative audio RTTs : >>| that was just opinion too - when we get to games (pace, Cheriton) same >>| applies in spades >>Are you arguing on the question of whether opnion can be "good enough", >>or on the question of whether something much more strong than opinion >>or localized (in space and time) measurements can be obtained with >>an affordable amount of effort? look at vern's work on characterising end to end paths, look at sculzrinne, and bolot's workl on charcartiering delay jitter and its effect on voice, look at abundent work on zipf law and not for web page download size/time, etc etc etc where is the _equivalent_ _experimental_ data for games, please? the point is that a lot of early work in this area (50s,60s, itu standards definitions for toll quality speech) was based on LAB experiments, often with small, culture specific samples. a LOT of recent internet measurement work is based on real world data, which is NOT magic, not impossible (its hard work, and has to be incremental, painstaking, and very careful, but there is a lot) - we just wanted to see where the work had got to in one more part of the space..... >>[in a long-ago CIDRD wg meeting when they were contentious] >>smd: well, that's just my opinion >>voice in crowd (tli? postel?): and we're ALL entitled to Sean's opinion thanks, given that sean's comments are heterogeneous, anisotropic and expanding and mutating, i guess we are. cheers jon From craig at aland.bbn.com Tue Mar 6 05:28:32 2001 From: craig at aland.bbn.com (Craig Partridge) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Your message of "Mon, 05 Mar 2001 22:17:11 GMT." <11047.983830631@cs.ucl.ac.uk> Message-ID: <200103061328.IAA19448@aland.bbn.com> In message <11047.983830631@cs.ucl.ac.uk>, Jon Crowcroft writes: >go look at the original bell labs papers on interative audio RTTs : >that was just opinion too Hi Jon: In defense of the Bell Labs folks. There were some badly done studies in the 1950s and 1960s [not all at Bell Labs if I recall] -- many of which had the property that the people doing the studies didn't understand echo cancellers, with the result that, *surprise surprise*, they all reported that interactive voice couldn't be sustained with a delay of more than something like 100ms (exact number no longer remembered) which was the point at which the lack of echo cancellation started to be a problem. But there were three very good studies, all out of Bell Labs, which did the tests properly. They're still worth reading: Riesz and Klemmer in Bell System Technical Journal of Nov 1963, Klemmer in Bell System Technical Journal of July-August 1967, and P.T. Brady's article in the Bell System Technical Journal of January 1971. Incidentally, Klemmer sent me a note in the early 1990s says that, in retrospect, his studies didn't account for learned delay sensitivity -- that is, a delay which previously was acceptable will become annoying if you've become used to a much shorter delay. Side note: the BSTJ studies often used the following test procedure: * every time you picked up your phone handset, a delay was randomly chosen from a pool of possible delay times * on the phone was a button that you could press if you were unhappy with the audio quality, and I think you were rewarded by having the delay eliminated Something like this might work for gaming (though we'd have to get the incentives right -- if pressing the button eliminates the delay, everyone will do it all the time) Craig From smd at ebone.net Tue Mar 6 06:04:00 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010306140400.EFF438A3@sean.ebone.net> | Something like this might work for gaming (though we'd have to get the | incentives right -- if pressing the button eliminates the delay, everyone | will do it all the time) If pressing the button debits the presser's account a couple of euros in our favour, I would be more than happy to see this supported as quickly as possible in my network. Sean. P.S.: Peter Lothberg & I like to argue how QoS stuff can be done in various interrelated networks which generally offer only "platinum" quality service (zero average queue length, zero drops most of the time) can offer lower-quality (gold, silver, bronze, lead, barbed wire) at more market-competitive prices when there is demand. The idea is to put up a web page where the customer can dial a flavour, such as moving a slide-bar between 0-100% drop probability, and one that lengthens and thickens the tail of one-way delays at the interface facing the customer. While turning the knobs, one would also see the list price change -- worse quality -> lower pricing. Prese here when satisfied. (We also like the idea of having pre-build "profiles". Click here for network XYZ's observed level of service, price is 5% less than network XYZ's list, that kind of thing). There is apparently a decades-long history of doing this in the X.25 world. From karir at wam.umd.edu Tue Mar 6 07:38:31 2001 From: karir at wam.umd.edu (Manish Karir) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010306140400.EFF438A3@sean.ebone.net> Message-ID: On Tue, 6 Mar 2001, Sean Doran wrote: > > P.S.: Peter Lothberg & I like to argue how QoS stuff can be done in > various interrelated networks which generally offer only "platinum" > quality service (zero average queue length, zero drops most of the time) > can offer lower-quality (gold, silver, bronze, lead, barbed wire) at > more market-competitive prices when there is demand. The idea is to > put up a web page where the customer can dial a flavour, such as moving > a slide-bar between 0-100% drop probability, and one that lengthens > and thickens the tail of one-way delays at the interface facing the > customer. While turning the knobs, one would also see the list > price change -- worse quality -> lower pricing. Prese here when satisfied. > > (We also like the idea of having pre-build "profiles". Click here for > network XYZ's observed level of service, price is 5% less than network > XYZ's list, that kind of thing). > > There is apparently a decades-long history of doing this in the X.25 world. > I think something similar to this was done under the INDEX project at berkeley, the paper is at: http://www.path.berkeley.edu/~varaiya/papers_ps.dir/networkpaper.pdf though the web site for the project iteself does'nt seem to exist anymore or has moved... manish karir From kuang at sask.trlabs.ca Sat Mar 3 15:55:01 2001 From: kuang at sask.trlabs.ca (Tianbo kuang) Date: Thu Mar 25 11:59:34 2004 Subject: [e2e] TEAR. In-Reply-To: <20010305194252.L33489@eek.org> Message-ID: Hi, I was just about to ask a question about TEAR. It seems unclear to me how TEAR calculates RTT in the technical report (TEAR: TCP emulation at receivers - flow control for multimedia streaming). Does the sender calculate it and send it to the receiver, or does the receiver calculate it (and how?)? In section 3.5, it does mention under the title _timeout_ that, "this information is embedded in the packet header by the sender". What does "this information" refer to? Cheers, --Tianbo ------------------------------------------------------ Kuang Tianbo TRlabs 111-116 Research Drive Saskatoon, Saskatchewan S7N 3R3 Tel: (306) 668-9325(office) (306) 343-9747 (home) kuang@sask.trlabs.ca ------------------------------------------------------ On Mon, 5 Mar 2001, foo wrote: > Date: Mon, 5 Mar 2001 19:42:52 -0600 > From: foo > To: end2end-interest@ISI.EDU, tcp-impl@lerc.nasa.gov > Subject: [e2e] TEAR. > Resent-Date: Mon, 5 Mar 2001 19:48:05 -0600 > Resent-From: foo@eek.org > Resent-To: end2end-interest@postel.org > > Does anyone have any experience with or thoughts about TEAR (TCP Emulation > at Receivers) developed by Injong Rhee at NCSU? > > http://www.csc.ncsu.edu/faculty/rhee/export/tear_page/ > > -Brian > From rhee at eos.ncsu.edu Tue Mar 6 09:01:50 2001 From: rhee at eos.ncsu.edu (Injong Rhee) Date: Thu Mar 25 11:59:34 2004 Subject: [e2e] TEAR. In-Reply-To: Message-ID: Hi, I can't help overhearing, and want to drop a few lines. The RTT calculation can be done by the sender through receiving the receiver report about the rate. The receiver sends back the time stamp of the last packet received to the sender with the sequence number which will be used by the sender to compute the RTT. This is one way to do it, and there are many other ways. For instance, you can use the same way that RTP does or use GPS to synchronize the clocks and measure the one-way time. Then use the one-way trip time in place of RTT -- I know in this case that TCP-friendliness may suffer, but it can at least give some bounded fairness. In fact this removes back-channel concerns completely from flow control scuh as losses and delays in the back channels. Some of nice things about TEAR are that (1) it does not use back channels much so sutiable for wireless comm; (2) rate control is very smooth; (3) TCP-friendly over various ranges of bandwidth --- TFRC has some prblems under very low bandwidth cases. We have improved TEAR quite bit from the initial work and TEAR is incorporated into an MPEG-4 stream player and stream server, and it seems to give pretty good performance over other existing streaming solutions. Other areas of exploration are multicast and wireless communication. Sorry I have not kept you guys up to date about the progress. I got tired of writing papers and have been digging into writing codes.....Maybe its time to come out and see the light :-) Injong > -----Original Message----- > From: end2end-interest-admin@postel.org > [mailto:end2end-interest-admin@postel.org]On Behalf Of Tianbo kuang > Sent: Saturday, March 03, 2001 6:55 PM > To: foo > Cc: end2end-interest@ISI.EDU; tcp-impl@lerc.nasa.gov; > end2end-interest@postel.org > Subject: Re: [e2e] TEAR. > > > Hi, > > I was just about to ask a question about TEAR. It seems unclear to me how > TEAR calculates RTT in the technical report (TEAR: TCP emulation at > receivers - flow control for multimedia streaming). Does the sender > calculate it and send it to the receiver, or does the receiver calculate > it (and how?)? In section 3.5, it does mention under the title _timeout_ > that, "this information is embedded in the packet header by the sender". > What does "this information" refer to? > > Cheers, > > --Tianbo > > ------------------------------------------------------ > Kuang Tianbo > TRlabs > 111-116 Research Drive > Saskatoon, Saskatchewan S7N 3R3 > Tel: (306) 668-9325(office) (306) 343-9747 (home) > > kuang@sask.trlabs.ca > ------------------------------------------------------ > On Mon, 5 Mar 2001, foo wrote: > > > Date: Mon, 5 Mar 2001 19:42:52 -0600 > > From: foo > > To: end2end-interest@ISI.EDU, tcp-impl@lerc.nasa.gov > > Subject: [e2e] TEAR. > > Resent-Date: Mon, 5 Mar 2001 19:48:05 -0600 > > Resent-From: foo@eek.org > > Resent-To: end2end-interest@postel.org > > > > Does anyone have any experience with or thoughts about TEAR > (TCP Emulation > > at Receivers) developed by Injong Rhee at NCSU? > > > > http://www.csc.ncsu.edu/faculty/rhee/export/tear_page/ > > > > -Brian > > > > From touch at ISI.EDU Tue Mar 6 11:35:19 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] References: <200103061328.IAA19448@aland.bbn.com> Message-ID: <3AA53BF7.665AC944@isi.edu> Craig Partridge wrote: > > Side note: the BSTJ studies often used the following test procedure: > > * every time you picked up your phone handset, a delay was randomly > chosen from a pool of possible delay times > * on the phone was a button that you could press if you were unhappy > with the audio quality, and I think you were rewarded by having > the delay eliminated > > Something like this might work for gaming (though we'd have to get the > incentives right -- if pressing the button eliminates the delay, everyone > will do it all the time) A variant of that is that when you press the button, the delay goes down, but so does the bandwidth. There's your disincentive. Joe From J.Crowcroft at cs.ucl.ac.uk Wed Mar 7 01:24:26 2001 From: J.Crowcroft at cs.ucl.ac.uk (Jon Crowcroft) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Your message of "Tue, 06 Mar 2001 08:00:35 GMT." <3377.983865635@cs.ucl.ac.uk> Message-ID: <7570.983957066@cs.ucl.ac.uk> interesting data- http://www.jisc-tau.ac.uk/linx-access.html has a nice graph of latency improving as local access speed increases and matches the in/out capacity better, but http://www.jisc-tau.ac.uk/usa-access.html shows how it aint that simple and as latent demand tracks supply, long haul latency goes up again...roughly speaking... j. From dpreed at reed.com Wed Mar 7 05:12:04 2001 From: dpreed at reed.com (David P. Reed) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <7570.983957066@cs.ucl.ac.uk> References: Message-ID: <5.0.2.1.2.20010307080716.024df030@mail.reed.com> If carriers at all points got "paid" based on average latency, the investment would be there to move latency to a better attractor, which would track latent demand. This is something I've been trying to get started for a long time. The movement to pay carriers based on traffic volume, rather than delay experienced, will always drive the system to its worst case latency. we need a closed loop congestion control that works in the time-scale of fiber deployment and LAN-speed upgrades. We don't have one that does this, and no one (other than me) seems to be even seriously thinking about it. I've even done something about it by advising some of the bandwidth exchanges. At 09:24 AM 3/7/01 +0000, Jon Crowcroft wrote: >interesting data- >http://www.jisc-tau.ac.uk/linx-access.html >has a nice graph of latency improving as local access speed increases >and matches the in/out capacity better, but >http://www.jisc-tau.ac.uk/usa-access.html >shows how it aint that simple and as latent demand tracks supply, long >haul latency goes up again...roughly speaking... > >j. - David -------------------------------------------- WWW Page: http://www.reed.com/dpr.html From david_zhang at ins.com Wed Mar 7 06:41:19 2001 From: david_zhang at ins.com (david_zhang@ins.com) Date: Thu Mar 25 11:59:34 2004 Subject: [e2e] (no subject) Message-ID: <00fc01c0a714$ac23b660$df59a4d0@C991473C> unsubscribe -------------- next part -------------- An HTML attachment was scrubbed... URL: http://www.postel.org/pipermail/end2end-interest/attachments/20010307/b000bc50/attachment.html From smd at ebone.net Wed Mar 7 08:46:42 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010307164642.B1BCB8A3@sean.ebone.net> [some graphs from www.jisc-tau.ac.uk] well, yes, one expects to move bottlenecks around from time to time, i don't suppose there's any chance of using RED on the US->Europe side to limit the delay, and doing a comparison to actual utilization? that would be very cool (but understandably may not be possible) Sean. From smd at ebone.net Wed Mar 7 08:55:34 2001 From: smd at ebone.net (Sean Doran) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <20010307165534.C7AA28A3@sean.ebone.net> | If carriers at all points got "paid" based on average latency, the | investment would be there to move latency to a better attractor, which | would track latent demand. This is something I've been trying to get | started for a long time. The movement to pay carriers based on traffic | volume, rather than delay experienced, will always drive the system to its | worst case latency. It's not going to be cheaper to have an empty network than to have one with a bottleneck here and there. It's also not like there are that many applications that are so inelastic that latency is worth paying real extra $ to remove, when volume-over-time figures are "good enough" to make the per-available-mbps or 95th-percentile-utilization charges worth it, and there is no obvious killer app that is unamenable to adapting to the Internet's "rough approximation" of good performance, on the grounds that it's cheaper to do that than to do fancy QoS everywhere. | we need a closed loop congestion control that works in the time-scale of | fiber deployment and LAN-speed upgrades. We don't have one that does this, Well, so convince people it's cheaper than what we have now, without eliminating (much) utility. Start with explaining what it takes to have a bounded queueing delay at every potential or real bottleneck. Sean. From djw1005 at cam.ac.uk Wed Mar 7 15:54:57 2001 From: djw1005 at cam.ac.uk (Damon Wischik) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010307165534.C7AA28A3@sean.ebone.net> Message-ID: On Wed, 7 Mar 2001, Sean Doran wrote: > It's not going to be cheaper to have an empty network than to have one > with a bottleneck here and there. It's also not like there are that > many applications that are so inelastic that latency is worth paying > real extra $ to remove, when volume-over-time figures are "good enough" > to make the per-available-mbps or 95th-percentile-utilization charges > worth it > ... Might it be that by reducing latency one can improve the performance even of elastic traffic? TCP, for example, controls its rate using a feedback loop: the lower the latency, the tighter the control loop. There are results to suggest that tighter control loops will improve the stability of the network; this could be worth paying for. For references, see http://www.statslab.cam.ac.uk/~frank/int/ particularly "Stability of distributed congestion control with heterogeneous feedback delays" (L.Massoulie) "End-to-end congestion control for the Internet: delays and stability" (R.Johari and D.Tan.) Damon Wischik. From vijay at umbc.edu Thu Mar 8 08:41:08 2001 From: vijay at umbc.edu (Vijay Gill) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: Message-ID: On Sun, 4 Mar 2001, Alberto Cerpa wrote: [snip snip] Some data regarding flows from the Pacific Rim to the US. Keep in mind that the sampling rate used was fairly low. period: 03/04/2001 15:55:18 - 03/05/2001 15:55:19 PST Protocol Pkts Pkts/sec Bytes Bits/sec -------- ------------- ------------- ------------- ------------- tcp 4566586 52 1118739284 103585 icmp 97183 1 59403325 5500 udp 207550 2 33848925 3134 esp 2019 0 1216156 112 skip 673 0 337732 31 gre 298 0 94225 8 ipip 93 0 49844 4 ospf 108 0 33204 3 ipv6 14 0 1327 0 rsvp 5 0 580 0 the rsvp is an anomaly. /vijay From P.Gevros at cs.ucl.ac.uk Thu Mar 8 09:17:49 2001 From: P.Gevros at cs.ucl.ac.uk (Panos GEVROS) Date: Thu Mar 25 11:59:34 2004 Subject: [e2e] Re: UDP vs. TCP distribution In-Reply-To: Your message of "Thu, 08 Mar 2001 11:41:08 EST." Message-ID: <1602.984071869@cs.ucl.ac.uk> it is true that providers care about the exchanged volume - and that packet statistics are much easier to gather compared to flow statistics - still "flow" statistics would be very interesting (e.g active flows per T-sec, for some defintion of flow) does anyone know whether such data have been published anywhere cheers, Panos Vijay Gill writes: |On Sun, 4 Mar 2001, Alberto Cerpa wrote: | | [snip snip] | | |Some data regarding flows from the Pacific Rim to the US. Keep in mind |that the sampling rate used was fairly low. | |period: 03/04/2001 15:55:18 - 03/05/2001 15:55:19 PST |Protocol Pkts Pkts/sec Bytes Bits/sec |-------- ------------- ------------- ------------- ------------- | tcp 4566586 52 1118739284 103585 | icmp 97183 1 59403325 5500 | udp 207550 2 33848925 3134 | esp 2019 0 1216156 112 | skip 673 0 337732 31 | gre 298 0 94225 8 | ipip 93 0 49844 4 | ospf 108 0 33204 3 | ipv6 14 0 1327 0 | rsvp 5 0 580 0 | |the rsvp is an anomaly. | |/vijay From braden at ISI.EDU Thu Mar 8 09:36:51 2001 From: braden at ISI.EDU (Bob Braden) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] Message-ID: <200103081736.RAA11946@gra.isi.edu> *> ipv6 14 0 1327 0 *> rsvp 5 0 580 0 *> *> the rsvp is an anomaly. *> *> /vijay *> *> *> *> *> *> "anomaly"? What does that mean? If you mean statistically, it would appear that IPv6 is also an anomaly. Bob Braden From vijay at umbc.edu Thu Mar 8 09:39:16 2001 From: vijay at umbc.edu (Vijay Gill) Date: Thu Mar 25 11:59:34 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <200103081736.RAA11946@gra.isi.edu> Message-ID: On Thu, 8 Mar 2001, Bob Braden wrote: > > *> ipv6 14 0 1327 0 > *> rsvp 5 0 580 0 > *> > *> the rsvp is an anomaly. > > "anomaly"? What does that mean? If you mean statistically, it would > appear that IPv6 is also an anomaly. Apologies for being cryptic. RSVP was a misconfiguration somewhere; once fixed, it did not come back. There are some people who feverently wish that v6 could also be fixed and not come back, so I'm watching the links with an eagle eye. /vijay From dpreed at reed.com Thu Mar 8 09:07:53 2001 From: dpreed at reed.com (David P. Reed) Date: Thu Mar 25 11:59:35 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback be generated...] In-Reply-To: <20010307165534.C7AA28A3@sean.ebone.net> Message-ID: <5.0.2.1.2.20010308120144.02fa35f0@mail.reed.com> At 05:55 PM 3/7/01 +0100, Sean Doran wrote: >It's not going to be cheaper to have an empty network than to have >one with a bottleneck here and there. This presumes that customers want the lowest price regardless of delay. Not true. And in any case, operating a network with queues mostly full (which increases utilization) a lot of the time is great strategy if you want to maximize profit when you are being paid by the byte (or by the portal access rate). Most applications benefit from low queueing delay, so this isn't about QoS differentiations. Only FTPs with no human in the loop want capacity with no delay constraint. From touch at ISI.EDU Thu Mar 8 11:20:45 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:35 2004 Subject: UDP vs. TCP distribution [was: Re: [e2e] Can feedback begenerated...] References: <5.0.2.1.2.20010308120144.02fa35f0@mail.reed.com> Message-ID: <3AA7DB8D.B0A9C8E7@isi.edu> "David P. Reed" wrote: > > At 05:55 PM 3/7/01 +0100, Sean Doran wrote: > >It's not going to be cheaper to have an empty network than to have > >one with a bottleneck here and there. > > This presumes that customers want the lowest price regardless of > delay. Not true. And in any case, operating a network with queues mostly > full (which increases utilization) a lot of the time is great strategy if > you want to maximize profit when you are being paid by the byte (or by the > portal access rate). > > Most applications benefit from low queueing delay, so this isn't about QoS > differentiations. Only FTPs with no human in the loop want capacity with > no delay constraint. NTP wants it too. Capacity can also be used to mask latency, PROVIDED the variability in the feedback loop can be described (if not predicted). E.g., even FTPs with people in the loop work - you send the whole directory when the person does a "cd" (see Infocom 1995). (both the above cases are related; the variability is derived from the application, rather than the network). Further, many interactive systems are more sensitive to variability in the latency itself than in the latency value (e.g., NTP, it turns out). Joe From P.Gevros at cs.ucl.ac.uk Thu Mar 8 13:04:13 2001 From: P.Gevros at cs.ucl.ac.uk (Panos GEVROS) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] Re: UDP vs. TCP distribution In-Reply-To: Your message of "Thu, 08 Mar 2001 11:20:45 PST." <3AA7DB8D.B0A9C8E7@isi.edu> Message-ID: <1805.984085453@cs.ucl.ac.uk> Joe Touch writes: | | |"David P. Reed" wrote: |> |> At 05:55 PM 3/7/01 +0100, Sean Doran wrote: |> >It's not going to be cheaper to have an empty network than to have |> >one with a bottleneck here and there. |> |> This presumes that customers want the lowest price regardless of |> delay. Not true. And in any case, operating a network with queues mostly |> full (which increases utilization) a lot of the time is great strategy if |> you want to maximize profit when you are being paid by the byte (or by the |> portal access rate). |> |> Most applications benefit from low queueing delay, so this isn't about QoS |> differentiations. Only FTPs with no human in the loop want capacity with |> no delay constraint. | |NTP wants it too. | |Capacity can also be used to mask latency, PROVIDED the variability |in the feedback loop can be described (if not predicted). E.g., even |FTPs with people in the loop work - you send the whole directory |when the person does a "cd" (see Infocom 1995). low delay (or jitter) is good but whether this should be the network design goal is a different matter; if capacity was not a constrain i would be willing to tolerate an extra delay to download the whole "structure" (or a specific subset) of a web site and do all the searching/browsing locally, also store it for future reference in case i find it interesting enough, and save subsequent network accesses the fact that only a fraction of the information downloaded would be of interest may be irrelevant (because there are no capacity constrains) of course this does not apply to using the web for transactions or dynamic content, so ftp-style transport may not be completely out of fashion yet, it all depends on whether need for interactive experience or access to information proves to be the killer app - but if bounded delay is a necessity and customers are prepared to pay for it then we may be better of with something where one directly "dials the web server" Panos From touch at ISI.EDU Thu Mar 8 14:01:15 2001 From: touch at ISI.EDU (Joe Touch) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] Re: UDP vs. TCP distribution References: <1805.984085453@cs.ucl.ac.uk> Message-ID: <3AA8012B.FC1BFE58@isi.edu> Panos GEVROS wrote: > > low delay (or jitter) is good but whether this should be the network design > goal is a different matter; ... > it all depends on whether need for interactive experience or access to > information proves to be the killer app - but if bounded delay is a necessity > and customers are prepared to pay for it then we may be better of with > something where one directly "dials the web server" My concern is optimizing the entire network for a single class of applications as well. There are different goals - maximizing BW, minimizing latency, minimizing jitter. What matters is how flexible the infrastructure is to providing these, hopefully concurrently. Joe From vern at ee.lbl.gov Fri Mar 9 16:39:19 2001 From: vern at ee.lbl.gov (Vern Paxson) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] paper on "Difficulties in Simulating the Internet" now available Message-ID: <200103100039.f2A0dJN00605@daffy.ee.lbl.gov> The following paper is to appear in IEEE/ACM Transactions on Networking. It's a revision of a previous paper titled "Why We Don't Know How to Simulate the Internet". Vern & Sally Difficulties in Simulating the Internet Sally Floyd & Vern Paxson AT&T Center for Internet Research at ICSI (ACIRI) {floyd,vern}@aciri.org http://www.aciri.org/vern/papers/sim-difficulty.TON.2001.ps.gz http://www.aciri.org/vern/papers/sim-difficulty.TON.2001.pdf Simulating how the global Internet behaves is an immensely challenging undertaking because of the network's great heterogeneity and rapid change. The heterogeneity ranges from the individual links that carry the network's traffic, to the protocols that interoperate over the links, to the "mix" of different applications used at a site, to the levels of congestion seen on different links. We discuss two key strategies for developing meaningful simulations in the face of these difficulties: searching for invariants, and judiciously exploring the simulation parameter space. We finish with a brief look at a collaborative effort within the research community to develop a common network simulator. From vijay at umbc.edu Fri Mar 9 23:00:23 2001 From: vijay at umbc.edu (Vijay Gill) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] Some stats broken down by protocol In-Reply-To: Message-ID: Based on some queries regarding protocols and flows, here are some of the protocol breakdowns. > Some data regarding flows from the Pacific Rim to the US. Keep in mind > that the sampling rate used was fairly low. > > period: 03/04/2001 15:55:18 - 03/05/2001 15:55:19 PST > Protocol Pkts Pkts/sec Bytes Bits/sec > -------- ------------- ------------- ------------- ------------- > tcp 4566586 52 1118739284 103585 > icmp 97183 1 59403325 5500 > udp 207550 2 33848925 3134 > esp 2019 0 1216156 112 > skip 673 0 337732 31 > gre 298 0 94225 8 > ipip 93 0 49844 4 > ospf 108 0 33204 3 > ipv6 14 0 1327 0 > rsvp 5 0 580 0 > > the rsvp is an anomaly. Adding to the above. Regarding Panos' query about flows: Here is what the statman sayeth (josh wepman) 1. How hard would it be to quantify the traffic in terms of "flow" (src/dst/port) pair? "Flow" statistics would be very interesting (e.g active flows per T-sec, for some definition of flow). I'm working with some folks regarding TCP ECN and flow data would be very useful. Number flows (N) at time (T) (a snapshot) or Number flows (N) over time T1 -> T2 (counter over time) Realtime data is of course out of the question. We only get "expired" flow information exported to cflowd. Historical values can be gotten with a bit of work. This is not data available in the cflowd tables maintained in ARTS data. #Flows is not an attribute maintained. So we have to view the raw flow files. A snapshot could be obtained by counting all flows in flows files whose start/end time encompass T(snapshot). The latter (counter over time) could be determined by counting all the flows seen from Time1 to time2. The value is NOT real as a representation of flows on a link. They are a value based on "flows" exported from the router as determined by a router. A flow could be terminated and exported because a FIN occurred, because it was idle for time T, or because the total time of the flow exceeded a limit time value. In order to do either of the above, it needs to be clear that the value represented is NOT flows on a link at a given time, but flows seen exported based on flow-export criteria. It should also be mentioned that the functionality to do this does not currently exist in cflowd, so any efforts here would have to be part of a larger Flow Development effort. Re: DNS To or from port53. More generically, we can use artsprotos to characterize tcp vs udp vs icmp vs whatever else is seen. The time domain can be manipulated to what you may be looking for. Since protocol is an ARTS stored value, we have historical data to work with. Likewise, DNS (port 53TCP/UDP) is maintained in ARTS and available via artsportms and artsports. We can state from T1 -> T2, what was the protocol distribution, and for a set of ports, what were the port distributions. As with the first question, we do not have #Flows, but we do have pkt/byte data. # /usr/local/arts/bin/artsports daily/arts.20010308.ports router: blah blah blah ifIndex: 27 period: 03/07/2001 15:55:20 - 03/08/2001 15:55:18 PST selected ports: 20-21,53,80,119,443 Port InPkts InBytes OutPkts OutBytes ----- ------------- ------------- ------------- ------------- http 5920632 422354154 1350887 1368382195 ELSE 1055537 252207931 3039 2648398 nntp 269884 80456315 32733 2257472 ftp-data 39530 4840976 52149 76430542 domain 69178 4482451 82340 13109974 https 9266 1253745 4470 2222407 ftp 10868 560030 6164 319734 This data was based on a 1:64 packet sampling rate and has not been extrapolated to 1:1 values. An optimal N for sampling has not been determined for this class of link, so the degree of skew in the above numbers cannot be stated with any certainty. If we assumed that 1:64 sampling did correctly represent the true population, then multiplying out the values by 64 would give you the approximate real values. --end statman Hope this was useful. /vijay From gr224 at hermes.cam.ac.uk Sat Mar 10 03:37:53 2001 From: gr224 at hermes.cam.ac.uk (Gaurav Raina) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] paper on "Difficulties in Simulating the Internet" now available In-Reply-To: <200103100039.f2A0dJN00605@daffy.ee.lbl.gov> Message-ID: On Fri, 9 Mar 2001, Vern Paxson wrote: > Difficulties in Simulating the Internet .... > and judiciously exploring the simulation parameter space. We finish with a > brief look at a collaborative effort within the research community to develop > a common network simulator. Apologies if I am merely stating the obvious, but along with developing simulation tools I think it is *imperative* to try and develop theoretical tools which might give some rules of thumb on how the results could scale to a network as large and complex as the Internet. Apart from the obvious industries involved in Internet Research - *some* of the academic reseach groups are : http://netlab.caltech.edu/ http://www-net.cs.umass.edu/ http://www.statslab.cam.ac.uk/~frank/int/ http://comm.csl.uiuc.edu:80/~srikant/pub.html/ An exhaustive list is not possible... Might it be a good idea to consider having a common database/pool for research papers/preprints dealing with the different research topics? Like the way the physics community has the Los Alamos archive. Gaurav From hgs at cs.columbia.edu Sat Mar 10 06:01:14 2001 From: hgs at cs.columbia.edu (Henning G. Schulzrinne) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] paper on "Difficulties in Simulating the Internet" nowavailable References: Message-ID: <3AAA33AA.5FCAA3B9@cs.columbia.edu> Gaurav Raina wrote: > Might it be a good idea to consider having a common database/pool for > research papers/preprints dealing with the different research topics? Like > the way the physics community has the Los Alamos archive. Well, there's netbib, with about 55,000 networking-related papers, http://www.cs.columbia.edu/~hgs/netbib > > Gaurav > > -- Henning Schulzrinne http://www.cs.columbia.edu/~hgs From J.Crowcroft at cs.ucl.ac.uk Sun Mar 11 05:58:05 2001 From: J.Crowcroft at cs.ucl.ac.uk (Jon Crowcroft) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] paper on "Difficulties in Simulating the Internet" nowavailable In-Reply-To: Your message of "Sat, 10 Mar 2001 09:01:14 EST." <3AAA33AA.5FCAA3B9@cs.columbia.edu> Message-ID: <18880.984319085@cs.ucl.ac.uk> In message <3AAA33AA.5FCAA3B9@cs.columbia.edu>, "Henning G. Schulzrinne" typed: >>> Might it be a good idea to consider having a common database/pool for >>> research papers/preprints dealing with the different research topics? Like >>> the way the physics community has the Los Alamos archive. there was/is an attempt to do exactly this, but it takes time to build - i'm not sure what its current status is in the meantime, henning is right - his is the nearest we have, and given the efforts of IEEE Infocom and ACM SIGCOMM and other releated conferences to encourage the archival of conference proceedings online for all, you can generally find most timely information via netbib (and citeseer) now without recourse to walking over the your real libray (btw, you have quite a good one below that ugly tower in cambridge that domiantes the skyline from many approaches:-) >>Well, there's netbib, with about 55,000 networking-related papers, >>http://www.cs.columbia.edu/~hgs/netbib & i personally freel that distributed lassez-fair approaches work better for a diverse growing community than the focuessed approach that math & phsyics people have enjoyed in their social context... cheers jon From hgs at cs.columbia.edu Sun Mar 11 06:28:19 2001 From: hgs at cs.columbia.edu (Henning G. Schulzrinne) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] paper on "Difficulties in Simulating the Internet" nowavailable References: <18880.984319085@cs.ucl.ac.uk> Message-ID: <3AAB8B83.E0C0D23A@cs.columbia.edu> Jon Crowcroft wrote: > > > in the meantime, henning is right - his is the nearest we have, and given > the efforts of IEEE Infocom and ACM SIGCOMM and other releated > conferences to encourage the archival of conference proceedings online > for all, you can generally find most timely information via netbib > (and citeseer) now without recourse to walking over the your real > libray (btw, you have quite a good one below that ugly tower in > cambridge that domiantes the skyline from many approaches:-) Also, if your favorite paper (or your own paper) is not yet in netbib, you're encouraged to enter this information (same web site). > > >>Well, there's netbib, with about 55,000 networking-related papers, > >>http://www.cs.columbia.edu/~hgs/netbib > > & i personally freel that distributed lassez-fair approaches work better > for a diverse growing community than the focuessed approach that math > & phsyics people have enjoyed in their social context... There is also the http://arXiv.org service, but from what I can tell, it is being used extremely rarely by this community. > > > cheers > > jon -- Henning Schulzrinne http://www.cs.columbia.edu/~hgs From demir at usc.edu Wed Mar 14 15:36:31 2001 From: demir at usc.edu (demir) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] [Diffserv-interest] A question on Adaptive Protocols vs Expected Service Classes of Diffserv Message-ID: Hi, There has been vast amount of research on how TCP will react on top of services based on AF/AF-alike PHBs. However, I am not aware of a research that elaborates TCP on top of EF/EF-alike PHBs from service perspective (it seems this is unneccessary at all???). I am aware of that TCP is a widely implemented and used protocol for congestion control and avoidance. I assume, in a "short" time scale, TCP seems reasonable to be used for AF PHB-based services cause TCP-friendly traffic conditioners would refine the TCP's behavior. It seems to me that, may be, we need different adaptive protocols for different service classes (TCP has been developed for the "best-effort" service class). Any ideas/insights/comments? I appreciate very much. Alper K. Demir From nichols at packetdesign.com Wed Mar 14 16:25:38 2001 From: nichols at packetdesign.com (Kathleen Nichols) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] [Diffserv-interest] A question on Adaptive Protocols vs ExpectedService Classes of Diffserv References: Message-ID: <3AB00C02.A6C2BA33@packetdesign.com> Kedar Poduri carried out some simulations like this quite a while back when we were talking about something called a "virtual leased line" built from the EF PHB described in RFC2598. This is real easy to do (of course, you need a shaper at the edge, but Van's been saying that since the origins of this as his "premium" service). You can look at slide 23 of the talk at: http://www.nren.nasa.gov/CFP/nichols_pres/index.htm, a NASA NREN workshop on QoS. It's supposed to "look like a wire". Kathie demir wrote: > > Hi, > There has been vast amount of research on how TCP will react on top of > services based on AF/AF-alike PHBs. However, I am not aware of a research > that elaborates TCP on top of EF/EF-alike PHBs from service perspective > (it seems this is unneccessary at all???). I am aware of that TCP is a > widely implemented and used protocol for congestion control and > avoidance. I assume, in a "short" time scale, TCP seems reasonable to be > used for AF PHB-based services cause TCP-friendly traffic conditioners > would refine the TCP's behavior. It seems to me that, may be, we need > different adaptive protocols for different service classes (TCP has been > developed for the "best-effort" service class). Any > ideas/insights/comments? I appreciate very much. > > Alper K. Demir From bsikdar at networks.ecse.rpi.edu Wed Mar 14 16:42:07 2001 From: bsikdar at networks.ecse.rpi.edu (bsikdar@networks.ecse.rpi.edu) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] Vegas on Linux In-Reply-To: Message-ID: Hi, Could anyone please direct me to any TCP Vegas implementations for Linux? Or for any other platforms? And are there any differences in these implementations? Thanks a lot, Biplab Sikdar Dept. of ECSE Rensselaer Polytechnic Inst., Troy NY 12180 From demir at usc.edu Wed Mar 14 16:44:28 2001 From: demir at usc.edu (demir) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] [Diffserv-interest] A question on Adaptive Protocols vs ExpectedService Classes of Diffserv In-Reply-To: <3AB00C02.A6C2BA33@packetdesign.com> Message-ID: Kathie, From nseddigh at tropicnetworks.com Thu Mar 15 07:53:14 2001 From: nseddigh at tropicnetworks.com (Nabil Seddigh) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] [Diffserv-interest] A question on Adaptive Protocols vs ExpectedService Classes of Diffserv References: Message-ID: <3AB0E56A.1C9DD55C@tropicnetworks.com> There has been work on both TCP modifications as well as "Intelligent" or "TCP-friendly" traffic conditioners to address issues with TCP over the AF PHB. We discovered some interesting results in our experiments with the latter approach - send me email if you're interested. In principle, it should be easier to incorporate changes in edge device traffic conditioners than to affect TCP standard modifications. However, at the same time, there has been limited implementation of intelligent traffic conditioners or policers in deployed products. Best, Nabil Seddigh demir wrote: > My intention > of asking this question was "do we need such an (adaptive) protocol > complexity"? If not, then "do we need to have a different levels of > complexity in the protocols that can used for each service class"? /Or > "One uniform adaptive protocol will suffice cause traffic conditioners can > take care of this"? I assume all is a possibility. What would be a proper > trend to go? Thank you very much > From floyd at aciri.org Thu Mar 15 09:08:56 2001 From: floyd at aciri.org (Sally Floyd) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] two questions about the Internet Message-ID: <200103151708.f2FH8u523767@elk.aciri.org> I maintain a web page http://www.aciri.org/floyd/questions.html of (mostly unanswered) questions about the Internet. I just posted two new questions to that page, and I thought I would also mention them here, in case anyone on this list knows any (partial) answers to any of them. The new questions: ROUND-TRIP TIMES (HOPS, NUMBER OF ASes) OF PACKETS? For packets on a particular link, each packet could be assigned an estimated round-trip time, a number of ASes for the end-to-end path, etc, based on the IP source and destination addresses for that packet. For packets on a particular link, what can we say about the distribution of round-trip times, or of the number of hops traversed, or the number of ASes traversed, or number of continents traversed, or (this is harder) the number of congested links traversed? [Example: For link X, can we say that most packets/bytes stay on that continent? Or that most packets have a minimum round-trip time of at least S seconds? Or that most packets on that link during this period of time traverse more than one congested link on their path from source to destination?] PERIODS OF EXTREME CONGESTION AT A ROUTER? For those routers in the network that do occationally experience congestion, how can we characterize their rare periods of *extreme* congestion (defining extreme congestion, say, as packet drop rates above 5%)? How frequently to these periods of extreme congestion occur, and how long do they last? What fraction can be attributed to flash crowds? to Denial of Service attacks? to fiber cuts or other routing changes? Many thanks, - Sally -------------------------------- http://www.aciri.org/floyd/ -------------------------------- From dovrolis at mail.eecis.udel.edu Thu Mar 15 14:33:18 2001 From: dovrolis at mail.eecis.udel.edu (Constantinos Dovrolis) Date: Thu Mar 25 11:59:35 2004 Subject: [e2e] two questions about the Internet In-Reply-To: <200103151708.f2FH8u523767@elk.aciri.org> Message-ID: Sally, I may have a (very partial) answer to the first question, i.e., what is the distribution of round-trip times (RTTs) for the packets on a certain link. A couple of initial "disclaimers": a) an RTT can be only associated to a packet of a closed-loop protocol, and so our measurements only looked at TCP packets b) our measurements do not refer to per-packet RTTs, but to per-connection RTTs. It is likely that there are important differences in these two distributions (short-RTT flows may tend to carry more data, causing the per-packet RTT distribution to be heavier on lower RTT values). So, the measurements that I refer to were done in the summer of 99 at CAIDA. We were processing traffic traces captured from passive monitors on certain links (note that we get two different traces, one for each direction of the link). In a certain trace, we were estimating the RTT of each TCP connection using the following two rules: a) if we observe the flow from the caller to the callee, the RTT is estimated as the time interval from the SYN to the SYN-ACK. b) if we observe the flow from the callee to the caller (which is usually the traffic from the server to the client), the RTT is estimated from the time spacing of the first 2 or 3 slow-start bursts. The code to do this is tricky (I can send to you, or to anyone else, the code if you want to play with this). So, using these tricks we were measuring the distribution of RTTs in the TCP connections that were present in each (unidirectional) trace. Just as an example of the distributions that we were getting, take a look at: http://www.cis.udel.edu/~dovrolis/rtt-sdsc.eps The graph shows two RTT distributions, one for each direction of the OC-3 link that used to connect UCSD with CERFnet. A few major points from the graph: - About 35% of the connections have RTT < 50ms - About 60% of the connections have RTT < 100ms - There is a significant fraction of connections (20-30%) with RTT>200ms (which is probably close to the upper bound for any type of interactive applications). - About 10% of the RTTs are quite large (some of them in the order of multiple seconds), which may indicate errors in our measurement methodology. This is why I did not include that fraction of RTTs in the graph. Some very interesting measurements on this subject also appear at Mark Allman's "A Web server's view of the transport layer" published at CCR Oct-2000. Mark's measurements originate from traces of the server's traffic (instead of a passive monitor in the the network). Also, he could measure the RTTs more accurately based on the time distance between a non-retransmitted packet and the corresponding ACK. Obviously we cannot do the same, because we don't have the flow of ACKs in the trace. It is interesting that Mark's measurements (see Figure 9) are not *very* different from the graph that I mentioned before. Specifically, his graph shows: - About 35% of the RTTs < 100msec - About 60-70% of the RTTs < 200msec - About 85% of the RTTs < 500msec. Of course Mark's measurements/analysis were much more methodically done (my measurements were only done to get some reasonable values for simulations about other stuff). I hope that this helps. I am also very interested in answers to the rest of your questions. Constantinos Computer and Information Sciences - University of Delaware http://www.cis.udel.edu/~dovrolis/ On Thu, 15 Mar 2001, Sally Floyd wrote: > I maintain a web page > http://www.aciri.org/floyd/questions.html > of (mostly unanswered) questions about the Internet. > I just posted two new questions to that page, and I thought I would > also mention them here, in case anyone on this list knows any > (partial) answers to any of them. > > The new questions: > > ROUND-TRIP TIMES (HOPS, NUMBER OF ASes) OF PACKETS? > For packets on a particular link, each packet could be assigned an > estimated round-trip time, a number of ASes for the end-to-end > path, etc, based on the IP source and destination addresses for > that packet. For packets on a particular link, what can we say > about the distribution of round-trip times, or of the number of hops > traversed, or the number of ASes traversed, or number of continents > traversed, or (this is harder) the number of congested links traversed? > > [Example: For link X, can we say that most packets/bytes stay on > that continent? Or that most packets have a minimum round-trip > time of at least S seconds? Or that most packets on that link > during this period of time traverse more than one congested link > on their path from source to destination?] > > PERIODS OF EXTREME CONGESTION AT A ROUTER? > For those routers in the network that do occationally experience > congestion, how can we characterize their rare periods of *extreme* > congestion (defining extreme congestion, say, as packet drop rates > above 5%)? How frequently to these periods of extreme congestion > occur, and how long do they last? What fraction can be attributed > to flash crowds? to Denial of Service attacks? to fiber cuts or > other routing changes? > > Many thanks, > - Sally > -------------------------------- > http://www.aciri.org/floyd/ > -------------------------------- > From nahum at watson.ibm.com Thu Mar 15 15:51:11 2001 From: nahum at watson.ibm.com (Erich Nahum) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: from "Constantinos Dovrolis" at Mar 15, 2001 05:33:18 PM Message-ID: <200103152351.SAA33906@orinoco.watson.ibm.com> Constantinos Dovrolis writes: > > It is interesting that Mark's measurements (see Figure 9) are not *very* > different from the graph that I mentioned before. Specifically, > his graph shows: > - About 35% of the RTTs < 100msec > - About 60-70% of the RTTs < 200msec > - About 85% of the RTTs < 500msec. > Of course Mark's measurements/analysis were much more methodically > done (my measurements were only done to get some reasonable values > for simulations about other stuff). Srini Seshan (when he was here at Watson) had some packet trace data from the 1996 Olympic Web server, but it's a bit old now. The technique was similar to what Mark Allman did. For the record, though, it had: - 25% of the RTTs < 115 ms - 50% of the RTTs < 338 ms - 75% of the RTTs < 778 ms The RTTs are obviously going to vary depending on what kind of connection you have (T3, OC-768) as well as where your clients are (NY, CA, Greece). -Erich -- Erich M. Nahum IBM T.J. Watson Research Center Networking Research P.O. Box 704 nahum@watson.ibm.com Yorktown Heights NY 10598 From ggm at dstc.edu.au Thu Mar 15 16:23:51 2001 From: ggm at dstc.edu.au (George Michaelson) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: Your message of "Thu, 15 Mar 2001 18:51:11 EST." <200103152351.SAA33906@orinoco.watson.ibm.com> Message-ID: <12360.984702231@dstc.edu.au> Srini Seshan (when he was here at Watson) had some packet trace data from the 1996 Olympic Web server, but it's a bit old now. The technique was similar to what Mark Allman did. For the record, though, it had: - 25% of the RTTs < 115 ms - 50% of the RTTs < 338 ms - 75% of the RTTs < 778 ms The RTTs are obviously going to vary depending on what kind of connection you have (T3, OC-768) as well as where your clients are (NY, CA, Greece). -Erich The 96 Olympics were hosted behind multiple backends, geographically distributed? I thought Nagano was, I went to a seminar by IBM on it. Because if so, there were presumably frontend boxes making decisions on backend server, which would either intuit best-fit path or else map it into some simple model like BGP AS or link-based region and so skew RTT in favour of shorter-hop and/or ligher-load hosts. -George -- George Michaelson | DSTC Pty Ltd Email: ggm@dstc.edu.au | University of Qld 4072 Phone: +61 7 3365 4310 | Australia Fax: +61 7 3365 4311 | http://www.dstc.edu.au From nahum at watson.ibm.com Thu Mar 15 18:04:46 2001 From: nahum at watson.ibm.com (Erich Nahum) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: <12360.984702231@dstc.edu.au> from "George Michaelson" at Mar 16, 2001 10:23:51 AM Message-ID: <200103160204.VAA53100@orinoco.watson.ibm.com> George Michaelson writes: > > The 96 Olympics were hosted behind multiple backends, geographically > distributed? I thought Nagano was, I went to a seminar by IBM on it. > > Because if so, there were presumably frontend boxes making decisions > on backend server, which would either intuit best-fit path or else > map it into some simple model like BGP AS or link-based region and > so skew RTT in favour of shorter-hop and/or ligher-load hosts. 96 (Atlanta) was the first olympics that IBM hosted, and I believe it was just one complex in Southbury, CT. 98 (Nagano) and 2000 (Australia) were hosted by 4 main sites: Bethesda (for Europe), Shaumberg IL and someplace in Ohio (for the Americas) and Tokyo (for Asia). The request routing was done on a very course-grain level, basically through the routing tables. E.g., if you were in Europe, olympics.com pointed to Bethesda. I think it was done at the routing layer and not through DNS. The front ends of each cluster were IBM network dispatcher TCP sprayers, which routed to back-end nodes on the same LAN. So I believe the RTT distribution seen by a complex would be the same across nodes within that cluster. -Erich -- Erich M. Nahum IBM T.J. Watson Research Center Networking Research P.O. Box 704 nahum@watson.ibm.com Yorktown Heights NY 10598 From hari at chive.lcs.mit.edu Thu Mar 15 19:02:17 2001 From: hari at chive.lcs.mit.edu (Hari Balakrishnan) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: Message from Erich Nahum of "Thu, 15 Mar 2001 21:04:46 EST." <200103160204.VAA53100@orinoco.watson.ibm.com> Message-ID: <200103160302.f2G32Ha28828@chive.lcs.mit.edu> Erich, > George Michaelson writes: > > > > The 96 Olympics were hosted behind multiple backends, geographically > > distributed? I thought Nagano was, I went to a seminar by IBM on it. > > > > Because if so, there were presumably frontend boxes making decisions > > on backend server, which would either intuit best-fit path or else > > map it into some simple model like BGP AS or link-based region and > > so skew RTT in favour of shorter-hop and/or ligher-load hosts. > > 96 (Atlanta) was the first olympics that IBM hosted, and I believe it was > just one complex in Southbury, CT. For the 1996 Atlanta games, IBM actually ran multiple servers for the Olympics, but they weren't transparent (i.e., they had distinct DNS names). The data being referred to here was collected at Southbury, CT. The other sites were, if I recall, at Keio (Japan), Cornell (NY), Karlsruhe (Germany), and Hursley (UK). The Southbury site was connected via T3 links to 4 US NAPs: Chicago (Bellcore & Ameritech), SF Bay Area (Bellcore and PacBell), NY (Sprint), and DC (MFS Datanet). > 98 (Nagano) and 2000 (Australia) > were hosted by 4 main sites: Bethesda (for Europe), Shaumberg IL and > someplace in Ohio (for the Americas) and Tokyo (for Asia). The > request routing was done on a very course-grain level, basically > through the routing tables. E.g., if you were in Europe, > olympics.com pointed to Bethesda. I think it was done at > the routing layer and not through DNS. > The front ends of each cluster were IBM network dispatcher TCP > sprayers, which routed to back-end nodes on the same LAN. So > I believe the RTT distribution seen by a complex would be the same > across nodes within that cluster. Sounds about right, if you believe the load-balancing was working correctly. :} (I'm not saying it wasn't!) Hari > > -Erich > > -- > Erich M. Nahum IBM T.J. Watson Research Center > Networking Research P.O. Box 704 > nahum@watson.ibm.com Yorktown Heights NY 10598 From widmer at informatik.uni-mannheim.de Fri Mar 16 05:37:48 2001 From: widmer at informatik.uni-mannheim.de (Joerg Widmer) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] SIG on Networked Computer Games Message-ID: <3AB2172C.EA67F19A@informatik.uni-mannheim.de> Hi all, With respect to the recent discussion about networked computer games this SIG might be of interest. Cheers, J?rg/Martin Call for Participation Special Interest Group on Networked Computer Games (SIG NetGame) Over the past three to four years networked computer games have been a tremendous commercial success. Games like Ultima Online, Everquest, Doom, Quake, Diablo II and others have attracted an audience of several million players, worldwide. They are one of the few Internet services for which end users are actually willing to pay money. As the Internet becomes ubiquitous through wireless and/or cheaper Internet access the the audience for networked computer games will increase rapidly, creating a mass market with a multi-billion dollar volume. However, most - if not all - of the successful networked computer games have encountered a large number of technical challenges that are inherent to this application area. These range from inadequate support by network and transport protocols to consistency problems and security breaches (or cheating as players prefer to call it). At the same time scientists have begun to discover networked computer games as an extremely challenging and rewarding area of research. What makes this area of research particularly fascinating is that solutions found for networked computer games tend to solve related problems in other areas such as computer supported collaborative work, distance education and telemedicine. It is the aim of this SIG to bring together developers of commercial and non-commercial networked computer games, service providers, scientists, and interested individuals in order to discuss - and possibly solve - technical challenges of networked computer games. Topics of interest include, but are certainly not limited to: - network and transport protocols - application-level protocol design - architectures for service providers - consistency mechanisms - security / cheating prevention - middle-ware (e.g. Direct Play) - billing and charging ... for networked computer games. You can subscribe to the NetGame mailing list through the NetGame webpage: http://www.informatik.uni-mannheim.de/netgame/index.html or by sending a mail to: netgame-l-request@pi4.informatik.uni-mannheim.de with the following line in the BODY of the message: subscribe I'm looking forward to interesting discussions on netgame-l. Sincerely, Martin Mauve Disclaimer: This SIG is currently not affiliated with any other organization. From oleg at inforocket.com Fri Mar 16 06:46:43 2001 From: oleg at inforocket.com (Oleg Vishnepolsky) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: <200103160204.VAA53100@orinoco.watson.ibm.com> Message-ID: Erich M. Nahum writes: >96 (Atlanta) was the first olympics that IBM hosted, and I believe it was >just one complex in Southbury, CT. 98 (Nagano) and 2000 (Australia) >were hosted by 4 main sites: Bethesda (for Europe), Shaumberg IL and >someplace in Ohio (for the Americas) and Tokyo (for Asia). The >request routing was done on a very course-grain level, basically >through the routing tables. E.g., if you were in Europe, >olympics.com pointed to Bethesda. I think it was done at >the routing layer and not through DNS. How is it even possible not to involve DNS ? If DNS was giving out the same IP address to olympics.com irrespective of the where requests came from, then routing would have been real tricky, to say the least. Oleg Vishnepolsky From pingpan at juniper.net Fri Mar 16 09:40:48 2001 From: pingpan at juniper.net (Ping Pan) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet References: <200103151708.f2FH8u523767@elk.aciri.org> Message-ID: <3AB25020.99C75551@cs.columbia.edu> Sally Floyd wrote: > > The new questions: > > ROUND-TRIP TIMES (HOPS, NUMBER OF ASes) OF PACKETS? > For packets on a particular link, each packet could be assigned an > estimated round-trip time, a number of ASes for the end-to-end > path, etc, based on the IP source and destination addresses for > that packet. For packets on a particular link, what can we say > about the distribution of round-trip times, or of the number of hops > traversed, or the number of ASes traversed, or number of continents > traversed, or (this is harder) the number of congested links traversed? > Hi, Hop-counters: http://www.nlanr.net/NA/Learn/wingspan.html AS length: http://moat.nlanr.net/ASPL/ (from University of Oregon) BTW, there are several good pages on Internet questions: 1. Henning Schulzrinne: http://www.cs.columbia.edu/~hgs/internet/traffic.html 2. Merit: http://www.merit.edu/ipma/reports/ 3. NLANR: http://www.nlanr.net/NA/Learn/ - Ping Pan From pingpan at juniper.net Fri Mar 16 10:07:48 2001 From: pingpan at juniper.net (Ping Pan) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet References: <200103151708.f2FH8u523767@elk.aciri.org> Message-ID: <3AB25674.B3EE78A9@juniper.net> Sally Floyd wrote: > > PERIODS OF EXTREME CONGESTION AT A ROUTER? > For those routers in the network that do occationally experience > congestion, how can we characterize their rare periods of *extreme* > congestion (defining extreme congestion, say, as packet drop rates > above 5%)? How frequently to these periods of extreme congestion > occur, and how long do they last? What fraction can be attributed > to flash crowds? to Denial of Service attacks? to fiber cuts or > other routing changes? > Almost forgot, please take a look at http://www.nordu.net/stats/. This is one of the better places where you can monitor the link traffic for both average and peak rates, and draw your own conclusion on link congestion and duration. In the past several years, most of US providers stop showing their networks, and only provide the average bw utilization, which is low anyway. It is believed that the peak/average ratio is around 3-4 or higher, but I have not seen solid evidence on this since NSFNET. I don't think there are too many fiber cuts in the network (well... on the other hand, China-US undersea cable was cut many days ago, and the link was still down.) But in some networks, providers do shift traffic between links quite often. - Ping From nahum at watson.ibm.com Fri Mar 16 11:00:51 2001 From: nahum at watson.ibm.com (Erich Nahum) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] two questions about the Internet In-Reply-To: from "Oleg Vishnepolsky" at Mar 16, 2001 09:46:43 AM Message-ID: <200103161900.OAA33888@orinoco.watson.ibm.com> Oleg Vishnepolsky writes: > > >96 (Atlanta) was the first olympics that IBM hosted, and I believe it was > >just one complex in Southbury, CT. 98 (Nagano) and 2000 (Australia) > >were hosted by 4 main sites: Bethesda (for Europe), Shaumberg IL and > >someplace in Ohio (for the Americas) and Tokyo (for Asia). The > >request routing was done on a very course-grain level, basically > >through the routing tables. E.g., if you were in Europe, > >olympics.com pointed to Bethesda. I think it was done at > >the routing layer and not through DNS. > > How is it even possible not to involve DNS ? If DNS was giving out the > same IP address to olympics.com irrespective of the where requests > came from, then routing would have been real tricky, to say the least. I wasn't the one who did the work, so take my recollections with a grain of salt. Hari was one of the authors on the SigMetrics 97 and InfoCom 98 papers that describe this work, so I would trust him on this one about the 96 olympics. As for the later ones, this is what I've been told. It doesn't seem tricky to me, but I'm not a routing person. I'll try to dig up the info and post it here next week. -Erich -- Erich M. Nahum IBM T.J. Watson Research Center Networking Research P.O. Box 704 nahum@watson.ibm.com Yorktown Heights NY 10598 From dino.saija at libero.it Tue Mar 20 01:57:41 2001 From: dino.saija at libero.it (dino.saija@libero.it) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] TCP Traffic generator Message-ID: I'd like to use a free TCP traffic generator(i.e in c++).Where can I find? thank you From srh at merit.edu Tue Mar 20 05:50:12 2001 From: srh at merit.edu (Susan Harris) Date: Thu Mar 25 11:59:36 2004 Subject: [e2e] NANOG 22 CFP Message-ID: * * * * * * * * * * * * * * * * * CALL FOR PRESENTATIONS * * NANOG 22 * * May 20 - 22, 2001 * * * * * * * * * * * * * * * * * The North American Network Operators' Group (NANOG) will hold its 22nd meeting in Scottsdate, Arizona, between May 20-22, 2001. The meeting will be hosted by CenterGate Research Group. NANOG conferences provide a forum for the coordination and dissemination of technical information related to large-scale (i.e., national/international) Internet backbone networking technologies and operational practices. NANOG meetings, held three times each year, include two days of short presentations, plus afternoon/evening tutorial sessions. Meetings are informal, with an emphasis on relevance to current backbone engineering practices. The conference draws over 600 participants, mainly consisting of engineering staff from large national service providers, and members of the research and education community. Now in its sixth year, NANOG evolved from the NSFNET "regional-techs" meetings, where technical staff from the regional networks met to discuss operational issues of common concern. With the emergence of the commercial Internet, NANOG meetings evolved to include a broader base of providers, network operators, and researchers. The meeting will be held at the DoubleTree Paradise Valley. For more information about NANOG meetings, schedules, and logistics, see: http://www.nanog.org ------------------------------------------------------------------------------ CALL FOR PRESENTATIONS NANOG invites presentations on backbone engineering, coordination, and research topics. Presentations should highlight issues relating to technology already deployed or soon to be deployed in core Internet backbones and exchange points. Previous meetings have included presentations on: - Backbone traffic engineering - Coordination of inter-provider QoS - Deployment experience with queueing disciplines (CAR, RED) - Inter-provider security and routing protocol authentication - Routing scalability in backbone infrastructures - Security issues for the Internet core - Routing policy specification and backbone router configuration - Building large-scale measurement infrastructure - Cooperative inter-provider caching - Alternatives to hot-potato routing - Recommendations on queue management and congestion avoidance - Experience with differentiated services - Reports from next-generation networks (CANARIE, Internet2)) - Inter-domain multicast deployment - Backbone network failure analysis Tutori