From wheinzel at ece.rochester.edu Mon Jun 1 20:14:40 2009 From: wheinzel at ece.rochester.edu (Wendi Heinzelman) Date: Mon, 1 Jun 2009 23:14:40 -0400 Subject: [e2e] INFOCOM 2010 Call for Papers Message-ID: <00e301c9e330$44d9f580$ce8de080$@rochester.edu> ***************************** **** IEEE INFOCOM 2010 **** **** **** **** CALL FOR PAPERS **** ***************************** The 29th Annual Conference of the IEEE Communications Society March 15 - 19, 2010 San Diego, California USA http://www.ieee-infocom.org/2010 The IEEE INFOCOM 2010 conference seeks papers describing significant research contributions to the field of computer and data communication networks. We invite submissions on network architecture, design, implementation, operations, analysis, measurement, performance, and simulation. Topics of interest include, but are not limited to, the following: * Ad hoc mobile networks * Addressing & location management * Broadband access technologies * Capacity planning * Cellular & broadband wireless nets * Cognitive radio networking * Congestion control * Content distribution * Cross layer design & optimization * Cyber-physical computing & networking * Denial of service * Delay/disruption tolerant networks * Future Internet design * Implementation & experimental testbeds * Middleware support for networking * Mobility models & systems * Multicast & anycast * Multimedia protocols * Network applications & services * Network architectures * Network control * Network management * Network simulation & emulation * Novel network architectures * Optical networks * Peer-to-peer networks * Performance evaluation * Power control & management * Pricing & billing * Quality of service * Resource allocation & management * Routing protocols * Scheduling & buffer management * Security, trust, & privacy * Self-organizing networks * Sensor nets & embedded systems * Service overlays * Social computing * Switches & switching * Topology characterization & inference * Traffic measurement and analysis * Traffic engineering & control * Virtual & overlay networks * Web services & performance * Wireless mesh networks & protocols Important Dates: * Abstract due: July 24, 2009 * Full paper due: July 31, 2009 * Notification of acceptance: November 21, 2009 Organizing Committee: General Chair: Giridhar Mandyam (Qualcomm, USA) Vice Chair: Cedric Westphal (Docomo Labs, USA) Technical Program Co-Chairs: Mooi Choo Chuah (Lehigh University, USA), Reuven Cohen (Technion, Israel) and Guoliang Xue (Arizona State University, USA) Standing Committee Chair: Harvey Freeman (HAF Consulting, Inc., USA) -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090601/c29bbdf7/attachment.html From yzhang at cs.utexas.edu Tue Jun 2 23:11:43 2009 From: yzhang at cs.utexas.edu (Yin Zhang) Date: Wed, 3 Jun 2009 01:11:43 -0500 (CDT) Subject: [e2e] ANCS 2009 Call for Papers Message-ID: [We apologize in advance if you receive multiple copies of this message] CALL FOR PAPERS The 5th ACM/IEEE Symposium on Architectures for Networking and Communications Systems ANCS 2009 http://www.ancsconf.org October 19-20, 2009 Princeton, New Jersey, USA Sponsored by: ACM Special Interest Group on Computer Architecture (SIGARCH) ACM Special Interest Group on Communications (SIGCOMM) IEEE Computer Society Tech. Committee on Computer Architecture IMPORTANT DATES Paper registration and abstract: June 22, 2009 Submission deadline: June 29, 2009 Author notification: August 10, 2009 CONFERENCE OVERVIEW ANCS is a systems-oriented research conference, presenting original work that explores the relationship between the architecture of modern computer networks and the architecture of the individual hardware and software elements from which these networks are built. This year's conference will particularly emphasize insight into broader systems issues in its paper selection, to recognize and foster the growth of research that lies at the intersection of computer and network systems architecture. Topics of interest include, but are not limited to: * System design for future network architectures * Network architectures enabled by converged platforms * Virtualized infrastructure architectures, rationale, and devices * Converged router, server, and storage platforms * Content-centric architectures, platforms, and mechanisms * Scalable programming and application frameworks * High performance / high function packet processing platforms * Power- and size-optimized computer and communications platforms * High-speed networking mechanisms and algorithms * Network security architectures and security anchor/ enhancement devices * Single-chip platform integration * Network measurement techniques, architectures, and devices * Techniques and systems for large-scale data analysis * Host-network interface issues * Data center architectures * Router and switch architectures The PAPER DEADLINE for submissions is June 29, 2009 at 11:59PM PST (US). ANCS will use double-blind reviewing, so submitted papers should not include the authors' names. Paper registration and submission must be done electronically through EDAS (edas.info). Registration, including the abstract, must be completed no later than June 22, 2008 at 11:59PM PDT (US). All papers must be submitted in PDF format on letter-size paper. Submissions must be viewable by Adobe Acrobat Reader (version 5.0 or higher) and should not exceed 10 pages in ACM/SIG conference paper format using 10 pt font. Submissions exceeding the maximum limit will not be reviewed by the program committee. Camera-ready versions of the accepted papers will be required to use the ACM SIG format (http://www.acm.org/sigs/pubs/proceed/template.html). We encourage submissions containing original ideas. Like other conferences, ANCS requires that papers not be submitted simultaneously to any other conferences or publications; that submissions not be previously published; and that accepted papers not be subsequently published elsewhere. Notification of Acceptance: August 10, 2009. Contact the program chairs with any questions at ancsTPC at arl.wustl.edu. CONFERENCE COMMITTEE GENERAL CHAIRS Peter Z. Onufryk, IDT K. K. Ramakrishnan, AT&T Labs Research PROGRAM CHAIRS Patrick Crowley, Washington University John Wroclawski, USC/ISI PROGRAM COMMITTEE Jack Brassil, HP Srihari Cadambi, NEC Labs Chita Das, Penn. State Univ. Bruce Davie, Cisco Will Eatherton, Cisco Hans Eberle, Sun Nick Feamster, Georgia Tech. Manolis Katevenis, FORTH/Univ. of Crete T.V. Lakshman, Bell Labs Bill Lin, Univ. of California, San Diego Bin Liu, Tsinghua Univ. Dave Maltz, Microsoft Laurent Mathy, Lancaster University Derek McAuley, Univ. of Nottingham Jayaram Mudigonda, HP Eugene Ng, Rice University Robert Olsen, Cisco Vijay Pai, Purdue University Dhabaleswar Panda, Ohio State Univ. Craig Partridge, BBN Viktor Prasanna, USC Chuck Thacker, Microsoft Jonathan Turner, Washington University Manish Vachharajani, U. Colorado at Boulder Tilman Wolf, Univ. of Massachusetts Fang Yu, Microsoft FINANCE CHAIR Srihari Cadambi, NEC Labs PUBLICITY CHAIR Yin Zhang, UT Austin POSTER SESSION CHAIR Bill Lin, Univ. of California, San Diego STEERING COMMITTEE Laxmi Bhuyan, UC-Riverside Patrick Crowley, Washington U. Mark Franklin, Washington U. Derek McAuley, Univ. of Nottingham Nick McKeown, Stanford Univ. Peter Z. Onufryk, IDT K. K. Ramakrishnan, AT&T Labs Research Raj Yavatkar, Intel From tang at cs.montana.edu Wed Jun 3 07:19:06 2009 From: tang at cs.montana.edu (Jian (Neil) Tang) Date: Wed, 3 Jun 2009 10:19:06 -0400 Subject: [e2e] Call for Participation - IWQoS 2009 Message-ID: <200906031019055049198@cs.montana.edu> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Call for Participation +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ IEEE IWQoS 2009 17th IEEE International Workshop on Quality of Service July 13-15, 2009 Charleston, South Carolina http://www.ieee-iwqos.org/ +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ You are cordially invited to participate in the upcoming 17th IEEE International Workshop on Quality of Service (IEEE IWQoS 2009) sponsored by IEEE Communications Society. IWQoS 2009 will offer 9 exciting technical sessions with 28 regular papers and 10 short papers. It will also feature two keynote talks by Prof. Peter Steenkiste (Carnegie Mellon University) and Prof. Roch Guerin (University of Pennsylvania). For more details, please visit http://www.ieee-iwqos.org/program.php. Please note that the early registration deadline is June 26 and the deadline for reduced room rate in Charleston Place Hotel is June 13. We look forward to welcoming you in Charleston, SC. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From guylzodi at gmail.com Mon Jun 15 08:04:00 2009 From: guylzodi at gmail.com (Guy - Alain Lusilao-Zodi) Date: Mon, 15 Jun 2009 17:04:00 +0200 Subject: [e2e] RTP-Lite implementation Message-ID: <55c1a2ac0906150804x5138225fo82f020e606abfed@mail.gmail.com> Dear all, Does anyone have an implementation of RTP-lite or a compressed header implementation of RTP, in NS2 or any other platform? Regards -- ---------------------------------------------------------- G.-A. Lusilao-Zodi voice : 0216554019 PhD in Video streaming cell :082687993 Communication group guylzodi at gmail.com university of cape-Town ----------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090615/2648d6f2/attachment.html From pganti at gmail.com Fri Jun 19 12:20:05 2009 From: pganti at gmail.com (Paddy Ganti) Date: Fri, 19 Jun 2009 12:20:05 -0700 Subject: [e2e] Why Buffering? Message-ID: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> For the question of "why do routers buffer", Vadim Antonov provided the following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) --------------------------------------------------------------------------------------------------------------------------------------------------------- Well, so far nobody provided a valid explanation for the necessity of buffering in routers (and any other stochastically multiplexing devices). The real reason for having buffers is the fact that information about congestions takes some time to propagate. (In TCP/IP congestion are detected by seeing lost packets). If buffers are not sufficient to hold packets until TCP speakers see congestion and slow down, the system becomes unstable. Buffers are, essentially, inertial elements in the delayed negative-feedback control loop. Insufficient inertia causes oscillations in such systems, which is sometimes useful, but in case of networks leads to decreased througoutput because the wire is utilized fully only at upswings and underutilized on downswings (collapsed TCP windows aggravate the effect futher). Consequently, the holding capacity of buffers should be sufficient to bring the inertia of the system up to the reaction time of the negative feedback (this is a simplification, of course). This reaction time is about one round-trip time for end-to-end packets. In real networks, the RTTs differ for different paths, so some "characteristic" RTT is used. So, to hold packets until TCPs slow down a router needs cRTT * BW bits of buffer memory (where BW is the speed of the interface). This rule can be somewhat relaxed if congestion control loop is engaged proactively before congestion occured (by using Random Early Detection), but not much. Even with properly sized buffers, sessions with longer RTTs suffer from congestions disproportionately because TCPs on the ends never get enough time to recover fully (i.e. to bring windows to large enough size to maintain steady stream of packets), while small-RTT sessions recover quickly, and, therefore, get bigger share of bandwidth. But I'm digressing :) --vadim -------------------------------------------------------------------------------------------------------------------------------- My question to the group is : what other reasons not mentioned here can be reasons to buffer in a router? -Paddy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090619/09376348/attachment.html From yganjali at cs.toronto.edu Fri Jun 19 16:42:20 2009 From: yganjali at cs.toronto.edu (Yashar Ganjali) Date: Fri, 19 Jun 2009 19:42:20 -0400 Subject: [e2e] Why Buffering? In-Reply-To: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: <005201c9f137$968fedc0$c3afc940$@toronto.edu> Paddy, You might want to take a look at these. The first one has a great overview of why we need buffers, explains the Bandwitdth x RTT in detail, and shows we might be able to reduce the buffer sizes down to $Bandwidth x RTT / \sqrt(N)$ . The second one shows that in networks with slow access links (or paced TCP) you can reduce the buffer sizes down to 10-20 packets. The last one describes test-bed and real network experiments on buffer sizing. Hope this answers your question. --Yashar "Sizing Router Buffers" Guido Appenzeller, Isaac Keslassy and Nick McKeown ACM SIGCOMM 2004, Portland, August 2004. pdf, ps M. Enachescu, Y. Ganjali, A. Goel, N. McKeown, and T. Roughgarden, "Routers with Very Small Buffers," Proceedings of the IEEE INFOCOM'06, Barcelona, Spain: 2006. and N. Beheshti, Y. Ganjali, M. Ghobadi, N. McKeown, and G. Salmon, "Experimental Study of Router Buffer Sizing," Proceedings of Internet Measurement Conference (IMC), Vouliagmeni, Greece: 2008, pp. 197-210 ---------------------------------------------------------------------------- -- Yashar Ganjali Bahen Center for Information Technology Assistant Professor 40 St. George Street, Room BA 5238 Department of Computer Science Toronto, ON M5S 2E4 University of Toronto Phone: (416) 978-2952 http://www.cs.toronto.edu/~yganjali Fax: (416) 978-4765 ---------------------------------------------------------------------------- -- From: end2end-interest-bounces at postel.org [mailto:end2end-interest-bounces at postel.org] On Behalf Of Paddy Ganti Sent: Friday, June 19, 2009 3:20 PM To: end2end-interest at postel.org Subject: [e2e] Why Buffering? For the question of "why do routers buffer", Vadim Antonov provided the following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- - Well, so far nobody provided a valid explanation for the necessity of buffering in routers (and any other stochastically multiplexing devices). The real reason for having buffers is the fact that information about congestions takes some time to propagate. (In TCP/IP congestion are detected by seeing lost packets). If buffers are not sufficient to hold packets until TCP speakers see congestion and slow down, the system becomes unstable. Buffers are, essentially, inertial elements in the delayed negative-feedback control loop. Insufficient inertia causes oscillations in such systems, which is sometimes useful, but in case of networks leads to decreased througoutput because the wire is utilized fully only at upswings and underutilized on downswings (collapsed TCP windows aggravate the effect futher). Consequently, the holding capacity of buffers should be sufficient to bring the inertia of the system up to the reaction time of the negative feedback (this is a simplification, of course). This reaction time is about one round-trip time for end-to-end packets. In real networks, the RTTs differ for different paths, so some "characteristic" RTT is used. So, to hold packets until TCPs slow down a router needs cRTT * BW bits of buffer memory (where BW is the speed of the interface). This rule can be somewhat relaxed if congestion control loop is engaged proactively before congestion occured (by using Random Early Detection), but not much. Even with properly sized buffers, sessions with longer RTTs suffer from congestions disproportionately because TCPs on the ends never get enough time to recover fully (i.e. to bring windows to large enough size to maintain steady stream of packets), while small-RTT sessions recover quickly, and, therefore, get bigger share of bandwidth. But I'm digressing :) --vadim ---------------------------------------------------------------------------- ---------------------------------------------------- My question to the group is : what other reasons not mentioned here can be reasons to buffer in a router? -Paddy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090619/2d5433ce/attachment.html From dhc2 at dcrocker.net Fri Jun 19 21:23:04 2009 From: dhc2 at dcrocker.net (Dave CROCKER) Date: Fri, 19 Jun 2009 21:23:04 -0700 Subject: [e2e] Why Buffering? In-Reply-To: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: <4A3C6428.30002@dcrocker.net> Paddy Ganti wrote: > The real reason for having buffers is the fact that information about > congestions takes some time to propagate. (In TCP/IP congestion are > detected by seeing lost packets). In the late 70s and 80s, Kleinrock gave a rather simple explanation for using queuing (buffering) that I think is compatible with Antonov's point: After a lengthy and detailed introduction, he put up a graph of throughput (x) versus delay (y). It had a very shallow increase until hitting a very sharp knee and then was almost vertical. He observed that before the knee, you don't need queuing because you don't have any congestion. And after the knee, queuing doesn't help because you simply don't have enough capacity. Queuing is for transient problems rather than an excessive average: The knee of the curve. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net From Jon.Crowcroft at cl.cam.ac.uk Sat Jun 20 02:36:30 2009 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Sat, 20 Jun 2009 10:36:30 +0100 Subject: [e2e] Why Buffering? In-Reply-To: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: at the physical layer you've got an asyncronous net for some time now so you've had to store and forward each packet in the link layer we do error detection so you need the whole packet to compute the final cjeck to see if you discard before sending on in the IP layer, we do statistical multiplexing and work conservation (at least by convention anyhow) so burstiness propagates (slow down in one flow speeds up other flows), and we dont do admission control so unless by some random luck, all senders send in a traffic matrix perfectly matched to the network provisioning, you will get packets arriving on multiple interfaces destined for one output interface which isn't fast enough, even just with 1 packet per RTT (i.e. just a set of clients sending a connection setup/TCP SYN packet alone to one server) so irrespective of end-to-end error correction, flow and congestion control, you have to have _some_ buffers the question then is how many more than the absolute minumum and for that, see yashar ganjali's comprehensive reference list for the latest thinking ... of course, we could implement a hop-by-hop error and flow control and determinisically go for the small buffer case (except there are other reasons people don't like the hop-by-hop designs) or we could have rate controlled sources and admission control and get determinstically small buffers...but that too would lead to a lot more complexity in the network (a question never satisfactorally answered is what the relative cost of computational complexity in switches for either of these two architectures, compared with the cost of the currently, apparently, oversized buffers in core routers is... of course, this cost is a moving target as is the end-to-end behaviour of the net so its a hard problem). there are lots of larger, longer term buffers in the net too - CDNs and P2P systems act as whole or partial file buffers to deal with mismatched timing between senders and recipients ...one might consider a network architecture where distribution of new content (e.g. software updates, or new films and music and games) to caches on a CDN was completely scheduled (some do this a bit to avoid overloading the net already) on the other hand, the _resource pooling_ and law of large numbrs getting you small buffering seems to be a smarter way for the internet to evolve (assuming one's goal is to scale the net up but get latency (and jitter) to go down to something a bit more pleasing for interactive (human-human) apps like voip and games... In missive <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5 at mail.gmail.com>, Paddy Ganti typed: >>--001e680f1b406effbc046cb8699c >>Content-Type: text/plain; charset=ISO-8859-1 >>Content-Transfer-Encoding: 7bit >> >>For the question of "why do routers buffer", Vadim Antonov provided the >>following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) >>--------------------------------------------------------------------------------------------------------------------------------------------------------- >>Well, so far nobody provided a valid explanation for the necessity of >>buffering in routers (and any other stochastically multiplexing devices). >> >>The real reason for having buffers is the fact that information about >>congestions takes some time to propagate. (In TCP/IP congestion are >>detected by seeing lost packets). >> >>If buffers are not sufficient to hold packets until TCP speakers see >>congestion and slow down, the system becomes unstable. Buffers are, >>essentially, inertial elements in the delayed negative-feedback control >>loop. Insufficient inertia causes oscillations in such systems, which is >>sometimes useful, but in case of networks leads to decreased througoutput >>because the wire is utilized fully only at upswings and underutilized on >>downswings (collapsed TCP windows aggravate the effect futher). >> >>Consequently, the holding capacity of buffers should be sufficient to >>bring the inertia of the system up to the reaction time of the negative >>feedback (this is a simplification, of course). This reaction time is >>about one round-trip time for end-to-end packets. >> >>In real networks, the RTTs differ for different paths, so some >>"characteristic" RTT is used. So, to hold packets until TCPs slow down a >>router needs cRTT * BW bits of buffer memory (where BW is the speed of the >>interface). This rule can be somewhat relaxed if congestion control loop >>is engaged proactively before congestion occured (by using Random Early >>Detection), but not much. >> >>Even with properly sized buffers, sessions with longer RTTs suffer from >>congestions disproportionately because TCPs on the ends never get enough >>time to recover fully (i.e. to bring windows to large enough size to >>maintain steady stream of packets), while small-RTT sessions recover >>quickly, and, therefore, get bigger share of bandwidth. But I'm >>digressing :) >> >>--vadim >>-------------------------------------------------------------------------------------------------------------------------------- >> >>My question to the group is : what other reasons not mentioned here can be >>reasons to buffer in a router? >> >>-Paddy >> >>--001e680f1b406effbc046cb8699c >>Content-Type: text/html; charset=ISO-8859-1 >>Content-Transfer-Encoding: quoted-printable >> >>
For the question of "why do routers buffer", Vadim Antonov p= >>rovided the following rationale (>nog/0204/0298.html">http://www.irbs.net/internet/nanog/0204/0298.html)<= >>/div> >>
----------------------------------------------------------------------= >>---------------------------------------------------------------------------= >>--------
Well, so far nobody provided a valid explanation for the= >> necessity of
>>buffering in routers (and any other stochastically multiplexing devices). <= >>br>
The real reason for having buffers is the fact that information abou= >>t
congestions takes some time to propagate. (In TCP/IP congestion are <= >>br> >>detected by seeing lost packets).

If buffers are not sufficient to = >>hold packets until TCP speakers see
congestion and slow down, the syste= >>m becomes unstable. Buffers are,
essentially, inertial elements in the = >>delayed negative-feedback control
>>loop. Insufficient inertia causes oscillations in such systems, which is >r>sometimes useful, but in case of networks leads to decreased througoutput= >>
because the wire is utilized fully only at upswings and underutilized = >>on
>>downswings (collapsed TCP windows aggravate the effect futher).

Con= >>sequently, the holding capacity of buffers should be sufficient to
brin= >>g the inertia of the system up to the reaction time of the negative
>>feedback (this is a simplification, of course). This reaction time is
a= >>bout one round-trip time for end-to-end packets.

In real networks, = >>the RTTs differ for different paths, so some
"characteristic"= >> RTT is used. So, to hold packets until TCPs slow down a
>>router needs cRTT * BW bits of buffer memory (where BW is the speed of the = >>
interface). This rule can be somewhat relaxed if congestion control loo= >>p
is engaged proactively before congestion occured (by using Random Ear= >>ly
>>Detection), but not much.

Even with properly sized buffers, session= >>s with longer RTTs suffer from
congestions disproportionately because T= >>CPs on the ends never get enough
time to recover fully (i.e. to bring w= >>indows to large enough size to
>>maintain steady stream of packets), while small-RTT sessions recover
qu= >>ickly, and, therefore, get bigger share of bandwidth. But I'm
digre= >>ssing :)

--vadim
-----------------------------------------------= >>---------------------------------------------------------------------------= >>------=A0
>>

My question to the group is : what other reasons not me= >>ntioned here can be reasons to buffer in a router?

>>-Paddy=A0
>> >>--001e680f1b406effbc046cb8699c-- cheers jon From dpreed at reed.com Sat Jun 20 17:41:20 2009 From: dpreed at reed.com (David P. Reed) Date: Sat, 20 Jun 2009 20:41:20 -0400 Subject: [e2e] Why Buffering? In-Reply-To: <4A3C6428.30002@dcrocker.net> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> Message-ID: <4A3D81B0.6070704@reed.com> Dave - This is variously known as Little's Theorem or Little's Lemma. The general pattern is true for many stochastic arrival processes into queues. It precedes Kleinrock, and belongs to queueing theory. I continue to be shocked, and dismayed, at the number of practicing protocol designers who have never learned (or even *studied* without learning) basic queueing theory. In my opinion, one cannot be qualified to speak about protocol engineering without working knowledge of queueing theory, control theory, and information theory. Yet most CS depts. fail their students by completely ignoring these disciplines in favor of teaching network protocols as a course in bit-field layouts. One can actually get a Ph.D. in Computer Networking without ever studying or using these important mathematical tools. On 06/20/2009 12:23 AM, Dave CROCKER wrote: > > > Paddy Ganti wrote: >> The real reason for having buffers is the fact that information about >> congestions takes some time to propagate. (In TCP/IP congestion are >> detected by seeing lost packets). > > > In the late 70s and 80s, Kleinrock gave a rather simple explanation > for using queuing (buffering) that I think is compatible with > Antonov's point: > > After a lengthy and detailed introduction, he put up a graph of > throughput (x) versus delay (y). It had a very shallow increase until > hitting a very sharp knee and then was almost vertical. > > He observed that before the knee, you don't need queuing because > you don't have any congestion. And after the knee, queuing doesn't > help because you simply don't have enough capacity. > > Queuing is for transient problems rather than an excessive average: > The knee of the curve. > > d/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090620/fe95b3ca/attachment.html From lachlan.andrew at gmail.com Sat Jun 20 18:58:08 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sat, 20 Jun 2009 18:58:08 -0700 Subject: [e2e] Why Buffering? In-Reply-To: <4A3D81B0.6070704@reed.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> Message-ID: Greetings David and Dave, 2009/6/20 David P. Reed : > Dave - This is variously known as Little's Theorem or Little's Lemma. The > general pattern is true for many stochastic arrival processes into queues. > It precedes Kleinrock, and belongs to queueing theory. > > I continue to be shocked, and dismayed, at the number of practicing protocol > designers who have never learned (or even *studied* without learning) basic > queueing theory. I agree that knowing queueing theory is very useful. Note that queueing theory also tells us that closed queueing networks (like sources running TCP / window flow control) behave very differently from open queueing networks. [There was recently a thread about whether Poisson arrivals are relevant. *Packet* arrivals are definitely not Poisson, even though session (and sometimes flow) arrivals can usefully be modelled as such.] Stochastic variability is the reason we need to have *some* buffering, but today's large buffers are basically there because of (a) standard TCP's outdated congestion control (b) the need in some situations for a small number of TCP flow to get high throuhput. As a complement to the Stanford work, I highly recommend the INFOCOM 2008 paper "Impact of file arrivals and departures on buffer sizing in core routers", by A Lakshmikantha, R Srikant, C Beck. It suggests that, with the current (broken?) TCP algorithm, some edge routers need much bigger buffers than the Stanford group recommend, while core routers typically only need small buffers to absorb the stochastic fluctuations already mentioned in this thread. If we fix TCP (no longer halve windows), then we don't need large buffers anywhere. In response to Paddy's post, large buffers are definitely not needed for stability. In fact, having large full buffers increases the RTT, which typically degrades stability (depending on your definition of "stable"). Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From detlef.bosau at web.de Sun Jun 21 02:33:50 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 21 Jun 2009 11:33:50 +0200 Subject: [e2e] Why Buffering? In-Reply-To: <4A3D81B0.6070704@reed.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> Message-ID: <4A3DFE7E.1040000@web.de> David P. Reed wrote: > Dave - This is variously known as Little's Theorem or Little's Lemma. > The general pattern is true for many stochastic arrival processes > into queues. It precedes Kleinrock, and belongs to queueing theory. Little's Theorem can be easily applied in wired networks where a link's capacity is easily expressed as "latency throghput product", often referred to as "latency bandwidth product" which is in fact a bit sloppy. The situation becomes a bit more complicated in wireless networks, particularly WWAN, where the preconditions for Little's Theorem may not hold, particularly the service time may not be stationary or stable. I sometimes wonder about papers who claim quite impressive "latency bandwidth products" for wireless networks - and actually the authors simply miss the fact that the transportation system is highly occupied by local retransmissions and that we have a relationship between average service, average throughput and the average amount of data being in flight. I even remember a paper which claims latency bandwidth products for GPRS in the range of MBytes IIRC. At a first glance, I wondered where this huge amount of data would fit onto the air interface ;-) So, we should be extremely careful in applying Little's Theorem on WWAN. As a consequence, we should even reconsider approaches like packet pair, packet train and the like and whether they really hold in WWAN or similar networks with highly volatile line conditions. Detlef -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From arunv at ee.unsw.edu.au Sun Jun 21 02:58:07 2009 From: arunv at ee.unsw.edu.au (Arun Vishwanath) Date: Sun, 21 Jun 2009 19:58:07 +1000 Subject: [e2e] Why Buffering? In-Reply-To: <4A3DFE7E.1040000@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> Message-ID: <4A3E042F.2070006@ee.unsw.edu.au> Hello Everyone, We recently published a survey paper on the topic of router buffer sizing titled "Perspectives on Router Buffer Sizing: Recent Results and Open Problems". This appeared in the ACM SIGCOMM Computer Communication Review Editorial Zone, vol. 39, no. 2, April 2009. This is just for your kind information. Thanks -Arun Detlef Bosau wrote: > David P. Reed wrote: >> Dave - This is variously known as Little's Theorem or Little's Lemma. >> The general pattern is true for many stochastic arrival processes >> into queues. It precedes Kleinrock, and belongs to queueing theory. > > Little's Theorem can be easily applied in wired networks where a link's > capacity is easily expressed as "latency throghput product", often > referred to as "latency bandwidth product" which is in fact a bit sloppy. > > The situation becomes a bit more complicated in wireless networks, > particularly WWAN, where the preconditions for Little's Theorem may not > hold, particularly the service time may not be stationary or stable. > > I sometimes wonder about papers who claim quite impressive "latency > bandwidth products" for wireless networks - and actually the authors > simply miss the fact that the transportation system is highly occupied > by local retransmissions and that we have a relationship between average > service, average throughput and the average amount of data being in > flight. > > I even remember a paper which claims latency bandwidth products for GPRS > in the range of MBytes IIRC. > > At a first glance, I wondered where this huge amount of data would fit > onto the air interface ;-) > > So, we should be extremely careful in applying Little's Theorem on WWAN. > As a consequence, we should even reconsider approaches like packet pair, > packet train and the like and whether they really hold in WWAN or > similar networks with highly volatile line conditions. > > Detlef > From lachlan.andrew at gmail.com Sun Jun 21 10:24:48 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sun, 21 Jun 2009 10:24:48 -0700 Subject: [e2e] Why Buffering? In-Reply-To: <4A3DFE7E.1040000@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> Message-ID: 2009/6/21 Detlef Bosau : > David P. Reed wrote: >> >> Dave - This is variously known as Little's Theorem or Little's Lemma. > > Little's Theorem can be easily applied in wired networks where a link's > capacity is easily expressed as "latency throghput product". > > The situation becomes a bit more complicated in wireless networks, > particularly WWAN, where the preconditions for Little's Theorem may not > hold, particularly the service time may not be stationary or stable. Little's law (N = T lambda) holds in basically all scenarios. It doesn't rely on stationarity. It refers to long-run averages, which wash out all of the fluctuations. If the service time were "not stable" in the sense of tending to infinity as time progresses, then it would not apply simply (we'd have infinite T, and either infinite N or zero lambda), but that is not the case in WWANs. If your point is that Little's law doesn't tell us anything about short-term behaviour, then that is certainly true. The only difference between WWANs and wired networks is what we can consider to be "short-term". (If your point is that having the knee depends on more than Little's law, then that is also true...) Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From detlef.bosau at web.de Sun Jun 21 12:23:11 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 21 Jun 2009 21:23:11 +0200 Subject: [e2e] Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> Message-ID: <4A3E889F.4080803@web.de> Lachlan Andrew wrote: > 2009/6/21 Detlef Bosau : > >> David P. Reed wrote: >> >>> Dave - This is variously known as Little's Theorem or Little's Lemma. >>> >> Little's Theorem can be easily applied in wired networks where a link's >> capacity is easily expressed as "latency throghput product". >> >> The situation becomes a bit more complicated in wireless networks, >> particularly WWAN, where the preconditions for Little's Theorem may not >> hold, particularly the service time may not be stationary or stable. >> > > Little's law (N = T lambda) holds in basically all scenarios. It > doesn't rely on stationarity. It refers to long-run averages, which > wash out all of the fluctuations. I'm not quite sure here. At least, to my knowledge the law relies on "rates". http://en.wikipedia.org/wiki/Little%27s_law So, the question is the meaning of the term "rate" here. Now, let's have a look at the original paper. http://www.doc.ic.ac.uk/~uh/for-paul/A%20PROOF%20FOR%20THE%20QUEUING%20FORMULA%20L=lW.%207689045.pdf I may have a closer look to the paper in the next days, but in Little's paper it is said that the formula holds in "steady state processes". > If the service time were "not > stable" in the sense of tending to infinity as time progresses, then > it would not apply simply (we'd have infinite T, and either infinite N > or zero lambda), but that is not the case in WWANs. > > Like on any lossy link, in WWAN we have to choices. Either we guarantee error free delivery along a link with errors, achieved by data retransmission if necessary, - in that case the delivery time can grow beyond all limits. Or we impose an upper limit on the delivery time, as we actually do in order to prevent unlimited head of line blocking, and have a residual possibility for the packet to see corruption. In terms of queuing theory, the packet is not being served in that case. I'm a bit out of practice in queuing theory - but wouldn't this be modeled by an infinite service time for that packet with the inevitable consequence that an average vale for the service time, or an expectation for the service time and hence an average number of customers in the system wouldn't even exist? > If your point is that Little's law doesn't tell us anything about > short-term behaviour, then that is certainly true. Particularly in WWAN, one could hardly be interested in the long term behavior. (Argh.... my Thunderbird uses an American spell checker.... ;-)) Consider a user suffering from multipath fading who moves just a few centimeters from an interference maximum to an interference minimum. I don't think he is interested in the fact that the "over the day average" of his throughput is "high". Actually, I've seen users who used mobiles for IP comunication more than I ever did, and sometimes performed some kind of "Aerobic" to achieve satisfactory throughput while reading or sending mails or the like. > The only > difference between WWANs and wired networks is what we can consider to > be "short-term". > > (If your point is that having the knee depends on more than Little's > law, then that is also true...) > Now, the thought becomes important when we want to adapt a rate controlled source to actual network conditions, e.g. in media streaming or in rate control TCP flavors. (Oh, that spell checker... ;-)) In addition, when a mobile runs into an interference minimum (some guy talked to me about "short time disconnection", which matches the policy of some NO who suspend channels with insufficient SNR from being serviced), there is absolutely no guarantee that this situation will ever change. So, what's an "average" then? (O.k., after some time the mobile's battery will run low - and this will certainly stop the situation, which is of course some kind of "change" ;-)) The other point I wanted to address is the "latency bandwidth product" which is sometimes given or WWAN links and which frequently ignores that for a packet being in service not only the "original copy" is on the link (ignore RLP breakup of packets for simplicity) but the retransmissions require capacity as well. So, either we regard retransmissions as packets being in service as well, or we have to reconsider the term "latency bandwidth product" for WWAN. Detlef -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From huitema at windows.microsoft.com Sun Jun 21 13:49:40 2009 From: huitema at windows.microsoft.com (Christian Huitema) Date: Sun, 21 Jun 2009 20:49:40 +0000 Subject: [e2e] Why Buffering? In-Reply-To: <4A3E889F.4080803@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> <4A3E889F.4080803@web.de> Message-ID: <6B55F0F93C3E9D45AF283313B8D342BA01586B@TK5EX14MBXW653.wingroup.windeploy.ntdev.microsoft.com> > > Little's law (N = T lambda) holds in basically all scenarios. It > > doesn't rely on stationarity. It refers to long-run averages, which > > wash out all of the fluctuations. > > I'm not quite sure here. The law certainly holds on all scenarios, but it is about averages. Remember the statistician who drowned in a pool of water that was 1 inch deep, in average? -- Christian Huitema From detlef.bosau at web.de Sun Jun 21 14:13:03 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 21 Jun 2009 23:13:03 +0200 Subject: [e2e] Why Buffering? In-Reply-To: <6B55F0F93C3E9D45AF283313B8D342BA01586B@TK5EX14MBXW653.wingroup.windeploy.ntdev.microsoft.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> <4A3E889F.4080803@web.de> <6B55F0F93C3E9D45AF283313B8D342BA01586B@TK5EX14MBXW653.wingroup.windeploy.ntdev.microsoft.com> Message-ID: <4A3EA25F.9050106@web.de> Christian Huitema wrote: >>> Little's law (N = T lambda) holds in basically all scenarios. It >>> doesn't rely on stationarity. It refers to long-run averages, which >>> wash out all of the fluctuations. >>> >> I'm not quite sure here. >> > > The law certainly holds on all scenarios, but it is about averages. Remember the statistician who drowned in a pool of water that was 1 inch deep, in average? > > -- Christian Huitema > > Of course. Actually, that shows that mathematical models may be a bit abstract ;-) Actually, for the theorem to describe the reality with extremely long service times, the queue may grow extremely large ;-) On the one hand, this shows that all these mathematical models tend to be an extremely abstract idealization. On the other hand, this is exactly one reason for queuing at all. As Jon pointed out, senders may be asynchronous, there is no synchronization between flows, particularly as we do not have packet schedulers in the Internet in most cases. (There may be exceptions in labs ;-)) So, we need queues to achieve the "average". However, as far as I know, there is some common sense to keep queues small. (And from what I've read, I did not read Len Kleinrock's text book myself but quotations from there, that's one of the lessons from Kleinrock's book. It's simply not feasible to run a system "far beyond the knee", IIRC this can even drastically decrease a system's throughput.) -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From L.Wood at surrey.ac.uk Mon Jun 22 04:23:25 2009 From: L.Wood at surrey.ac.uk (Lloyd Wood) Date: Mon, 22 Jun 2009 12:23:25 +0100 Subject: [e2e] Why Buffering? In-Reply-To: <4A3D81B0.6070704@reed.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> Message-ID: This is why an engineering degree provides better preparation for work in protocol design. It's far more likely to cover those topics. On 21 Jun 2009, at 01:41, David P. Reed wrote: > > I continue to be shocked, and dismayed, at the number of practicing > protocol designers who have never learned (or even *studied* without > learning) basic queueing theory. In my opinion, one cannot be > qualified to speak about protocol engineering without working > knowledge of queueing theory, control theory, and information > theory. Yet most CS depts. fail their students by completely > ignoring these disciplines in favor of teaching network protocols as > a course in bit-field layouts. > > One can actually get a Ph.D. in Computer Networking without ever > studying or using these important mathematical tools. DTN work: http://info.ee.surrey.ac.uk/Personal/L.Wood/saratoga/ From dpreed at reed.com Mon Jun 22 06:23:07 2009 From: dpreed at reed.com (David P. Reed) Date: Mon, 22 Jun 2009 09:23:07 -0400 Subject: [e2e] Why Buffering? In-Reply-To: <4A3DFE7E.1040000@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A3C6428.30002@dcrocker.net> <4A3D81B0.6070704@reed.com> <4A3DFE7E.1040000@web.de> Message-ID: <4A3F85BB.4040607@reed.com> As I noted, Little's theorem (at least its intuitive conclusion) applies to a wide variety of arrival processes and a wide variety of queues. Did I say that Little's theorem is the only knowledge one needs to get from studying queueing theory? Of course not. At least some of this discussion moved on to recognize that Little's theorem does NOT apply very well to arrival processes that involve closed-loop control of packet admission (such as TCP) do not follow Little's theorem, and arrival processes that involve even more complex control systems such as dynamic routing or packet scheduling involving propagation in WLANs or scheduled admission in WWANs. So, as Detlef suggests, we should be careful in saying that Little's theorem should be gospel everywhere, without understanding how it derives from its assumptions. That is why one needs to *learn* queueing theory, not as a cookbook, but as a way of thinking that one can adapt to complex problems. I still remain shocked that the people who designed DOCSIS put many *seconds* of buffering into the downlink and uplink, which get filled (because the DOCSIS modem is the rate limiting device), and the result is that except for FTP, which cares little about latency, DOCSIS modems are *unusable* without implementing a "supervisory" layer on top of them that refuses to use the buffers inserted into the end-to-end path. Clearly this is due to bad decision making on the part of DOCSIS modem designers and deployers - assuming they wanted to support Internet service, rather than degrade it. On 06/21/2009 05:33 AM, Detlef Bosau wrote: > David P. Reed wrote: >> Dave - This is variously known as Little's Theorem or Little's >> Lemma. The general pattern is true for many stochastic arrival >> processes into queues. It precedes Kleinrock, and belongs to >> queueing theory. > > Little's Theorem can be easily applied in wired networks where a > link's capacity is easily expressed as "latency throghput product", > often referred to as "latency bandwidth product" which is in fact a > bit sloppy. > > The situation becomes a bit more complicated in wireless networks, > particularly WWAN, where the preconditions for Little's Theorem may > not hold, particularly the service time may not be stationary or stable. > > I sometimes wonder about papers who claim quite impressive "latency > bandwidth products" for wireless networks - and actually the authors > simply miss the fact that the transportation system is highly occupied > by local retransmissions and that we have a relationship between > average service, average throughput and the average amount of data > being in flight. > > I even remember a paper which claims latency bandwidth products for > GPRS in the range of MBytes IIRC. > > At a first glance, I wondered where this huge amount of data would fit > onto the air interface ;-) > > So, we should be extremely careful in applying Little's Theorem on > WWAN. As a consequence, we should even reconsider approaches like > packet pair, packet train and the like and whether they really hold in > WWAN or similar networks with highly volatile line conditions. > > Detlef > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090622/8780ed87/attachment.html From algold at rnp.br Mon Jun 22 19:38:09 2009 From: algold at rnp.br (Alexandre Grojsgold) Date: Mon, 22 Jun 2009 23:38:09 -0300 Subject: [e2e] RES: Why Buffering? In-Reply-To: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: Paddy, I?m sorry, but I am afraid I have to strongly disagree with Antonov?s view on buffer length needs, and perhaps with most people?s opinion on this list. About buffer sizes on routers, I suggest you look for Nick McKeown?s presentation ?Buffers: How we fell in love with them, and why we need a divorce?. It makes sense to me. IMHO, most of you seem to make confusion between buffers needed at the *ends* of a TCP connection, and buffers needed middle way. At the ends, where congestion is detected, and TCP anti-congestion mechanisms act, large buffers are necessary when you want to obtain a high bit rate over a long round trip delay path. This has nothing to do with buffers within routers. A lot of flows flow through a backbone router. Some of them have large BW, same have very little BW. Some of them have very very far endpoints, some of them come from the host in the other side of the town. The router has no clue on the BW.delay product of each flow, and cannot buffer accordingly ? supposing this could help someway. In his text, Antonov says that buffers add ?inertia?. This does not seem true. In physical systems, inertia is added by mass. Buffers are more like springs. If you think of a car suspension, a buffer too long is like a suspension too soft. Maybe good to isolate from road irregularities, but quite unstable. As buffers grow large, the information about congestion takes longer time to come back to a TCP sender, what makes the system less stable (not more stable), and more prone to oscillatory behavior. My vision of what a router does: what a router sees are output interfaces, with a given bandwidth and some characteristics. Under light traffic, the output queue will be almost always empty, at most with a sending packet and another waiting. If the integration of multiple flows traversing that output interface gets close to the maximum bandwidth of the interface, and the system is close to a stable situation, some probability distribution function must exist that can describe the queue length. Without any mathematical proof, I dare to say that the average queue length in this condition should be of a few (less than 10?) packets. Some fluctuations can occur due to slight changes in traffic load, and since that does not mean ?congestion?, there must be enough buffer to keep these few packets in the game. But when the number of packets waiting to be transmitted rises above a certain limit (20? 40?) packets, this is seen by the router as sign that there are more flows trying to cross that output interface than it can accommodate. So time to start dropping some packets, and make a few transmitting TCP guys to slow down. The buffer must have the proper size to drop packets when there is a probable congestion, and do not drop packets under normal fluctuations. Where does delay considerations get here? They don?t. n Alexandre. De: end2end-interest-bounces at postel.org [mailto:end2end-interest-bounces at postel.org] Em nome de Paddy Ganti Enviada em: sexta-feira, 19 de junho de 2009 16:20 Para: end2end-interest at postel.org Assunto: [e2e] Why Buffering? For the question of "why do routers buffer", Vadim Antonov provided the following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) ---------------------------------------------------------------------------- ---------------------------------------------------------------------------- - Well, so far nobody provided a valid explanation for the necessity of buffering in routers (and any other stochastically multiplexing devices). The real reason for having buffers is the fact that information about congestions takes some time to propagate. (In TCP/IP congestion are detected by seeing lost packets). If buffers are not sufficient to hold packets until TCP speakers see congestion and slow down, the system becomes unstable. Buffers are, essentially, inertial elements in the delayed negative-feedback control loop. Insufficient inertia causes oscillations in such systems, which is sometimes useful, but in case of networks leads to decreased througoutput because the wire is utilized fully only at upswings and underutilized on downswings (collapsed TCP windows aggravate the effect futher). Consequently, the holding capacity of buffers should be sufficient to bring the inertia of the system up to the reaction time of the negative feedback (this is a simplification, of course). This reaction time is about one round-trip time for end-to-end packets. In real networks, the RTTs differ for different paths, so some "characteristic" RTT is used. So, to hold packets until TCPs slow down a router needs cRTT * BW bits of buffer memory (where BW is the speed of the interface). This rule can be somewhat relaxed if congestion control loop is engaged proactively before congestion occured (by using Random Early Detection), but not much. Even with properly sized buffers, sessions with longer RTTs suffer from congestions disproportionately because TCPs on the ends never get enough time to recover fully (i.e. to bring windows to large enough size to maintain steady stream of packets), while small-RTT sessions recover quickly, and, therefore, get bigger share of bandwidth. But I'm digressing :) --vadim ---------------------------------------------------------------------------- ---------------------------------------------------- My question to the group is : what other reasons not mentioned here can be reasons to buffer in a router? -Paddy -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090622/a8f7f824/attachment.html From Jon.Crowcroft at cl.cam.ac.uk Tue Jun 23 04:15:02 2009 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Tue, 23 Jun 2009 12:15:02 +0100 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: the naive argument for bw*delay buffer was that AIMD involves rate halving so by corollary, just before the rate halved it was twice the bottleneck's capacity operating point, and to avoid wasting the bw*delay (-1 lost) of packets just sent you'd like to have them buffered. if the bottleneck is usually at the edge (e.g. the router in the home or just after the dslam and the "core" (some fictional middle of the internet) never sees packet loss, then you are right... if there's lots of flows and the individual flows are all even reasonably unsynchronised in their AIMD phase (very good chance due to random local perturbation) (law of large number argument says..) then you are right (well, nick mckeown's right:) if you use some smart queue management and ECN then you're right if most flows stop before reaching their operating point your right (well actually, there's a way to be wrong in a quite ghastly way if you have many sy cnhronised flows in "slow start", heading thru the same bottleneck... but it ought to be quite a rare event - like black swans and mrket meltdowns:-) In missive , "Alexandre Grojsgold" typed: >>I=92m sorry, but I am afraid I have to strongly disagree with = >>Antonov=B4s view >>on buffer length needs, and perhaps with most people=B4s opinion on this = >>list. >> >>=20 >> >>About buffer sizes on routers, I suggest you look for Nick McKeown=92s >>presentation =93Buffers: How we fell in love with them, and why we need = >>a >>divorce=94. It makes sense to me. >> >>=20 >> >>IMHO, most of you seem to make confusion between buffers needed at the >>*ends* of a TCP connection, and buffers needed middle way. At the ends, >>where congestion is detected, and TCP anti-congestion mechanisms act, = >>large >>buffers are necessary when you want to obtain a high bit rate over a = >>long >>round trip delay path. >> >>=20 >> >>This has nothing to do with buffers within routers. A lot of flows flow >>through a backbone router. Some of them have large BW, same have very = >>little >>BW. Some of them have very very far endpoints, some of them come from = >>the >>host in the other side of the town. The router has no clue on the = >>BW.delay >>product of each flow, and cannot buffer accordingly =96 supposing this = >>could >>help someway. >> >>=20 >> >>In his text, Antonov says that buffers add =93inertia=94. This does not = >>seem >>true. In physical systems, inertia is added by mass. Buffers are more = >>like >>springs. If you think of a car suspension, a buffer too long is like a >>suspension too soft. Maybe good to isolate from road irregularities, but >>quite unstable. >> >>=20 >> >>As buffers grow large, the information about congestion takes longer = >>time to >>come back to a TCP sender, what makes the system less stable (not more >>stable), and more prone to oscillatory behavior. >> >>=20 >> >>My vision of what a router does: what a router sees are output = >>interfaces, >>with a given bandwidth and some characteristics. Under light traffic, = >>the >>output queue will be almost always empty, at most with a sending packet = >>and >>another waiting. If the integration of multiple flows traversing that = >>output >>interface gets close to the maximum bandwidth of the interface, and the >>system is close to a stable situation, some probability distribution >>function must exist that can describe the queue length. Without any >>mathematical proof, I dare to say that the average queue length in this >>condition should be of a few (less than 10?) packets. Some fluctuations = >>can >>occur due to slight changes in traffic load, and since that does not = >>mean >>=93congestion=94, there must be enough buffer to keep these few packets = >>in the >>game. But when the number of packets waiting to be transmitted rises = >>above a >>certain limit (20? 40?) packets, this is seen by the router as sign = >>that >>there are more flows trying to cross that output interface than it can >>accommodate. So =85 time to start dropping some packets, and make a few >>transmitting TCP guys to slow down. The buffer must have the proper = >>size >>to drop packets when there is a probable congestion, and do not drop = >>packets >>under normal fluctuations. Where does delay considerations get here? = >>They >>don=92t. >> >>=20 >> >>n Alexandre. >> >>=20 >> >>=20 >> >>=20 >> >>=20 >> >>De: end2end-interest-bounces at postel.org >>[mailto:end2end-interest-bounces at postel.org] Em nome de Paddy Ganti >>Enviada em: sexta-feira, 19 de junho de 2009 16:20 >>Para: end2end-interest at postel.org >>Assunto: [e2e] Why Buffering? >> >>=20 >> >>For the question of "why do routers buffer", Vadim Antonov provided the >>following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) >> >>-------------------------------------------------------------------------= >>--- >>-------------------------------------------------------------------------= >>--- >>- >> >>Well, so far nobody provided a valid explanation for the necessity of=20 >>buffering in routers (and any other stochastically multiplexing = >>devices).=20 >> >>The real reason for having buffers is the fact that information about=20 >>congestions takes some time to propagate. (In TCP/IP congestion are=20 >>detected by seeing lost packets).=20 >> >>If buffers are not sufficient to hold packets until TCP speakers see=20 >>congestion and slow down, the system becomes unstable. Buffers are,=20 >>essentially, inertial elements in the delayed negative-feedback control=20 >>loop. Insufficient inertia causes oscillations in such systems, which is = >> >>sometimes useful, but in case of networks leads to decreased = >>througoutput=20 >>because the wire is utilized fully only at upswings and underutilized on = >> >>downswings (collapsed TCP windows aggravate the effect futher).=20 >> >>Consequently, the holding capacity of buffers should be sufficient to=20 >>bring the inertia of the system up to the reaction time of the negative=20 >>feedback (this is a simplification, of course). This reaction time is=20 >>about one round-trip time for end-to-end packets.=20 >> >>In real networks, the RTTs differ for different paths, so some=20 >>"characteristic" RTT is used. So, to hold packets until TCPs slow down a = >> >>router needs cRTT * BW bits of buffer memory (where BW is the speed of = >>the=20 >>interface). This rule can be somewhat relaxed if congestion control loop = >> >>is engaged proactively before congestion occured (by using Random Early=20 >>Detection), but not much.=20 >> >>Even with properly sized buffers, sessions with longer RTTs suffer from=20 >>congestions disproportionately because TCPs on the ends never get enough = >> >>time to recover fully (i.e. to bring windows to large enough size to=20 >>maintain steady stream of packets), while small-RTT sessions recover=20 >>quickly, and, therefore, get bigger share of bandwidth. But I'm=20 >>digressing :)=20 >> >>--vadim >>-------------------------------------------------------------------------= >>--- >>----------------------------------------------------=20 >> >>=20 >> >>My question to the group is : what other reasons not mentioned here can = >>be >>reasons to buffer in a router? >> >>=20 >> >>-Paddy=20 >> >> >>------=_NextPart_000_0050_01C9F392.7FF50A70 >>Content-Type: text/html; >> charset="iso-8859-1" >>Content-Transfer-Encoding: quoted-printable >> >>>xmlns:o=3D"urn:schemas-microsoft-com:office:office" = >>xmlns:w=3D"urn:schemas-microsoft-com:office:word" = >>xmlns:x=3D"urn:schemas-microsoft-com:office:excel" = >>xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = >>xmlns=3D"http://www.w3.org/TR/REC-html40"> >> >> >>>charset=3Diso-8859-1"> >> >> >> >> >> >> >> >>
>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>Paddy,

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>I’m sorry, but I am afraid I have to strongly = >>disagree >>with Antonov=B4s view on buffer length needs, and perhaps with most = >>people=B4s >>opinion on this list.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>About buffer sizes on routers, I suggest you look for = >>Nick >>McKeown’s presentation “Buffers: How we fell in love = >>with >>them, and why we need a divorce”. It makes sense to = >>me.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>IMHO, most of you seem to make confusion between buffers = >>needed >>at the *ends* of a TCP connection, and buffers needed middle way.=A0 At = >>the ends, >>where congestion is detected, and TCP anti-congestion mechanisms act, = >>large >>buffers are necessary when you want to obtain a high bit rate over a = >>long round >>trip delay path.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>This has nothing to do with buffers within = >>routers. A lot >>of flows flow through a backbone router. Some of them have large BW, = >>same have >>very little BW. Some of them have very very far endpoints, some of them = >>come >>from the host in the other side of the town. The router has no clue on = >>the >>BW.delay product of each flow, and cannot buffer accordingly – = >>supposing >>this could help someway.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>In his text, Antonov says that buffers add >>“inertia”. This does not seem true. In physical systems, = >>inertia is >>added by mass. Buffers are more like springs. If you think of a car = >>suspension, >>a buffer too long is like a suspension too soft. Maybe good to isolate = >>from >>road irregularities, but quite unstable.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>As buffers grow large, the information about congestion = >>takes >>longer time to come back to a TCP sender, what makes the system less = >>stable >>(not more stable), and more prone to oscillatory = >>behavior.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>My vision of what a router does: what a router sees are = >>output >>interfaces, with a=A0 given bandwidth and some characteristics. Under = >>light >>traffic, the output queue will be almost always empty, at most with a = >>sending >>packet and another waiting. If the integration of multiple flows = >>traversing >>that output interface gets close to the maximum bandwidth of the = >>interface, and >>the system is=A0 close to a stable situation, some probability = >>distribution >>function must exist that can describe the queue length. Without any >>mathematical proof, I dare to say that the average queue length in this >>condition should be of a few (less than 10?) packets. Some fluctuations = >>can >>occur due to slight changes in traffic load, and since that does not = >>mean >>“congestion”, there must be enough buffer to keep these few = >>packets >>in the game. But when the number of packets waiting to be transmitted = >>rises >>above a certain limit (20? 40?) packets, this is seen=A0 by the router = >>as sign >>that there are more flows trying to cross that output interface than it = >>can >>accommodate. So … time to start dropping some packets, and make a = >>few >>transmitting TCP guys to slow down. The buffer must=A0 have the = >>proper=A0 size to drop >>packets when there is a probable congestion, and do not drop packets = >>under >>normal fluctuations. Where does delay considerations get here? They >>don’t.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>level1 lfo1'>>lang=3DEN-US = >>style=3D'font-size:11.0pt;font-family:Wingdings;color:#1F497D'>>style=3D'mso-list:Ignore'>n>Roman"'>  >lang=3DEN-US = >>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>Alexandre.

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'>=A0

>> >>

>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; >>color:#1F497D'> 

>> >>
>0cm 4.0pt'> >> >>
>> >>
>0cm 0cm 0cm'> >> >>

>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>De:>b>>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> = >>end2end-interest-bounces at postel.org >>[mailto:end2end-interest-bounces at postel.org] Em nome de Paddy = >>Ganti
>>Enviada em: sexta-feira, 19 de junho de 2009 16:20
>>Para: end2end-interest at postel.org
>>Assunto: [e2e] Why Buffering?

>> >>
>> >>
>> >>

 

>> >>
>> >>

For the question of "why do routers = >>buffer", Vadim >>Antonov provided the following rationale (>href=3D"http://www.irbs.net/internet/nanog/0204/0298.html">http://www.irb= >>s.net/internet/nanog/0204/0298.html)

>> >>
>> >>
>> >>

>class=3DMsoNormal>-------------------------------------------------------= >>-------------------------------------------------------------------------= >>-------------------------

>> >>
>> >>
>> >>

Well, so far nobody provided a valid explanation = >>for the >>necessity of
>>buffering in routers (and any other stochastically multiplexing = >>devices).
>>
>>The real reason for having buffers is the fact that information about = >>
>>congestions takes some time to propagate. (In TCP/IP congestion are
>>detected by seeing lost packets).
>>
>>If buffers are not sufficient to hold packets until TCP speakers see = >>
>>congestion and slow down, the system becomes unstable. Buffers are,
>>essentially, inertial elements in the delayed negative-feedback control = >>
>>loop. Insufficient inertia causes oscillations in such systems, which is = >>
>>sometimes useful, but in case of networks leads to decreased = >>througoutput
>>because the wire is utilized fully only at upswings and underutilized on = >>
>>downswings (collapsed TCP windows aggravate the effect futher).
>>
>>Consequently, the holding capacity of buffers should be sufficient to = >>
>>bring the inertia of the system up to the reaction time of the negative = >>
>>feedback (this is a simplification, of course). This reaction time is = >>
>>about one round-trip time for end-to-end packets.
>>
>>In real networks, the RTTs differ for different paths, so some
>>"characteristic" RTT is used. So, to hold packets until TCPs = >>slow >>down a
>>router needs cRTT * BW bits of buffer memory (where BW is the speed of = >>the
>>interface). This rule can be somewhat relaxed if congestion control loop = >>
>>is engaged proactively before congestion occured (by using Random Early = >>
>>Detection), but not much.
>>
>>Even with properly sized buffers, sessions with longer RTTs suffer from = >>
>>congestions disproportionately because TCPs on the ends never get enough = >>
>>time to recover fully (i.e. to bring windows to large enough size to = >>
>>maintain steady stream of packets), while small-RTT sessions recover = >>
>>quickly, and, therefore, get bigger share of bandwidth. But I'm
>>digressing :)
>>
>>--vadim
>>-------------------------------------------------------------------------= >>------------------------------------------------------- <= >>/p> >> >>

>> >>
>> >>

 

>> >>
>> >>
>> >>

My question to the group is : what other reasons = >>not >>mentioned here can be reasons to buffer in a router?

>> >>
>> >>
>> >>

 

>> >>
>> >>
>> >>

-Paddy 

>> >>
>> >>
>> >>
>> >> >> >> >> >>------=_NextPart_000_0050_01C9F392.7FF50A70-- >> >> cheers jon From detlef.bosau at web.de Tue Jun 23 09:02:39 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Tue, 23 Jun 2009 18:02:39 +0200 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> Message-ID: <4A40FC9F.9060504@web.de> Jon Crowcroft wrote: > the naive argument for bw*delay buffer > was that AIMD involves rate halving > It's not the rate being halved but the congestion window. And I'm always curious to see that some people expect some easy relationship between rate and sending window / congestion window. Maybe, there is one. Than it's certainly not a trivial one. > so by corollary, just before the rate halved > it was twice the bottleneck's capacity > Yes. That's the common rationale. However, Alexandre is correct here: Some router in between has no idea of a path's capacity - at least as this cannot be easily described by some "latency bandwidth product" in some networks. So, the idea of having the "latency bandwidth product" of a path doubled as buffer capacity in a router is a nice one for theoretical papers. But I'm not quite sure what this should mean for reality: Consider a router in Rotterdam which sees both, traffic from Wladiwostok to London, Hamburg to New York and from Den Haag to Santiago to Chile. Which is "the" latency bandwidth product which should be doubled? Surely not a different one for any possible flow. > if there's lots of flows > and the individual flows are all even reasonably > unsynchronised in their AIMD phase > (very good chance due to random local perturbation > > (law of large number argument says..) > then you are right (well, nick mckeown's right:) > > if you use some smart queue management and ECN > then you're right > > if most flows stop before reaching their operating point > your right > (well actually, there's a way to be wrong > in a quite ghastly way if you have many sy cnhronised flows > in "slow start", heading thru the same bottleneck... > but it ought to be quite a rare event - like > black swans and mrket meltdowns:-) > > So, after a long list of scenarios where Alexandre is right: Is there a scenario, where he is wrong? ;-) -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From cai at ECE.UVic.CA Tue Jun 23 16:52:03 2009 From: cai at ECE.UVic.CA (Lin Cai) Date: Tue, 23 Jun 2009 16:52:03 -0700 (PDT) Subject: [e2e] RES: Why Buffering? Message-ID: We can use a simple math model to explain why we need buffer and how large it is necessary. Consider a simple case. Let N TCP flows with round-trip delay R (sec) share a bottleneck link with capacity C (packet/sec). The router is RED-capable with parameter K_p (Drop-Tail queue can be viewed as a special case of RED). Using a fluid-flow model, we can easily derive the system equilibrium point of each flow's congestion window size (W*), and the queue length (q*): W*=RC/N, q*=3N^2/(R^2 C^2 K_p). The first equation tells us, in equilibrium (the ideal case), the total traffic arrival rate (W*N/R) equals the link capacity (C). The 2nd equation tells us, in equilibrium, we still see a queue length proportional to N^2/(R^2 C^2 K_p). If the buffer size is smaller than q*, it means that the TCP flows cannot reach the equilibrum to fully utilize the link capacity. Note that q* actually decrease if we have a larger RC value. Consider the feedback delay of TCP control loop (packet losses need time to be learned by TCP senders), the TCP/RED system very often oscillates around the equilibrium point (TCP senders will overshoot the link capacity a bit, then backoff to the state with W < W*, so on and so forth). With a bit more math involved, we can derive the upper bound of W and q, as a function of N, R, C, K_p. The surprising results include: 1) although the TCP/RED system is more likely to be (asymptotically) unstable with a larger value of R and C (i.e., the system unlikely converges and stays in the equilibrium point forever), the oscillation amplitude of the system actually decreases w.r.t. R C, so the maximum queue length is smaller, i.e., we need less buffer without affecting the TCP throughput. 2) although TCP flows can adapt their sending rates according to the available bandwidth, larger number of flows (N) leads to longer queueing delay in the TCP/RED system and we need a larger buffer size. This might be an issue when more and more P2P applications open up tens and hundreds of parallel TCP connections. Details can be found in the following paper: http://www.ece.uvic.ca/~cai/bounds-of-tcp-icc08.pdf L. Wang, L. Cai, X. Liu, and X. Shen, Practical Stability and Bounds of Heterogeneous AIMD/RED System with Time Delay, Proc. IEEE ICC08, Beijing, China, May 2008. Best Regards, Lin === Lin Cai, E&CE UVic, (250) 721-8691, http://www.ece.uvic.ca/~cai On Tue, 23 Jun 2009, end2end-interest-request at postel.org wrote: > Today's Topics: > > 1. RES: Why Buffering? (Alexandre Grojsgold) > 2. Re: RES: Why Buffering? (Jon Crowcroft) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 22 Jun 2009 23:38:09 -0300 > From: "Alexandre Grojsgold" > Subject: [e2e] RES: Why Buffering? > To: "'Paddy Ganti'" , > Message-ID: > > > Content-Type: text/plain; charset="iso-8859-1" > > Paddy, > > > > I?m sorry, but I am afraid I have to strongly disagree with Antonov?s view > on buffer length needs, and perhaps with most people?s opinion on this list. > > > > About buffer sizes on routers, I suggest you look for Nick McKeown?s > presentation ?Buffers: How we fell in love with them, and why we need a > divorce?. It makes sense to me. > > > > IMHO, most of you seem to make confusion between buffers needed at the > *ends* of a TCP connection, and buffers needed middle way. At the ends, > where congestion is detected, and TCP anti-congestion mechanisms act, large > buffers are necessary when you want to obtain a high bit rate over a long > round trip delay path. > > > > This has nothing to do with buffers within routers. A lot of flows flow > through a backbone router. Some of them have large BW, same have very little > BW. Some of them have very very far endpoints, some of them come from the > host in the other side of the town. The router has no clue on the BW.delay > product of each flow, and cannot buffer accordingly ? supposing this could > help someway. > > > > In his text, Antonov says that buffers add ?inertia?. This does not seem > true. In physical systems, inertia is added by mass. Buffers are more like > springs. If you think of a car suspension, a buffer too long is like a > suspension too soft. Maybe good to isolate from road irregularities, but > quite unstable. > > > > As buffers grow large, the information about congestion takes longer time to > come back to a TCP sender, what makes the system less stable (not more > stable), and more prone to oscillatory behavior. > > > > My vision of what a router does: what a router sees are output interfaces, > with a given bandwidth and some characteristics. Under light traffic, the > output queue will be almost always empty, at most with a sending packet and > another waiting. If the integration of multiple flows traversing that output > interface gets close to the maximum bandwidth of the interface, and the > system is close to a stable situation, some probability distribution > function must exist that can describe the queue length. Without any > mathematical proof, I dare to say that the average queue length in this > condition should be of a few (less than 10?) packets. Some fluctuations can > occur due to slight changes in traffic load, and since that does not mean > ?congestion?, there must be enough buffer to keep these few packets in the > game. But when the number of packets waiting to be transmitted rises above a > certain limit (20? 40?) packets, this is seen by the router as sign that > there are more flows trying to cross that output interface than it can > accommodate. So ? time to start dropping some packets, and make a few > transmitting TCP guys to slow down. The buffer must have the proper size > to drop packets when there is a probable congestion, and do not drop packets > under normal fluctuations. Where does delay considerations get here? They > don?t. > > > > n Alexandre. > > > > > > > > > > De: end2end-interest-bounces at postel.org > [mailto:end2end-interest-bounces at postel.org] Em nome de Paddy Ganti > Enviada em: sexta-feira, 19 de junho de 2009 16:20 > Para: end2end-interest at postel.org > Assunto: [e2e] Why Buffering? > > > > For the question of "why do routers buffer", Vadim Antonov provided the > following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) > > ---------------------------------------------------------------------------- > ---------------------------------------------------------------------------- > - > > Well, so far nobody provided a valid explanation for the necessity of > buffering in routers (and any other stochastically multiplexing devices). > > The real reason for having buffers is the fact that information about > congestions takes some time to propagate. (In TCP/IP congestion are > detected by seeing lost packets). > > If buffers are not sufficient to hold packets until TCP speakers see > congestion and slow down, the system becomes unstable. Buffers are, > essentially, inertial elements in the delayed negative-feedback control > loop. Insufficient inertia causes oscillations in such systems, which is > sometimes useful, but in case of networks leads to decreased througoutput > because the wire is utilized fully only at upswings and underutilized on > downswings (collapsed TCP windows aggravate the effect futher). > > Consequently, the holding capacity of buffers should be sufficient to > bring the inertia of the system up to the reaction time of the negative > feedback (this is a simplification, of course). This reaction time is > about one round-trip time for end-to-end packets. > > In real networks, the RTTs differ for different paths, so some > "characteristic" RTT is used. So, to hold packets until TCPs slow down a > router needs cRTT * BW bits of buffer memory (where BW is the speed of the > interface). This rule can be somewhat relaxed if congestion control loop > is engaged proactively before congestion occured (by using Random Early > Detection), but not much. > > Even with properly sized buffers, sessions with longer RTTs suffer from > congestions disproportionately because TCPs on the ends never get enough > time to recover fully (i.e. to bring windows to large enough size to > maintain steady stream of packets), while small-RTT sessions recover > quickly, and, therefore, get bigger share of bandwidth. But I'm > digressing :) > > --vadim > ---------------------------------------------------------------------------- > ---------------------------------------------------- > > > > My question to the group is : what other reasons not mentioned here can be > reasons to buffer in a router? > > > > -Paddy > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > http://mailman.postel.org/pipermail/end2end-interest/attachments/20090622/a8f7f824/attachment-0001.html > > ------------------------------ > > Message: 2 > Date: Tue, 23 Jun 2009 12:15:02 +0100 > From: Jon Crowcroft > Subject: Re: [e2e] RES: Why Buffering? > To: "Alexandre Grojsgold" > Cc: end2end-interest at postel.org > Message-ID: > Content-Type: text/plain; charset="us-ascii" > > the naive argument for bw*delay buffer > was that AIMD involves rate halving > so by corollary, just before the rate halved > it was twice the bottleneck's capacity > operating point, and to avoid > wasting the bw*delay (-1 lost) of packets just sent > you'd like to have them buffered. > > if the bottleneck is usually at the edge > (e.g. the router in the home or just after the dslam > and the "core" (some fictional middle of the internet) > never sees packet loss, > then you are right... > > if there's lots of flows > and the individual flows are all even reasonably > unsynchronised in their AIMD phase > (very good chance due to random local perturbation) > (law of large number argument says..) > then you are right (well, nick mckeown's right:) > > if you use some smart queue management and ECN > then you're right > > if most flows stop before reaching their operating point > your right > (well actually, there's a way to be wrong > in a quite ghastly way if you have many sy cnhronised flows > in "slow start", heading thru the same bottleneck... > but it ought to be quite a rare event - like > black swans and mrket meltdowns:-) > > In missive > , > "Alexandre > Grojsgold" typed: > > >>I=92m sorry, but I am afraid I have to strongly disagree with = > >>Antonov=B4s view > >>on buffer length needs, and perhaps with most people=B4s opinion on this = > >>list. > >> > >>=20 > >> > >>About buffer sizes on routers, I suggest you look for Nick McKeown=92s > >>presentation =93Buffers: How we fell in love with them, and why we need = > >>a > >>divorce=94. It makes sense to me. > >> > >>=20 > >> > >>IMHO, most of you seem to make confusion between buffers needed at the > >>*ends* of a TCP connection, and buffers needed middle way. At the ends, > >>where congestion is detected, and TCP anti-congestion mechanisms act, = > >>large > >>buffers are necessary when you want to obtain a high bit rate over a = > >>long > >>round trip delay path. > >> > >>=20 > >> > >>This has nothing to do with buffers within routers. A lot of flows flow > >>through a backbone router. Some of them have large BW, same have very = > >>little > >>BW. Some of them have very very far endpoints, some of them come from = > >>the > >>host in the other side of the town. The router has no clue on the = > >>BW.delay > >>product of each flow, and cannot buffer accordingly =96 supposing this = > >>could > >>help someway. > >> > >>=20 > >> > >>In his text, Antonov says that buffers add =93inertia=94. This does not = > >>seem > >>true. In physical systems, inertia is added by mass. Buffers are more = > >>like > >>springs. If you think of a car suspension, a buffer too long is like a > >>suspension too soft. Maybe good to isolate from road irregularities, but > >>quite unstable. > >> > >>=20 > >> > >>As buffers grow large, the information about congestion takes longer = > >>time to > >>come back to a TCP sender, what makes the system less stable (not more > >>stable), and more prone to oscillatory behavior. > >> > >>=20 > >> > >>My vision of what a router does: what a router sees are output = > >>interfaces, > >>with a given bandwidth and some characteristics. Under light traffic, = > >>the > >>output queue will be almost always empty, at most with a sending packet = > >>and > >>another waiting. If the integration of multiple flows traversing that = > >>output > >>interface gets close to the maximum bandwidth of the interface, and the > >>system is close to a stable situation, some probability distribution > >>function must exist that can describe the queue length. Without any > >>mathematical proof, I dare to say that the average queue length in this > >>condition should be of a few (less than 10?) packets. Some fluctuations = > >>can > >>occur due to slight changes in traffic load, and since that does not = > >>mean > >>=93congestion=94, there must be enough buffer to keep these few packets = > >>in the > >>game. But when the number of packets waiting to be transmitted rises = > >>above a > >>certain limit (20? 40?) packets, this is seen by the router as sign = > >>that > >>there are more flows trying to cross that output interface than it can > >>accommodate. So =85 time to start dropping some packets, and make a few > >>transmitting TCP guys to slow down. The buffer must have the proper = > >>size > >>to drop packets when there is a probable congestion, and do not drop = > >>packets > >>under normal fluctuations. Where does delay considerations get here? = > >>They > >>don=92t. > >> > >>=20 > >> > >>n Alexandre. > >> > >>=20 > >> > >>=20 > >> > >>=20 > >> > >>=20 > >> > >>De: end2end-interest-bounces at postel.org > >>[mailto:end2end-interest-bounces at postel.org] Em nome de Paddy Ganti > >>Enviada em: sexta-feira, 19 de junho de 2009 16:20 > >>Para: end2end-interest at postel.org > >>Assunto: [e2e] Why Buffering? > >> > >>=20 > >> > >>For the question of "why do routers buffer", Vadim Antonov provided the > >>following rationale (http://www.irbs.net/internet/nanog/0204/0298.html) > >> > >>-------------------------------------------------------------------------= > >>--- > >>-------------------------------------------------------------------------= > >>--- > >>- > >> > >>Well, so far nobody provided a valid explanation for the necessity of=20 > >>buffering in routers (and any other stochastically multiplexing = > >>devices).=20 > >> > >>The real reason for having buffers is the fact that information about=20 > >>congestions takes some time to propagate. (In TCP/IP congestion are=20 > >>detected by seeing lost packets).=20 > >> > >>If buffers are not sufficient to hold packets until TCP speakers see=20 > >>congestion and slow down, the system becomes unstable. Buffers are,=20 > >>essentially, inertial elements in the delayed negative-feedback control=20 > >>loop. Insufficient inertia causes oscillations in such systems, which is = > >> > >>sometimes useful, but in case of networks leads to decreased = > >>througoutput=20 > >>because the wire is utilized fully only at upswings and underutilized on = > >> > >>downswings (collapsed TCP windows aggravate the effect futher).=20 > >> > >>Consequently, the holding capacity of buffers should be sufficient to=20 > >>bring the inertia of the system up to the reaction time of the negative=20 > >>feedback (this is a simplification, of course). This reaction time is=20 > >>about one round-trip time for end-to-end packets.=20 > >> > >>In real networks, the RTTs differ for different paths, so some=20 > >>"characteristic" RTT is used. So, to hold packets until TCPs slow down a = > >> > >>router needs cRTT * BW bits of buffer memory (where BW is the speed of = > >>the=20 > >>interface). This rule can be somewhat relaxed if congestion control loop = > >> > >>is engaged proactively before congestion occured (by using Random Early=20 > >>Detection), but not much.=20 > >> > >>Even with properly sized buffers, sessions with longer RTTs suffer from=20 > >>congestions disproportionately because TCPs on the ends never get enough = > >> > >>time to recover fully (i.e. to bring windows to large enough size to=20 > >>maintain steady stream of packets), while small-RTT sessions recover=20 > >>quickly, and, therefore, get bigger share of bandwidth. But I'm=20 > >>digressing :)=20 > >> > >>--vadim > >>-------------------------------------------------------------------------= > >>--- > >>----------------------------------------------------=20 > >> > >>=20 > >> > >>My question to the group is : what other reasons not mentioned here can = > >>be > >>reasons to buffer in a router? > >> > >>=20 > >> > >>-Paddy=20 > >> > >> > >>------=_NextPart_000_0050_01C9F392.7FF50A70 > >>Content-Type: text/html; > >> charset="iso-8859-1" > >>Content-Transfer-Encoding: quoted-printable > >> > >> >>xmlns:o=3D"urn:schemas-microsoft-com:office:office" = > >>xmlns:w=3D"urn:schemas-microsoft-com:office:word" = > >>xmlns:x=3D"urn:schemas-microsoft-com:office:excel" = > >>xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = > >>xmlns=3D"http://www.w3.org/TR/REC-html40"> > >> > >> > >> >>charset=3Diso-8859-1"> > >> > >> > >> > >> > >> > >> > >> > >>
> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>Paddy,

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>I’m sorry, but I am afraid I have to strongly = > >>disagree > >>with Antonov=B4s view on buffer length needs, and perhaps with most = > >>people=B4s > >>opinion on this list.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>About buffer sizes on routers, I suggest you look for = > >>Nick > >>McKeown’s presentation “Buffers: How we fell in love = > >>with > >>them, and why we need a divorce”. It makes sense to = > >>me.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>IMHO, most of you seem to make confusion between buffers = > >>needed > >>at the *ends* of a TCP connection, and buffers needed middle way.=A0 At = > >>the ends, > >>where congestion is detected, and TCP anti-congestion mechanisms act, = > >>large > >>buffers are necessary when you want to obtain a high bit rate over a = > >>long round > >>trip delay path.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>This has nothing to do with buffers within = > >>routers. A lot > >>of flows flow through a backbone router. Some of them have large BW, = > >>same have > >>very little BW. Some of them have very very far endpoints, some of them = > >>come > >>from the host in the other side of the town. The router has no clue on = > >>the > >>BW.delay product of each flow, and cannot buffer accordingly – = > >>supposing > >>this could help someway.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>In his text, Antonov says that buffers add > >>“inertia”. This does not seem true. In physical systems, = > >>inertia is > >>added by mass. Buffers are more like springs. If you think of a car = > >>suspension, > >>a buffer too long is like a suspension too soft. Maybe good to isolate = > >>from > >>road irregularities, but quite unstable.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>As buffers grow large, the information about congestion = > >>takes > >>longer time to come back to a TCP sender, what makes the system less = > >>stable > >>(not more stable), and more prone to oscillatory = > >>behavior.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>My vision of what a router does: what a router sees are = > >>output > >>interfaces, with a=A0 given bandwidth and some characteristics. Under = > >>light > >>traffic, the output queue will be almost always empty, at most with a = > >>sending > >>packet and another waiting. If the integration of multiple flows = > >>traversing > >>that output interface gets close to the maximum bandwidth of the = > >>interface, and > >>the system is=A0 close to a stable situation, some probability = > >>distribution > >>function must exist that can describe the queue length. Without any > >>mathematical proof, I dare to say that the average queue length in this > >>condition should be of a few (less than 10?) packets. Some fluctuations = > >>can > >>occur due to slight changes in traffic load, and since that does not = > >>mean > >>“congestion”, there must be enough buffer to keep these few = > >>packets > >>in the game. But when the number of packets waiting to be transmitted = > >>rises > >>above a certain limit (20? 40?) packets, this is seen=A0 by the router = > >>as sign > >>that there are more flows trying to cross that output interface than it = > >>can > >>accommodate. So … time to start dropping some packets, and make a = > >>few > >>transmitting TCP guys to slow down. The buffer must=A0 have the = > >>proper=A0 size to drop > >>packets when there is a probable congestion, and do not drop packets = > >>under > >>normal fluctuations. Where does delay considerations get here? They > >>don’t.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>level1 lfo1'> >>lang=3DEN-US = > >>style=3D'font-size:11.0pt;font-family:Wingdings;color:#1F497D'> >>style=3D'mso-list:Ignore'>n >>Roman"'>  >>lang=3DEN-US = > >>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>Alexandre.

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'>=A0

> >> > >>

>>style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif"; > >>color:#1F497D'> 

> >> > >>
>>0cm 4.0pt'> > >> > >>
> >> > >>
>>0cm 0cm 0cm'> > >> > >>

>>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>De: >>b> >>style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> = > >>end2end-interest-bounces at postel.org > >>[mailto:end2end-interest-bounces at postel.org] Em nome de Paddy = > >>Ganti
> >>Enviada em: sexta-feira, 19 de junho de 2009 16:20
> >>Para: end2end-interest at postel.org
> >>Assunto: [e2e] Why Buffering?

> >> > >>
> >> > >>
> >> > >>

 

> >> > >>
> >> > >>

For the question of "why do routers = > >>buffer", Vadim > >>Antonov provided the following rationale ( >>href=3D"http://www.irbs.net/internet/nanog/0204/0298.html">http://www.irb= > >>s.net/internet/nanog/0204/0298.html)

> >> > >>
> >> > >>
> >> > >>

>>class=3DMsoNormal>-------------------------------------------------------= > >>-------------------------------------------------------------------------= > >>-------------------------

> >> > >>
> >> > >>
> >> > >>

Well, so far nobody provided a valid explanation = > >>for the > >>necessity of
> >>buffering in routers (and any other stochastically multiplexing = > >>devices).
> >>
> >>The real reason for having buffers is the fact that information about = > >>
> >>congestions takes some time to propagate. (In TCP/IP congestion are
> >>detected by seeing lost packets).
> >>
> >>If buffers are not sufficient to hold packets until TCP speakers see = > >>
> >>congestion and slow down, the system becomes unstable. Buffers are,
> >>essentially, inertial elements in the delayed negative-feedback control = > >>
> >>loop. Insufficient inertia causes oscillations in such systems, which is = > >>
> >>sometimes useful, but in case of networks leads to decreased = > >>througoutput
> >>because the wire is utilized fully only at upswings and underutilized on = > >>
> >>downswings (collapsed TCP windows aggravate the effect futher).
> >>
> >>Consequently, the holding capacity of buffers should be sufficient to = > >>
> >>bring the inertia of the system up to the reaction time of the negative = > >>
> >>feedback (this is a simplification, of course). This reaction time is = > >>
> >>about one round-trip time for end-to-end packets.
> >>
> >>In real networks, the RTTs differ for different paths, so some
> >>"characteristic" RTT is used. So, to hold packets until TCPs = > >>slow > >>down a
> >>router needs cRTT * BW bits of buffer memory (where BW is the speed of = > >>the
> >>interface). This rule can be somewhat relaxed if congestion control loop = > >>
> >>is engaged proactively before congestion occured (by using Random Early = > >>
> >>Detection), but not much.
> >>
> >>Even with properly sized buffers, sessions with longer RTTs suffer from = > >>
> >>congestions disproportionately because TCPs on the ends never get enough = > >>
> >>time to recover fully (i.e. to bring windows to large enough size to = > >>
> >>maintain steady stream of packets), while small-RTT sessions recover = > >>
> >>quickly, and, therefore, get bigger share of bandwidth. But I'm
> >>digressing :)
> >>
> >>--vadim
> >>-------------------------------------------------------------------------= > >>------------------------------------------------------- <= > >>/p> > >> > >>

> >> > >>
> >> > >>

 

> >> > >>
> >> > >>
> >> > >>

My question to the group is : what other reasons = > >>not > >>mentioned here can be reasons to buffer in a router?

> >> > >>
> >> > >>
> >> > >>

 

> >> > >>
> >> > >>
> >> > >>

-Paddy 

> >> > >>
> >> > >>
> >> > >>
> >> > >> > >> > >> > >> > >>------=_NextPart_000_0050_01C9F392.7FF50A70-- > >> > >> > > cheers > > jon > > > > ------------------------------ > > _______________________________________________ > end2end-interest mailing list > end2end-interest at postel.org > http://mailman.postel.org/mailman/listinfo/end2end-interest > > > End of end2end-interest Digest, Vol 63, Issue 7 > *********************************************** > From Jon.Crowcroft at cl.cam.ac.uk Wed Jun 24 01:27:08 2009 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Wed, 24 Jun 2009 09:27:08 +0100 Subject: [e2e] RES: Why Buffering? In-Reply-To: <4A40FC9F.9060504@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> Message-ID: In missive <4A40FC9F.9060504 at web.de>, Detlef Bosau typed: >>Jon Crowcroft wrote: >>> the naive argument for bw*delay buffer >>> was that AIMD involves rate halving=20 >>It's not the rate being halved but the congestion window. >>And I'm always curious to see that some people expect some easy=20 >>relationship between rate and sending window / congestion window. Maybe,=20 >>there is one. Than it's certainly not a trivial one. that's why i said "naive" argument:) the rate does halve over a v. long (lotsa rtts) (well, the arrival rate at the bottleneck) - the packet rate thru the bottleneck obviously can't exceed the capacity;) >> >>> so by corollary, just before the rate halved >>> it was twice the bottleneck's capacity >>Yes. That's the common rationale. However, Alexandre is correct here: >>Some router in between has no idea of a path's capacity - at least as >>this cannot be easily described by some "latency bandwidth product" in >>some networks. >>So, the idea of having the "latency bandwidth product" of a path doubled >>as buffer capacity in a router is a nice one for theoretical papers. But >>I'm not quite sure what this should mean for reality: Consider a router >>in Rotterdam which sees both, traffic from Wladiwostok to London, >>Hamburg to New York and from Den Haag to Santiago to Chile. >>Which is "the" latency bandwidth product which should be doubled? >>Surely not a different one for any possible flow. removing the RT dependence in TCP congestion window is a goal of quite a few researchers... >>> if there's lots of flows >>> and the individual flows are all even reasonably=20 >>> unsynchronised in their AIMD phase >>> (very good chance due to random local perturbation >>> =20 >>> (law of large number argument says..) >>> then you are right (well, nick mckeown's right:) >>> if you use some smart queue management and ECN >>> then you're right >>> >>> if most flows stop before reaching their operating point >>> your right=20 >>> (well actually, there's a way to be wrong >>> in a quite ghastly way if you have many sy cnhronised flows >>> in "slow start", heading thru the same bottleneck... >>> but it ought to be quite a rare event - like >>> black swans and mrket meltdowns:-) >>So, after a long list of scenarios where Alexandre is right: Is there a >>scenario, where he is wrong? ;-) probably..... cheers jon From lachlan.andrew at gmail.com Wed Jun 24 10:30:20 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Wed, 24 Jun 2009 10:30:20 -0700 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> Message-ID: 2009/6/24 Jon Crowcroft : > In missive <4A40FC9F.9060504 at web.de>, Detlef Bosau typed: > > ?>>Jon Crowcroft wrote: > ?>>> the naive argument for bw*delay buffer > ?>>> was that AIMD involves rate halving=20 > > ?>>It's not the rate being halved but the congestion window. > > that's why i said "naive" argument:) > > the rate does halve over a v. long (lotsa rtts) (well, the arrival > rate at the bottleneck) - the packet rate thru the bottleneck > obviously can't exceed the capacity;) ACK clocking means that the arrival rate equals the packet rate, over a long time. The arrival rate is halved if the buffer is negligible, but not if the buffer is large. > ?>>Surely not a different one for any possible flow. > > removing the RT dependence in TCP congestion window is a goal of quite > a few researchers... It is an easily achieved goal; there are lots of schemes which make the rate independent of the RTT. However, it is a moot point, since the call for BDP buffers is also easily fixed by small changes to TCP. The real motivation for large buffers is minimising complaints from customers. If a handful of important customers buy cards and use them to send a few TCP (Reno) flows over a long-RTT path, then the cards that those customers buy have to have "big" buffers to get high throughput. That causes Cisco to design for 100ms of buffering. (Not "one RTT", which is ambiguous, as Detlef pointed out.) > ?>>So, after a long list of scenarios where Alexandre is right: Is there a > ?>>scenario, where he is wrong? ;-) > > probably..... Yes, the one I listed earlier in this thread: A small number of long-lived Reno-like flows with large BDPs are bottlenecked at this router. Most routers don't carry this load, but when it was cheap to put in enough memory, it was a common enough case to influence router design, for better or worse (but not 'til death do us part). Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From Jon.Crowcroft at cl.cam.ac.uk Wed Jun 24 15:14:06 2009 From: Jon.Crowcroft at cl.cam.ac.uk (Jon Crowcroft) Date: Wed, 24 Jun 2009 23:14:06 +0100 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> Message-ID: of course if you have a bottleneck rate of R and a max rate somewhere else of R-epsilon and a receive ack rate of R-delta then you can't exceed R by much either so in reality, think of a DSL line with 8Mbps downlin and 384kbps uplink speed (or throttling) and a well-tempered network where the share of a bottlneck is close to to the share of links either side.... then when you have a send window of W, you get W acks, which, packet conservation allowing, lets you send W + W packets but not in the same time - in some longer time (quite a lot more than 1 RTT - if delta or epsilon are small) so then your rate creeps slightly above the output rate of the bottleneck share (indeed it may be another flow has fluctuted below too in this time) so exogeneous effects may mean you dont need BW*RTT at all of buffering... In missive , Lachl an Andrew typed: >>2009/6/24 Jon Crowcroft : >>> In missive <4A40FC9F.9060504 at web.de>, Detlef Bosau typed: >>> >>> =A0>>Jon Crowcroft wrote: >>> =A0>>> the naive argument for bw*delay buffer >>> =A0>>> was that AIMD involves rate halving=3D20 >>> >>> =A0>>It's not the rate being halved but the congestion window. >>> >>> that's why i said "naive" argument:) >>> >>> the rate does halve over a v. long (lotsa rtts) (well, the arrival >>> rate at the bottleneck) - the packet rate thru the bottleneck >>> obviously can't exceed the capacity;) >> >>ACK clocking means that the arrival rate equals the packet rate, over >>a long time. The arrival rate is halved if the buffer is negligible, >>but not if the buffer is large. >> >>> =A0>>Surely not a different one for any possible flow. >>> >>> removing the RT dependence in TCP congestion window is a goal of quite >>> a few researchers... >> >>It is an easily achieved goal; there are lots of schemes which make >>the rate independent of the RTT. However, it is a moot point, since >>the call for BDP buffers is also easily fixed by small changes to TCP. >> >>The real motivation for large buffers is minimising complaints from >>customers. If a handful of important customers buy cards and use them >>to send a few TCP (Reno) flows over a long-RTT path, then the cards >>that those customers buy have to have "big" buffers to get high >>throughput. That causes Cisco to design for 100ms of buffering. (Not >>"one RTT", which is ambiguous, as Detlef pointed out.) >> >>> =A0>>So, after a long list of scenarios where Alexandre is right: Is ther= >>e a >>> =A0>>scenario, where he is wrong? ;-) >>> >>> probably..... >> >>Yes, the one I listed earlier in this thread: A small number of >>long-lived Reno-like flows with large BDPs are bottlenecked at this >>router. Most routers don't carry this load, but when it was cheap to >>put in enough memory, it was a common enough case to influence router >>design, for better or worse (but not 'til death do us part). >> >>Cheers, >>Lachlan >> >>--=20 >>Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) >>Swinburne University of Technology, Melbourne, Australia >> >>Ph +61 3 9214 4837 cheers jon From detlef.bosau at web.de Thu Jun 25 11:54:09 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Thu, 25 Jun 2009 20:54:09 +0200 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> Message-ID: <4A43C7D1.2040905@web.de> Lachlan Andrew wrote: > ACK clocking means that the arrival rate equals the packet rate, over > Hm. First of all, ACK clocking means that packets are sent to the network when other packets leave the network. So, the arrival rate is determined by the ack rate. (Although I'm still a bit reluctant towards the term "rate".) > a long time. The arrival rate is halved if the buffer is negligible, > but not if the buffer is large. > > One problem is that packets don't travel all parts of a path with the same speed. TCP traffic may be bursty, perhaps links are temporarily unavailable. >> >>Surely not a different one for any possible flow. >> >> removing the RT dependence in TCP congestion window is a goal of quite >> a few researchers... >> > > The real motivation for large buffers is minimising complaints from > customers. If a handful of important customers buy cards and use them > to send a few TCP (Reno) flows over a long-RTT path, then the cards > that those customers buy have to have "big" buffers to get high > throughput. That causes Cisco to design for 100ms of buffering. (Not > "one RTT", which is ambiguous, as Detlef pointed out.) > > I once was told that a guy could drastically improve his throughput by enabling window scaling..... On a path from the US to Germany. I'm not quite sure whether the other users of the path were all amused about the one guy who enabled window scaling ;-) But I'm convinced that the RAM manufacturers were enthusiastic to see their chips fully used for the first time....;-) >> >>So, after a long list of scenarios where Alexandre is right: Is there a >> >>scenario, where he is wrong? ;-) >> >> probably..... >> > > Yes, the one I listed earlier in this thread: A small number of > long-lived Reno-like flows with large BDPs are bottlenecked at this > router. Most routers don't carry this load, but when it was cheap to > put in enough memory, it was a common enough case to influence router > design, for better or worse (but not 'til death do us part). > > Say's theorem ;-) Every offer gets his market ;-) -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From lachlan.andrew at gmail.com Thu Jun 25 12:22:59 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Thu, 25 Jun 2009 12:22:59 -0700 Subject: [e2e] RES: Why Buffering? In-Reply-To: <4A43C7D1.2040905@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> Message-ID: 2009/6/25 Detlef Bosau : > Lachlan Andrew wrote: >> >> ACK clocking means that the arrival rate equals the packet rate, over > > Hm. First of all, ACK clocking means that packets are sent to the network > when other packets leave the network. > > So, the arrival rate is determined by the ack rate. Absolutely. If buffers elsewhere in the network are emptying or filling, then the ack rate isn't exactly the rate at which packets leave this buffer. However, Jon seemed to be saying that the arrival rate at the queue could somehow exceed the departure rate in the long term. That can only happen if (a) the window is increasing or (b) a buffer somewhere else is emptying. > (Although I'm still a bit reluctant towards the term "rate".) Why? *Long term* rates are meaningful, even if there is short-term fluctuation in the spacing between adjacent pairs of packets. >> a long time. ?The arrival rate is halved if the buffer is negligible, >> but not if the buffer is large. > > One problem is that packets don't travel all parts of a path with the same > speed. TCP traffic may be bursty, perhaps links are temporarily unavailable. True. Buffers get their name from providing protection against (short timescale) fluctuation in rate. However, on the timescale of TCP's window halving (many RTTs), packets do travel at the same speed over all parts of the path. > I once was told that a guy could drastically improve his throughput by > enabling window scaling..... > On a path from the US to Germany. > > I'm not quite sure whether the other users of the path were all amused about > the one guy who enabled window scaling ;-) Yes, enabling window scaling does allow TCP to behave as it was intended on large BDP paths. If the others weren't amused, they could also configure their systems correctly. The comment suggests that you thought that window scaling somehow makes one user's window larger than TCP intended. It doesn't. It simply allows large windows to be signalled correctly, without imposing an artificially small limit. Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From keshav at uwaterloo.ca Fri Jun 26 08:17:24 2009 From: keshav at uwaterloo.ca (S. Keshav) Date: Fri, 26 Jun 2009 11:17:24 -0400 Subject: [e2e] end2end-interest Digest, Vol 63, Issue 11 In-Reply-To: References: Message-ID: <70908D4C-4189-481F-9B6F-7C35A4364D92@uwaterloo.ca> The whole idea of buffering, as I understand it, is to make sure that *transient* increases in arrival rates do not result in packet losses. If r(t) is the instantaneous arrival rate (packet size divided by inter-packet interval) and s(t) the instantaneous service rate, then a buffer of size B will avert packet loss when integral from t1 to t2 (r(t) - s(t)) < B. If B is 0, then any interval where r(t) is greater than s(t) will result in a packet loss. If you have a fluid system, where a source send packets as packets of infinitesimal size evenly spaced apart, and if routers do not add burstiness, then there is no need for buffering. Indeed, in the classical telephone network, where sources are 64kbps constant bit rate sources and switches do not add burstiness, we need only one sample's worth of buffering, independent of the bandwidth-delay product. A similar approach was proposed by Golestani in 1990 with 'Stop-and-go' queueing, which also decoupled the amount of buffering (equivalent to one 'time slot's worth) from the bandwidth-delay product. http://portal.acm.org/citation.cfm?id=99523 As Jon points out, if exogenous elements conspire to make your packet rate fluid-like, you get the same effect. On Jun 25, 2009, at 3:00 PM, Jon Crowcroft wrote: > so exogeneous effects may mean you dont need BW*RTT at all of > buffering... So, why the need for a bandwidth-delay product buffer rule? The BDP is the window size at *a source* to fully utilize a bottleneck link. If a link is shared, then the sum of windows at the sources must add up to at least the BDP for link saturation. Taking this into account, and the fact that link status is delayed by one RTT, there is a possibility that all sources maximally burst their window's worth of packets synchronously, which is a rough upper bound on s(t). With one BDP worth of buffering, there will be no packet loss even in this situation. So, it's a good engineering rule of thumb. In reality: (a) RTTs are not the same for sources (b) the sum of source window sizes often exceeds the BDP and (c) the worst-case synchronized burstiness rarely happens. These factors (hopefully) balance themselves, so that the BDP rule seemed reasonable. Of course, we have seen considerable work showing that in the network 'core' the regime is closer to fluid than bursty, so that we can probably do with far less. Hope this helps, keshav From dpreed at reed.com Fri Jun 26 10:35:12 2009 From: dpreed at reed.com (David P. Reed) Date: Fri, 26 Jun 2009 13:35:12 -0400 Subject: [e2e] end2end-interest Digest, Vol 63, Issue 11 In-Reply-To: <70908D4C-4189-481F-9B6F-7C35A4364D92@uwaterloo.ca> References: <70908D4C-4189-481F-9B6F-7C35A4364D92@uwaterloo.ca> Message-ID: <4A4506D0.3030709@reed.com> If the bottleneck router has too much buffering, and there are at least some users who are infinite data sources (read big FTP), then all users will suffer congestion at the bottleneck router proportional to the buffer size, *even though* the link will be "fully utilized" and therefore "economically maximized". This is the "end to end" list, not the "link maximum utilization" list. And a large percentage of end-to-end application requirements depend on keeping latency on bottleneck links very low, in order to make endpoint apps responsive - in their UIs, in the control loops that respond quickly and smoothly to traffic load changes, etc. Analyses that focus 100% on maximizing static throughput and utilization leave out some of the most important things. It's like desiging cars to only work well as fuel-injected dragsters that run on Bonneville salt flats. Nice hobby, but commercially irrelevant. On 06/26/2009 11:17 AM, S. Keshav wrote: > The whole idea of buffering, as I understand it, is to make sure that > *transient* increases in arrival rates do not result in packet losses. > If r(t) is the instantaneous arrival rate (packet size divided by > inter-packet interval) and s(t) the instantaneous service rate, then a > buffer of size B will avert packet loss when integral from t1 to t2 > (r(t) - s(t)) < B. If B is 0, then any interval where r(t) is greater > than s(t) will result in a packet loss. > > If you have a fluid system, where a source send packets as packets of > infinitesimal size evenly spaced apart, and if routers do not add > burstiness, then there is no need for buffering. Indeed, in the > classical telephone network, where sources are 64kbps constant bit > rate sources and switches do not add burstiness, we need only one > sample's worth of buffering, independent of the bandwidth-delay > product. A similar approach was proposed by Golestani in 1990 with > 'Stop-and-go' queueing, which also decoupled the amount of buffering > (equivalent to one 'time slot's worth) from the bandwidth-delay > product. http://portal.acm.org/citation.cfm?id=99523 > > As Jon points out, if exogenous elements conspire to make your packet > rate fluid-like, you get the same effect. > > On Jun 25, 2009, at 3:00 PM, Jon Crowcroft wrote: > >> so exogeneous effects may mean you dont need BW*RTT at all of >> buffering... > > So, why the need for a bandwidth-delay product buffer rule? The BDP is > the window size at *a source* to fully utilize a bottleneck link. If a > link is shared, then the sum of windows at the sources must add up to > at least the BDP for link saturation. > > Taking this into account, and the fact that link status is delayed by > one RTT, there is a possibility that all sources maximally burst their > window's worth of packets synchronously, which is a rough upper bound > on s(t). With one BDP worth of buffering, there will be no packet loss > even in this situation. So, it's a good engineering rule of thumb. > > In reality: (a) RTTs are not the same for sources (b) the sum of > source window sizes often exceeds the BDP and (c) the worst-case > synchronized burstiness rarely happens. These factors (hopefully) > balance themselves, so that the BDP rule seemed reasonable. Of course, > we have seen considerable work showing that in the network 'core' the > regime is closer to fluid than bursty, so that we can probably do with > far less. > > Hope this helps, > > keshav > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.postel.org/pipermail/end2end-interest/attachments/20090626/6962e3ed/attachment.html From lachlan.andrew at gmail.com Fri Jun 26 12:16:11 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Fri, 26 Jun 2009 12:16:11 -0700 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> Message-ID: Greetings Keshav, 2009/6/26 S. Keshav : > The whole idea of buffering, as I understand it, is to make sure that > *transient* increases in arrival rates do not result in packet losses. That was the original motivation, but I believe the current motivation is TCP throughput. If we fix TCP, then buffers can go back to their original role of smoothing transients, and I agree with your statements. > So, why the need for a bandwidth-delay product buffer rule? The BDP is the > window size at *a source* to fully utilize a bottleneck link. If a link is > shared, then the sum of windows at the sources must add up to at least the > BDP for link saturation. > > Taking this into account, and the fact that link status is delayed by one > RTT, there is a possibility that all sources maximally burst their window's > worth of packets synchronously, which is a rough upper bound on s(t). With > one BDP worth of buffering, there will be no packet loss even in this ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > situation. So, it's a good engineering rule of thumb. No buffer is big enough to stop a long TCP(Reno) flow losing packets, because it will increase its window until it does. The reason for one BDP is not to prevent loss, but to make sure that throughput is maintained when loss occurs. Of course, designers have gone way overboard with large buffers (especially in modems for low rate access links, as David pointed out). Still, if it weren't for TCP's loss-probing, then huge buffers would be harmless. Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From keshav at uwaterloo.ca Fri Jun 26 12:37:02 2009 From: keshav at uwaterloo.ca (S. Keshav) Date: Fri, 26 Jun 2009 15:37:02 -0400 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> Message-ID: <0E3DC2FB-C1EC-47E6-B277-2DDB733E21B2@uwaterloo.ca> On Jun 26, 2009, at 3:16 PM, Lachlan Andrew wrote: > No buffer is big enough to stop a long TCP(Reno) flow losing packets, > because it will increase its window until it does. I agree. What I meant to say was this: ideally, each source chooses a static window such that the sum of these window sizes is the BDP. In this case, the link is fully used and there is no queueing. Of course, the sources don't know these values, so they are forced to increase the window to probe the available capacity, which pushes the buffers towards fullness and the packets towards loss. In this situation, having large buffers is no help at all - it only delays the inevitable at the cost of increased delay to all. All I was trying to do was to explain, as I knew best, the thought process that leads to the one-BDP-buffer rule of thumb cheers keshav From detlef.bosau at web.de Sat Jun 27 11:44:37 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 27 Jun 2009 20:44:37 +0200 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> Message-ID: <4A466895.4010401@web.de> Lachlan Andrew wrote: > Absolutely. If buffers elsewhere in the network are emptying or > filling, then the ack rate isn't exactly the rate at which packets > leave this buffer. However, Jon seemed to be saying that the arrival > rate at the queue could somehow exceed the departure rate in the long > term. That can only happen if (a) the window is increasing or (b) a > buffer somewhere else is emptying. > > That's exactly the reason why I'm not comfortable with the term "rate". "Rate", as you say yourself, describes a long term behaviour. However, what you're talking about in the quoted paragraph is short term behaviour. > Why? *Long term* rates are meaningful, even if there is short-term > fluctuation in the spacing between adjacent pairs of packets. > Not only in the spacing between adjacent pairs of packets. I'm still thinking of WWAN. And in WWAN, even the time to convey a packet from a base station to a mobile or vice versa is subject to short-term fluctuation. >> One problem is that packets don't travel all parts of a path with the same >> speed. TCP traffic may be bursty, perhaps links are temporarily unavailable. >> > > True. Buffers get their name from providing protection against (short > timescale) fluctuation in rate. Is this their main objective? As Jon pointed out, buffers should compensate asynchrounicity and "somehow" care for work compensation. That's sometimes an interesting discussion between CS and EE guys. EE guys, particularly telco-guys quite often think of schedulers, while CS guys rarely use explicit scheduling. Think of TCP for instance, where we typically don't have schedulers at all. With WWAN as an important exception: In WWAN, there _are_ schedulers at the base stations / access points. That's one of the "collisions" in "EE thinking" and "CS thinking". >> I once was told that a guy could drastically improve his throughput by >> enabling window scaling..... >> On a path from the US to Germany. >> >> I'm not quite sure whether the other users of the path were all amused about >> the one guy who enabled window scaling ;-) >> > > Yes, enabling window scaling does allow TCP to behave as it was > intended on large BDP paths. If the others weren't amused, they could > also configure their systems correctly. > Which would leave the original user disappointed because he's not that smart as he thought he was ;-) However: Cui bono? If the only consequence of window scaling is an end of the commercial crisis, at least for DRAM manufactures, at the cost of extremely long round trip times, we should rather avoid it ;-) > The comment suggests that you thought that window scaling somehow > makes one user's window larger than TCP intended. No way. > It doesn't. It > simply allows large windows to be signalled correctly, without > imposing an artificially small limit. > > The problem is: Buffering shall provide for work conservation, as Jon pointed out. As soon as buffers "overcompensate" idle times and don't avoid idle times but introduce latency by themselves, the design is not really helpful. -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de From detlef.bosau at web.de Sat Jun 27 12:05:03 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sat, 27 Jun 2009 21:05:03 +0200 Subject: [e2e] end2end-interest Digest, Vol 63, Issue 11 In-Reply-To: <70908D4C-4189-481F-9B6F-7C35A4364D92@uwaterloo.ca> References: <70908D4C-4189-481F-9B6F-7C35A4364D92@uwaterloo.ca> Message-ID: <4A466D5F.8050906@web.de> S. Keshav wrote: > If you have a fluid system, where a source send packets as packets of > infinitesimal size evenly spaced apart, and if routers do not add > burstiness, then there is no need for buffering. Unfortunately, packet switching networks are anything but a fluid system. Although there is a huge amount of literature which uses fluid flow models for TCP/IP. > Indeed, in the classical telephone network, where sources are 64kbps > constant bit rate sources and switches do not add burstiness, we need > only one sample's worth of buffering, independent of the > bandwidth-delay product. That's exactly the conflict of paradigms I mentioned in my other message today ;-) EE: fluid flow like, with scheduling, if possible "quasi synchronous". CS: bursts are possible, there is no scheduling, the network may be as asynchronous as it could be.... -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3364 bytes Desc: S/MIME Cryptographic Signature Url : http://mailman.postel.org/pipermail/end2end-interest/attachments/20090627/fd9340a7/smime.bin From lachlan.andrew at gmail.com Sat Jun 27 13:44:08 2009 From: lachlan.andrew at gmail.com (Lachlan Andrew) Date: Sun, 28 Jun 2009 06:44:08 +1000 Subject: [e2e] RES: Why Buffering? In-Reply-To: <4A466895.4010401@web.de> References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> <4A466895.4010401@web.de> Message-ID: Greetings Detlef, 2009/6/28 Detlef Bosau : > Lachlan Andrew wrote: >> >> Jon seemed to be saying that the arrival >> rate at the queue could somehow exceed the departure rate in the long >> term. ?That can only happen if (a) the window is increasing or (b) a >> buffer somewhere else is emptying. > > That's exactly the reason why I'm not comfortable with the term "rate". > "Rate", as you say yourself, describes a long term behaviour. > However, what you're talking about in the quoted paragraph is short term > behaviour. The rate of any (realisation of a) discrete process is well defined over any interval (not necessarily infinite). When I said that "short term" rates are not defined, I meant that we can't talk about the rate "at a point in time". Also, it is not generally helpful to think of "rate" on a timescale less than the average inter-event time. (The rate over intervals on this scale is typically either 0 or very large. It is well defined, just not useful.) If you think of what I said in terms of "number of events divided by time interval", I think you'll find it makes sense. If not, feel free to point out an error. >> Why? ?*Long term* rates are meaningful, even if there is short-term >> fluctuation in the spacing between adjacent pairs of packets. > > Not only in the spacing between adjacent pairs of packets. > > I'm still thinking of WWAN. And in WWAN, even the time to convey a packet > from a base station to a mobile or vice versa is subject to short-term > fluctuation. In that case, we need to distinguish between the rate of *starting* to send packets and the rate of *completing* finishing packets. However, in the "long term" the two will still be roughly equal, where "long term" means "a time much longer than the time to send an individual packet". If a packet can take up to 3 seconds to send, then the two rates will roughly agree on timescales of 30s or more. >>> One problem is that packets don't travel all parts of a path with the >>> same speed. TCP traffic may be bursty, perhaps links are temporarily >>> unavailable. >> >> True. ?Buffers get their name from providing protection against (short >> timescale) fluctuation in rate. > > Is this their main objective? It was. Buffers in different places have different purposes. I've said many times that I think the current main objective of buffers on WAN interfaces of routers is to achieve high TCP throughput. (Saying it again doesn't make it more or less right, but nothing in this thread seems a compelling argument against it.) >>> I once was told that a guy could drastically improve his throughput by >>> enabling window scaling..... >>> On a path from the US to Germany. >>> >>> I'm not quite sure whether the other users of the path were all amused >>> about >>> the one guy who enabled window scaling ;-) >> >> Yes, enabling window scaling does allow TCP to behave as it was >> intended on large BDP paths. ?If the others weren't amused, they could >> also configure their systems correctly. > > However: Cui bono? If the only consequence of window scaling is an end of > the commercial crisis, at least for DRAM manufactures, at the cost of > extremely long round trip times, we should rather avoid it ;-) But that isn't all it does. On a high BDP link, if you don't use window scaling a single flow can get much less throughput than the 75% of capacity which is possible with window scaling and without significant buffering. > The problem is: Buffering shall provide for work conservation, as Jon > pointed out. As soon as buffers "overcompensate" idle times and don't avoid > idle times but introduce latency by themselves, the design is not really > helpful. True. A buffer which never empties is too big (for that situation). Cheers, Lachlan -- Lachlan Andrew Centre for Advanced Internet Architectures (CAIA) Swinburne University of Technology, Melbourne, Australia Ph +61 3 9214 4837 From detlef.bosau at web.de Sat Jun 27 15:42:03 2009 From: detlef.bosau at web.de (Detlef Bosau) Date: Sun, 28 Jun 2009 00:42:03 +0200 Subject: [e2e] RES: Why Buffering? In-Reply-To: References: <2ff1f08a0906191220x4de051d1w18e5285d3ef6abf5@mail.gmail.com> <4A40FC9F.9060504@web.de> <4A43C7D1.2040905@web.de> <4A466895.4010401@web.de> Message-ID: <4A46A03B.3060903@web.de> Hi Lachlan, Lachlan Andrew wrote: > If you think of what I said in terms of "number of events divided by > time interval", I think you'll find it makes sense. If not, feel free > to point out an error. > Perhaps, the difficulty is less the definition of the term "rate" but more the knowledge of an actual rate. I just had a _very_ first glance at the Golestani paper, Keshav pointed to. The term (r,T) smoothness reminds me of a "latency throughput product" for a certain period of time. And I think, that's the vicious circle, we may be trapped in: How large is a "latency throughput product" even for a well defined period of time? Basically, we define the rate as "customers served / service time". Fine. So, if 5 customers are served within 1 second, we have rate of 5 customers/s. The problem is: How do we know how many customers are served within this period of time? Particularly as customers are packets, and therefore the Declaration of Human Rights does not apply? I.e.: Packets may be killed, the death penalty is neither abolished nor does it depend on any judgement. Packets may be made to stay in some buffer in some intermediate system for an unknown time. Packets may get crippled or corrupted, o.k., L2 may care for some kind of Euthanasia then.... After my first attempts to work with wireless networks, it took me years to understand that exactly this is the problem: When we send a number of packets to a wireless link, we have actually no idea how many packets will see the receiver. Basically, the term "rate" is just another twist of "customers served" - btw: do we mean "served, no matter how"? Or do we mean "successfully served"? When you remember my paper submission this year, which was rejected, this was exactly the point I tried to address. From a user's perspective, I want to know the number of successfully served packets within a certain time. From the congestion control perspective, I want to know how much system capacity is occupied, or available respectively. Be it in computer networks or be it in some kind of public transport system. (Stop and go queuing reminds me of a typically form of "Swabian Train Queuing." When you go by train here in the city of Stuttgart, every few moments the train stops and it is said by a speaker: "Dear Passengers! Unfortunately, the route section before us is busy at the moment. We'll continue our travel shortly.") (There's some rumor of a British alternative, sometimes called: "London collision queuing", but it got only limited acceptance because of some minor shortcomings... ;-)) If the departure rate of our packets / trains are known and there are no problems along the route, things are quite easy. However, when we don't know in advance how many packets will be served, even ex post we generally don't know at link layer whether a packet has been _successfully_ served, it is in fact a moot discussion whether we ask for a rate or a number of served packets. Neither is known. And you're of course right: This does not depend on the length of the period of time we talk about. In this context, it is extremely important (and to the best of my knowledge, this difference is hardly being made at the moment) to make a difference between service at all and _successful_ service (i.e. intact delivery). Is this modeled in literature, i.e. does the TCP literature model (actually unknown!) loss processes on a line? In a nursing home, there is no difference: No matter whether an inhabitant moves away or passes away, the apartment is available afterwards. However, the local relocation company and the local undertaker are likely to see different arrival rates here... Formally spoken: There are several kinds of "death processes" here or, differently put, several kinds of servers. Some packets are delivered without errors, others are dropped, others are delivered partially correct - with acceptable errors. The latter is no problem for telcos, they take a SNMP attitude here and define noise and the like to be the problem of the customers. Formally spoken: For voice transfer with cell phones, there is no CRC check. Either the listener understands what I'm talking about - or it's bad luck. So, a telco is like a nursing home then. For packet transfer, the situation is different. And perhaps the most nasty thing here are partially correct packets. Apart from UDP Lite, we hardly talked about this issue in TCP/IP. Just a thought which comes to my mind at the moment: There is of course a difference between _congestion_ control and _flow_ control depending on whether a packet only leaves the path or successfully enqueues for being delivered to the application. So, the scenario I'm thinking about are lossy links and links where the service time is unknown or hardly to predict. Detlef > > >>> Why? *Long term* rates are meaningful, even if there is short-term >>> fluctuation in the spacing between adjacent pairs of packets. >>> >> Not only in the spacing between adjacent pairs of packets. >> >> I'm still thinking of WWAN. And in WWAN, even the time to convey a packet >> from a base station to a mobile or vice versa is subject to short-term >> fluctuation. >> > > In that case, we need to distinguish between the rate of *starting* to > send packets and the rate of *completing* finishing packets. However, > in the "long term" the two will still be roughly equal, where "long > term" means "a time much longer than the time to send an individual > packet". If a packet can take up to 3 seconds to send, then the two > rates will roughly agree on timescales of 30s or more. > > >>>> One problem is that packets don't travel all parts of a path with the >>>> same speed. TCP traffic may be bursty, perhaps links are temporarily >>>> unavailable. >>>> >>> True. Buffers get their name from providing protection against (short >>> timescale) fluctuation in rate. >>> >> Is this their main objective? >> > > It was. Buffers in different places have different purposes. I've > said many times that I think the current main objective of buffers on > WAN interfaces of routers is to achieve high TCP throughput. (Saying > it again doesn't make it more or less right, but nothing in this > thread seems a compelling argument against it.) > > >>>> I once was told that a guy could drastically improve his throughput by >>>> enabling window scaling..... >>>> On a path from the US to Germany. >>>> >>>> I'm not quite sure whether the other users of the path were all amused >>>> about >>>> the one guy who enabled window scaling ;-) >>>> >>> Yes, enabling window scaling does allow TCP to behave as it was >>> intended on large BDP paths. If the others weren't amused, they could >>> also configure their systems correctly. >>> >> However: Cui bono? If the only consequence of window scaling is an end of >> the commercial crisis, at least for DRAM manufactures, at the cost of >> extremely long round trip times, we should rather avoid it ;-) >> > > But that isn't all it does. On a high BDP link, if you don't use > window scaling a single flow can get much less throughput than the 75% > of capacity which is possible with window scaling and without > significant buffering. > > >> The problem is: Buffering shall provide for work conservation, as Jon >> pointed out. As soon as buffers "overcompensate" idle times and don't avoid >> idle times but introduce latency by themselves, the design is not really >> helpful. >> > > True. A buffer which never empties is too big (for that situation). > > Cheers, > Lachlan > > -- Detlef Bosau Galileistra?e 30 70565 Stuttgart phone: +49 711 5208031 mobile: +49 172 6819937 skype: detlef.bosau ICQ: 566129673 http://detlef.bosau at web.de